Skip to content
Snippets Groups Projects
  1. Aug 30, 2012
  2. Aug 22, 2012
    • Dimitris Aragiorgis's avatar
    • Dimitris Aragiorgis's avatar
    • Dimitris Aragiorgis's avatar
      Merge branch 'stable-esi' · c54ec0dc
      Dimitris Aragiorgis authored
      
      Conflicts:
      	Makefile.am
      	lib/cli.py
      	lib/client/gnt_instance.py
      	lib/cmdlib.py
      	lib/constants.py
      	lib/opcodes.py
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      c54ec0dc
    • Constantinos Venetsanopoulos's avatar
      Add external storage interface related man pages · 7c88c7af
      Constantinos Venetsanopoulos authored
      
       * ganeti-extstorage-interface man page
       * gnt-storage man page
      
      Signed-off-by: default avatarConstantinos Venetsanopoulos <cven@grnet.gr>
      7c88c7af
    • Constantinos Venetsanopoulos's avatar
      11d36c2e
    • Constantinos Venetsanopoulos's avatar
      Add the gnt-storage client · 9bcef16f
      Constantinos Venetsanopoulos authored
      
      Add a new client called 'gnt-storage'.
      The client interacts with the ExtStorage interface, similarly to
      the way gnt-os interacts with the OS interface.
      
      For now, only two commands are supported: 'info' and 'diagnose'.
      
      'diagnose' calculates the node status of each provider on each node,
      similarly to gnt-os diagnose. Furthermore, for every provider, it
      calculates it's nodegroup validity for each nodegroup. This is done
      inside the LU and not the client (marked as 'TODO' for the  global
      validity of gnt-os diagnose).
      
      In the future, gnt-storage can be used to manage storage pools,
      or even be extended to diagnose other storage types supported by
      Ganeti, such as lvm, drbd (INT_MIRROR) or rbd (EXT_MIRROR).
      
      Signed-off-by: default avatarConstantinos Venetsanopoulos <cven@grnet.gr>
      9bcef16f
    • Constantinos Venetsanopoulos's avatar
    • Constantinos Venetsanopoulos's avatar
      Multiple ExtStorage Providers and ext-params · e74f6291
      Constantinos Venetsanopoulos authored
      
      Add support for passing parameters to the ext template (ext-params).
      Take advantage of disk-params, that don't seem to make much sense in
      this template (ExtStorage Providers are not predefined and we don't
      know their needs) and use them to pass the ext-params dynamically to
      the template.
      
      ext-params are correlated with gnt-os-interface's os-params.
      All ext-params are exported to the ExtStorage Provider through it's
      environment, with variables prefixed with 'EXTP_' (similarly to the
      OS interface's 'OSP_' params).
      
      ext-params are passed through the --disk option. If the disk template
      is of type `ext' during instance add, then any additional options that
      are not in IDISK_PARAMS given to --disk are considered ext-params
      e.g.:
      
       gnt-instance add -t ext --disk=0:size=2G,param1=value1,param2=value2
      
      Finally, we introduce a new IDISK_PARAM called IDISK_PROVIDER, that is
      mandatory for template `ext' and is used to select the desired
      ExtStorage Provider. This parameter is not valid for other template
      types.
      
      The IDISK_PROVIDER parameter becomes the first element of the
      disk's unique_id tuple e.g.:
      
       unique_id = ('sample_provider1', 'UUID.ext.diskX')
      
      Example selecting different ExtStorage Providers for each disk and
      passing different ext-params to them:
      
       -t ext --disk=0:size=2G,provider=sample_provider1,param1=value1
              --disk=1:size=3G,provider=sample_provider2,param2=value2
      
      Signed-off-by: default avatarConstantinos Venetsanopoulos <cven@grnet.gr>
      e74f6291
    • Constantinos Venetsanopoulos's avatar
      Implement the External Storage Interface · 868eab67
      Constantinos Venetsanopoulos authored
      
      With this commit we introduce the External Storage Interface
      to Ganeti, abbreviated: ExtStorage Interface.
      
      The ExtStorage Interface provides Ganeti with the ability to interact
      with externally connected shared storage pools, visible by all
      VM-capable nodes. This means that Ganeti is able to handle VM disks
      that reside inside a NAS/SAN or any distributed block storage provider.
      
      The ExtStorage Interface provides a clear API, heavily inspired by the
      gnt-os-interface API, that can be used by storage vendors or sysadmins
      to write simple ExtStorage Providers (correlated to gnt-os-interface's
      OS Definitions). Those Providers will glue externally attached shared
      storage with Ganeti, without the need of preprovisioned block devices
      on Ganeti VM-capable nodes as confined be the current `blockdev' disk
      template.
      
      To do so, we implement a new disk template called `ext' (of type
      DTS_EXT_MIRROR) that passes control to externally provided scripts
      (the ExtStorage Provider) for the template's basic functions:
      
       create / attach / detach / remove / grow
      
      The scripts reside under ES_SEARCH_PATH (correlated to OS_SEARCH_PATH)
      and only one ExtStorage Provider is supported called `ext'.
      
      The disk's logical id is the tuple ('ext', UUID.ext.diskX), where UUID
      is generated as in disk template `plain' and X is the disk's index.
      
      Signed-off-by: default avatarConstantinos Venetsanopoulos <cven@grnet.gr>
      868eab67
    • Constantinos Venetsanopoulos's avatar
      Update the shared storage design document · f9a1f3eb
      Constantinos Venetsanopoulos authored
      
      Update the shared storage design document to reflect the current
      changes, after the implementation of the ExtStorage interface.
      
      Signed-off-by: default avatarConstantinos Venetsanopoulos <cven@grnet.gr>
      f9a1f3eb
    • Dimitris Aragiorgis's avatar
      Merge branch 'stable-hotplug' · a5e58089
      Dimitris Aragiorgis authored
      
      Conflicts:
      	lib/cli.py
      	lib/client/gnt_instance.py
      	lib/cmdlib.py
      	lib/objects.py
      	lib/opcodes.py
      	lib/rpc.py
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      a5e58089
    • Dimitris Aragiorgis's avatar
      Hotplug: client support · 50fab955
      Dimitris Aragiorgis authored
      
      Add --hotplug option.
      Only used by OpInstanceCreate()/OpInstanceSetParams.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      50fab955
    • Dimitris Aragiorgis's avatar
      Hotplug: cmdlib support · 9ac60360
      Dimitris Aragiorgis authored
      
      Hotplugging is done by functions invoked by ApplyContainerMods().  In
      order hotplugging to take place a --hotplug option must be passed.
      Hotplug can enabled for a VM if --hotplug is passed during
      gnt-instance add (hotplug_info will be added).
      
      In order initial devices of a VM to be hotpluggable hotplug_info must be
      early available. So _GetPCIInfo() gets either lu.hotplug_info in case of
      LUInstanceCreate() or invokes lu.cfg.GetPCIInfo() in case of
      LUInstanceSetParams(). To return a pci slot into the pool
      lu.cfg.UpdatePCIInfo() must be invoked.
      
      If no --hotplug is passed during gnt-instance modify --net add the nic
      won't get a pci value and will be placed automaticaly in a free pci slot
      after reboot. This means that is not going to be hotpluggable for the
      rest of the instance's life.
      
      To keep pci pool consistent we do not let a device that has been
      hotplugged (i.e. has pci value) to be removed without --hotplug.
      
      Before hotplugging a Disk it must be assembled. During
      LuInstanceSetParams() newly created disks are not added to the instance
      so _ExpandCheckDisks() in _AssembleInstanceDisks() will fail. So we make
      this check optional only for this case.
      
      In order to remove a disk (with blockdev_remove) it must be shutdown. So
      after unplugging the disk _ShutdownDiskInstanceDisks() must be invoked.
      
      Everything else is just like nic hotplugging.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      9ac60360
    • Dimitris Aragiorgis's avatar
      Hoplug: rpc support · 5ed75b63
      Dimitris Aragiorgis authored
      
      Introduce new RPCs that eventually invoke hypervisor specific
      hotplug functions.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      5ed75b63
    • Dimitris Aragiorgis's avatar
      Hotplug: KVM support · 1642a1d0
      Dimitris Aragiorgis authored
      
      KVM >= 1.0 is needed.
      
      Introduce new methods: HotAddNic/HotDelNic
      
      Both qemu monitor interface for hotplugging. If a device has
      hotplug info (idx, pci) then:
       - it gets named after it (e.g. virtio-net-pci.%d % idx ).
       - it gets placed to the pci slot (hex(pci)) of bus 0.
      If no hotplug info exist or kvm version is old then no hotplug takes
      place and modifications will take place after reboot:
       - devices will get traditionaly named
       - and placed automaticaly in empty pci slots
      
      NIC hotplug:
       - open a tap and get its file descriptor.
       - pass fd with SCM rights (using python-fdsend) via monitor socket
       - create netdev and device with proper qemu interface
      
      Disk hotplug:
       - create drive with drive%d % idx name
       - create device virtio-blk-pci.%d % idx
      
      In order to migrate a VM that has devices hotplugged, an identical
      VM should be booted with -incoming. To this end every time a hotplug
      takes place runtime file must be updated.
      
      For disk hotplug support we removeds disk from kvm_cmd in
      runtime file add added them separately just like nics. So code building
      disk related options has been moved out from _GenerateKVMRuntime()
      and added to _ExecuteKVMRuntime().
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      1642a1d0
    • Dimitris Aragiorgis's avatar
      Hoplug: objects modifications/additions · 41a1df34
      Dimitris Aragiorgis authored
      
      Introduce new HotplugInfo object as part of Instance object.
      
      hotplug_info { 'nics': 0, 'disks': 0, 'pci_pool': [16, 17, 18] }
      
      nics/disks is an index used for device identification and naming.
      Everytime a new device (disk/nic) is hotadded these values get incremented.
      
      pci_pool is a list of pci slots where a device can be placed. Every
      time a device gets hot-plugged/unplugged a pci slot gets popped/appended.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      41a1df34
    • Dimitris Aragiorgis's avatar
      Add tags in network objects · a8740a25
      Dimitris Aragiorgis authored
      
      Support:
       - gnt-network add --tags.
       - gnt-network list-tags/add-tags/remove-tags/.
       - gnt-network list -o +tags
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      a8740a25
    • Dimitris Aragiorgis's avatar
      5fb92325
    • Dimitris Aragiorgis's avatar
      66c844a3
    • Dimitris Aragiorgis's avatar
      Pass netinfo in rpcs · 8c4771b1
      Dimitris Aragiorgis authored
      
      If a nic has a network field then encapsulate a network object in
      netinfo slot for every rpc. This is needed to pass network info to
      scripts managing nics (kvm-vif-bridge).
      
      Introduce _BuildNetworkEnv()
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      8c4771b1
    • Dimitris Aragiorgis's avatar
      cfgupgrade for networks · db465855
      Dimitris Aragiorgis authored
      
      Add networks slot in config data.
      Add networks slot in nodegroups.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      db465855
    • Dimitris Aragiorgis's avatar
      Use network's mac prefix · 24ea7a54
      Dimitris Aragiorgis authored
      
      Modify existing GenerateMAC so that it takes network as an argument.
      
      Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix
      that chooses the prefix (network's or cluster's) and wraps
      _GenerateMACSuffix that generates the three remaining bytes.
      
      Check mac prefix validity when adding/modifing a network.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      24ea7a54
    • Dimitris Aragiorgis's avatar
    • Dimitris Aragiorgis's avatar
      Add gnt-network man page · cca3867d
      Dimitris Aragiorgis authored
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      cca3867d
    • Dimitris Aragiorgis's avatar
      Rapi support for networks · a833cd23
      Dimitris Aragiorgis authored
      
      Support:
       - GetNetwork(s)
       - CreateNetwork
       - ConnectNetwork
       - DisconnectNetwork
       - RemoveNetwork
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      a833cd23
    • Dimitris Aragiorgis's avatar
      ss_conf support for networks · c18bf8b2
      Dimitris Aragiorgis authored
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      c18bf8b2
    • Dimitris Aragiorgis's avatar
      Introduce client support for networks · f5422b71
      Dimitris Aragiorgis authored
      
      gnt-network is used to manipulate and handle networks that
      currently provides the following operations:
      
       * Add a new network:
         gnt-network add --network=1.2.3.0/28 --gateway=1.2.3.1
                         --add-reserved-ips=1.2.3.4,1.2.3.5 testnet
      
       * Remove an existing network:
         gnt-network remove testnet
      
       * Modify an existing network:
         gnt-network modify --gateway=1.2.3.6
                            --network-type=private
                            --network6=2001:648::/64
                            --gateway6=none testnet
                            --add-reserved-ips=1.2.3.10,1.2.3.10,
                            --remove-reserved-ips=1.2.3.20
                            testnet
      
       * Connect an existing network to a nodegroup:
         gnt-network connect testnet default bridged br100
         gnt-network connect testnet <nodegroup> <mode> <link>
         (pass all for <nodegroup> to connect to all nodegroups)
      
       * Disconnect an existing network from a nodegroup:
         gnt-network disconnect testnet <nodegroup>
         (pass all for <nodegroup> to disconnect from all nodegroups)
      
       * List available networks:
         gnt-network list
      
       * Show network info:
         gnt-network info [testnet]
      
      Introduce new option NOCONFLICTSCHECK_OPT for not checking for
      conflicting IPs. Using this might cause data inconsistency.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      f5422b71
    • Dimitris Aragiorgis's avatar
      Modify instance client to support networks · 24a6bd31
      Dimitris Aragiorgis authored
      
      Add --no-conflicts-check option.
      Modify instance query methods to return nic's network info.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      24a6bd31
    • Dimitris Aragiorgis's avatar
      Peripheral changes for networks · d45473fb
      Dimitris Aragiorgis authored
      
      Specifically:
       * export a new environment variable for the gnt-os-interface
       * export a new environment variable for KVM specific scripts
       * ovf support for networks
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      d45473fb
    • Dimitris Aragiorgis's avatar
      Modify LUInstanceSetParams() to support networks · 100c6d1a
      Dimitris Aragiorgis authored
      
      Keep backwards compatibility just like LUInstanceCreate() and
      support network parameter for NIC object.
      
      Add ec_id argument in Update() to be able to commit temporary
      IPs that were previously reserved.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      100c6d1a
    • Dimitris Aragiorgis's avatar
      Modify RemoveInstance() to support networks · a11b8875
      Dimitris Aragiorgis authored
      
      Release any IPs held the instance before actually removing.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      a11b8875
    • Dimitris Aragiorgis's avatar
      Modify LUInstanceCreate to support networks · f70bcabc
      Dimitris Aragiorgis authored
      
      Implement backend support, to export the IP pool management
      functionality to the clients. When the new NIC parameter 'network'
      is given, the ippool management system is triggered.
      If a NIC belongs to a network, it inherits the netparams
      (mode, link) as its nicparams. If a network is requested, then
      no 'mode' or 'link' should be specified. Backwards compatibility
      is preserved, if 'network' parameter is omitted.
      
      Examples for IP pool management system:
      
      --net 0:network=net1
        for an IPless NIC in a network
      
      --net 0:ip=pool,network=net1
        for automatically generated IP inside the network
      
      --net 0:ip=1.2.3.4,network=net1
        for specific IP inside the network
      
      --net 0:network=none
        for IPless NIC with default nicparams
      
      Examples for traditional assignment:
      
      --net 0:ip=1.2.3.4,link=br100,mode=bridged
      --net 0:ip=none,link=rt2,mode=routed
      --net 0
      
      Check for conflicting IPs to ensure IP uniqueness inside nodegroups.
      
      IAllocator should be extended to become network aware. If an instance
      has a NIC inside a network, then IAllocator must pick a node that its
      nodegroup is already connected with that network.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      f70bcabc
    • Dimitris Aragiorgis's avatar
      Config methods for reserving/releasing IPs · 0ca10882
      Dimitris Aragiorgis authored
      
      Use TemporaryReservationManager for IP handling to ensure
      config data consistency.
      
      Implement:
       - methods for reserving/releasing IPs
       - _UnlockedCommitTemporaryIps()
       - CheckIPInNodeGroup() used for conflicting IPs
       - GetGroupNetParams() used for inheriting netparams.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      0ca10882
    • Dimitris Aragiorgis's avatar
      Implement network/nodegroup mappings · 9870dce9
      Dimitris Aragiorgis authored
      
      LUNetworkConnect() connects a network to a nodegroup and defines
      its netparams (mode and link). Specifically:
       * Check if network already exists
       * Check if netparams are valid
       * Check if already connected to the same nodegroup
       * Produce errors if any conflicting IPs found
       * Update the 'networks' slot in NodeGroup config object
         with the network UUID and netparams requested. E.g.
         nodegroup_uuid {
           networks {
             network_uuid: {mode: 'bridged', link: 'br100'}
           }
         }
      
      LUNetworkDisconnect() unmaps a network from a nodegroup:
       * Produce errors if an instance has a NIC with IP inside the network
       * Remove corresponding entry in 'networks' slot of NodeGroup object
      
      Check for conflicting ips can be avoided with corresponding option.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      9870dce9
    • Dimitris Aragiorgis's avatar
      Implement LUNetworkSetParams · d0af39c0
      Dimitris Aragiorgis authored
      
      Support modifying all network parameters except for network (ip range)
      Cannot modify gateway and reserved ips at the same time.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      d0af39c0
    • Dimitris Aragiorgis's avatar
      Implement LUNetworkQuery · 7eb01378
      Dimitris Aragiorgis authored
      
      Summarily list all existing networks
      Supply detailed info for every existing network
       - List used/free IPs
       - List instances with NICs assigned to the corresponding network
       - List NIC index and IP for the above instances
      
      Implement complementary config methods for retrieving networks.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      7eb01378
    • Dimitris Aragiorgis's avatar
      Basic IP pool management logic · f2837050
      Dimitris Aragiorgis authored
      
      Implement LUs for corresponding opcodes:
       * LUNetworkAdd:
         - Check for IP validity
         - Reserves all necessary IPs
         - Create new Network config object
       * LUNetworkRemove:
         - Checks if connected to any nodegroup
         - Remove a Network config object
      
      Implement basic config methods:
       * LookupNetwork()
         - Given the network name return the network UUID
       * AddNetwork()
         - Add a new network to the config
       * RemoveNetwork()
         - Remove a network from the config
      
      Add new locking level: LEVEL_NETWORK
      
      Add various useful config methods for retrieving network info.
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      f2837050
    • Dimitris Aragiorgis's avatar
      Introduce new module for IP pool management · 8e5f43b1
      Dimitris Aragiorgis authored
      
      Add new library module lib/network.py.
      Introduce new class: AddressPool.
      
      AddressPool implements all operations needed for managing IPs
      inside the IP pool.
      
      Given a Network config object (nobj), the class:
      
       * initializes the corresponding IP pool object via
         network.AddressPool.InitializeNetwork(nobj)
       * obtains the corresponding IP pool object via
         network.AddressPool(nobj)
       * manipulates IPs inside the pool
      
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      8e5f43b1
Loading