GitLab

When an instance is being created, add an extra NIC (if necessary)
which is meant to be used by the instance communication.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Move rpc.py to rpc/node.py and modify imports in existing code.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add parameters to the Gluster disk template so Gluster can manage the
mount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Gluster still does not mount anything autonomously, but this commit
changes where Gluster expects its mountpoint to be.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add Gluster to Ganeti by essentially cloning the shared file behaviour
everywhere in the code base.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

If param=default and the param already exists then we remove
it from params dict. This is stolen by GetUpdatedParams() which
is used for hvparams modification/inheritance.

This means that 'default' value is not accepted for an arbitrary
param of an ext disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Disks of ext template are allowed to have arbitrary parameters
stored in the Disk object's params slot. Those parameters can be
passed during creation of a new disk, either in LUInstanceCreate()
or in LUInsanceSetParams(). Still those parameters can not be
changed afterwards. With this patch we override this limitation.

Currently, for the other disk templates we allow modifying only
'name' and 'mode'. Therefore, we introduce new constants
MODIFIABLE_IDISK_PARAM* to include those params. If any other
parameter is passed, _VerifyDiskModification() will raise an
exception.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This is required to prevent race conditions such as removing a network
from a group and adding an instance at the same time. (See issue 621#2.)
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

If the file driver of an instance with file based storage is not specified, the
default one is automatically added by the UpgradeConfig function.

Fixes Issue 571.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

In case no name is passed in disk modifications we should
keep the old one. If name=none then set disk name to None.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Both NODE and NODE_RES locks were acquired opportunistically if so
requested by the user. LUInstanceCreate requires, however, that the
actually locked elements on NODE and NODE_RES level are the same.

This patch changes the locking of NODE_RES such that those locks are not
acquired opportunistically any more. Instead, the mandatory locks are
set to the acquired NODE locks once they are actually granted.

This fixes issue 622.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This will be useful for an external entity using RAPI that
wants to modify devices of instances.

The common use case for that is:
"I want to add a NIC/disk to an instance. If it is running
then try to hotplug the device. If not, then just add it to config."

Another use case is that if we are unaware of the instance's
hypervisor: "I want do add a NIC/disk to an instance and if
its hypervisor is KVM try to hotplug it."

For all those cases just print a warning in CheckPrereq() and
disable any further hotplug related actions (and let
modifications take place after reboot).

Add NoHoplugSupport and HotplugIfPossible unittests an modify
existing by explicitly creating successful RPC results for
call_hotplug_supported.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The administrator should be able to assign an externally reserved IP
to a Ganeti instance manually, if desired. Currently this is not
supported. External reservations should act as holes in the pool and
not just as IPs already used by someone outside of Ganeti.
Automatic allocation with ip=pool will continue to exclude those IPs
as happens now.

To allow such functionality the administrator needs to pass explicitly
the desired IP along with the ``--no-conflicts-check`` option, or else
an error will be produced as happens now.

The aforementioned require the following changes:

 - Make IsReserved() to look either in reservations or external ones.
 - Make Reserve() and Release() to use IsReserved() with external
   argument True or False.
 - Pass extra option to ReserveIp() to bypass checking of external reservations
 - Update man pages and design doc for this change.

Furthermore, a side effect of this patch is that it fixes the
following problem:
Currently, one could not mark an IP as external if it was already
reserved (i.e. belonged to an instance). The code would produce a warning
and fail silently.

Fix config_mock.py so that if network and ip is given then reserve it in
the pool.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Ganeti does not support optional fields in parameters
(hypervisor-params, disk-params, etc.). OpenVSwitch related node
parameters were the exception to this rule, which caused numerous
problems related to import/export and (de-)serialization.

The reason for making those parameters optional in the first place was to
disallow them when OpenVSwitch is not used. This was not consistent with
other parts of Ganeti, where we allow parameters to be set even though they
are not actively used.

This patch makes all node parameters mandatory and provides sensible
defaults for them. Checks which make sure that certain parameters are
not set in some cases were removed, and the tests adapted. Also, the
inheritance logic from cluster -> node group -> node was implemented, as
it was missing previously.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

DTS_FILEBASED is a constant that exists and this commit makes sure
that it is used whenever sensible, rather than resorting to hardcoding
the pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Introduce new RPC hotplug_supported that invokes the corresponding
hypervisor's method which checks if hotplug is generally
supported. Call this RPC early in CheckPrereq() and abort if
hotplug is not supported. Currently only KVM hypervisor with
qemu versions > 1.0. Please note that this check is unaware of
target device and hotplug action. This latter check is still
done during hotplug_device() RPC.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

OpInstanceCreate now supports the 'compress' option. It allows to enable
compression during instance imports.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Support compressing instance data while sending it to the target node on
instance imports.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

OpInstanceMove now supports the 'compress' option. It allows to enable
compression for intra-cluster instance moves.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This unifies the inter- and intra-cluster moves and the backup code, so
less code needs to be maintained.

Also fix failing tests for LUInstanceMove by following the code changes
in the tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The cluster object is already retrieved from the configuration, so use
it consistently in LUInstanceMove.CheckPrereq().
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Correct comments which confuse the target node with the secondary node
of an instance in LUInstanceMove.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This fixes issue 519. When calling 'gnt-instance modify
--new-primary', an exception was raised, because a
RPC call was not called with the right hypervisor
parameters which caused the backend function to crash,
because it could not determine which Xen toolstack (xm/xl)
was configured to be used.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* Mention restrictions in NEWS
  - RBD userspace access mode
  - In case of a downgrade instances should suffer a reboot to be migrateable
* Bypass interactive verification in NIC modifications via --force option
  - Mention it in man page
* Print "modifications take place after restart" message no matter if
  --hotplug option is passed. Change cmdlib to append hotplug status info
  (if any) to the final result printed out to the user
* Change hotplug option description
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Use _SingleDiskDictDP() instead of _ObjectDict() for serializing
a disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Hotplugging is done by functions invoked by ApplyContainerMods(). In
order hotplugging to take place the --hotplug option must be passed
otherwise the modifications will take place after reboot.

NIC hotplug supports add, remove and modify. The modify is done by removing
the existing NIC and adding a new one in the same pci slot.

Disk hotplug supports add and remove.  Before hotplugging a Disk it
must be assembled. Use blockdev_assemble RPC, get the device link_name
and then call hotplug RPC command.

In order to remove a disk (with blockdev_remove) it must be shutdown. So
after unplugging the disk ShutdownDiskInstanceDisks() must be invoked.

For both device types we use the generic RPC call_hotplug_device.

Also adapt unit tests to follow the change.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch replaces all usages of the utility function
'GetLvmDiskTemplate' by the new 'DTS_LVM' constant
to make it consistant with the usage of other DTS_*
constants.

Additionally, it provides a unit tests to ensure
consistancy between DTS_LVM and the mapping of disk
templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch renames the constant 'LDS_DRBD' to 'DTS_DRBD'
to make it consistent with the renaming of LD_* constants
to DT_* constants.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

* Prevent incompatible hypervisor/disk template/access type
  combinations from being reached through the command line.
* Prevent the access config from taking illegal values such as "pink
  bunny" through the command line.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

All data traffic usually goes over the secondary network, but
gnt-instance move didn't. This patch corrects this problem by using the
target nodes secondary IP as move target.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit f56ab6d1)

All data traffic usually goes over the secondary network, but
gnt-instance move didn't. This patch corrects this problem by using the
target nodes secondary IP as move target.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Adding a disk to an instance used to leave the disk behind activated, no
matter how the disks_active flag of the instance was. This changes make
sure that new disks are only active if the other disks of the instance
are active too.

Tests are included.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

When creating an instance, gnt-instance waits for instance disks to
sync. Inconsistently, this was not the case for adding a disk to an
instance. This patch changes the default behavior to wait for sync when
adding a disk, but honor the --no-wait-for-sync option which
`gnt-instance modify` already supports.

Also adapt unit tests to mock the additional RPC call.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The 'physical_id' field of disk objects is no longer used, so remove it.
Also, all references are removed together with the code which made sure
that the physical_id is up to date when transmitted over RPC.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The disk field 'physical_id' has to be kept up to date whenever a disk
object is sent to a node via RPC. This is done with the SetDiskID method
manually, which is a source of bugs.

This patch replaces the use of 'physical_id' with a new field names
'dynamic_params'. The RPC code is adapted to update this field whenever
a disk object is sent to a node. Furthermore, this field is only ever
set on copies of disk objects which don't get written to the
configuration file. On the node side, the use of 'physical_id' is
removed and the new dynamic parameters are used now for the same
purpose.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Between the last two merges, nothing happened on stable-2.9. The
only change on stable-2.8 that touched lib/cmdlib/instance.py is
ff34fb97

. So make sure, the net change since the last merge to
this file is that of the said commit. This also fixes the doclint
errors.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The file-driver value, used by file-based instances, had a default value when
an instance was being created over the CLI, but not when the instance was
created through the RAPI.

This patch introduces a default value for the remote API and, while doing so,
also centralizes in a single place the definition of such a value, for both the
interfaces, in order to remove the possibility for them to be out of sync.

Fixes Issue 571.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Those two occurrences of warnings can be replaced the RpcResult.Warn
method, which is done in this patch.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The validation of the opcodes already checks the size parameter for the
right type, so this except block is not required.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>