GitLab

.. because instead of a job ID and a lock file, we now pass a single,
client ID object.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

.. to comply with the updated WConfd interface.

This makes the call less useful as the thread ID is often unknown to
users. This needs to be improved in the future.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This allows to distinguish threads that don't have a job id, which is
needed for answering queries.

Since Python thread IDs aren't guaranteed to be unique, in future it'd
be preferable to use a different, unique identifier.

Note that this breaks 'gnt-debug locks'.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

.. to test if `readJSON . showJSON = Ok`.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

.. instead of using a pair.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

... namely, unit tests and QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

... as this is now performed on the node.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

... include the RPC post process hook.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

... because 'CheckOSParams' already checks the OS variant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Move function 'CheckOSVariant' to the node and add parameter
'force_variant' to RPC 'os_validate', thus making the node verify the
OS variant together with the rest of the OS params.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

... which is a Web server URL that serves additional files needed by
QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

If QA storage is enabled, fetch OS images and use them to test
instance reinstalls.  Otherwise, just skip the two tests regarding OS
images.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

According to Ganeti design, files should not be copied from master to
nodes and instead they are assumed to exist and it is the user's
responsibility to assure that the file does exist.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

... because when the data source is infinite, truncation is not
necessary, but when the data source is finite and is, for example,
smaller than the device, truncation can reduce the disk size.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Turns out, rst wants to have a blank line before
enumerations. Also rearranging some paragraph to make the
lines align better.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This patch elaborates the node security design wrt to SSH
key handling to make sure it is feasible before starting
the implementation.
In this updated design the first and more simple proposal
of simply removing the private root key from normal nodes
was abandoned, because the implementation of various
node operations (adding/removing, promoting/demoting)
turned out to contain too many security problems so that
the second proposal, where each node get's a separate
key pair was chosen to be implemented.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

So far, the mcpu acquires locks that live in memory
of masterd. This design does not fit with our jobs-as-processes
goal. So make mcpu acquire the corresponding locks in WConfD
instead.

Note that this implies changes in various other files that call
to the Ganeti Lock Manager directly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

As our so-called unit tests for the logical units are more
like integration tests, calling the actual LU through an almost
live mcpu, we need a quite involved "mock" for the wconfd module,
called indirectly through mcpu. The difficulties arise from the
fact that both, the mcpu and the LUs verify lock ownership at
various places.

To keep things simple, we only keep track of the mocked state
of affairs where the result is actually checked. With more
consistency checks in the LUs, this module has to be implemented
further.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...as the will be used by mcpu for lock handling and that will
require tests to be adapted accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Export a function that downgrades all locks of an owner of a
given level to own them only shared.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a convenience function that downgrades all locks with a given
property to be held only at shared level.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.11
  Rename compression option in gnt-backup export
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The gnt-backup export option --compress did not do what most would
expect upon seeing the name: compress the exported image. Instead, it
used compression to try and speed up the transfer of the image,
decompressing it prior to writing the file. To reduce confusion, this
patch renames the option to --transport-compression. The change can
still be done in 2.11 as this option was introduced in this version
only, and it is beneficial as it allows a proper --compress or
--image-compression option to be added at a later time.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Extend QA with OS image tests for instance reinstall and instance
modify.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This patch allows instances to be reinstalled using OS images:
* extract OS scripts installation to a separate method
* add a new method to handle OS image installation
* refactor 'LUInstanceReinstall.Exec'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* Use the empty string in 'ganeti.objects.Instance.os' to signify that
  the instance does not use OS scripts.

* Make several calls related to OS scripts first test whether OS
  scripts are being used.

* Allow OS images to be used in 'LUInstanceCreate'.

* Update error message in unit tests related to missing OS type.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Extend 'LUInstanceCreate' to image the instance's first disk if an OS
image is specified via the OS params.  If the OS image is a file, it
will be copied to the node via SSH.  If ths OS image is a URL, it will
be passed directly to the node, which will then download the file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add helper function to check if the OS image parameter contained in
the OS parameters of an opcode is valid.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* Add 'ImageDisks' that actually calls the 'blockdev_image' RPC in a
  context where disk sync is paused.
* Add 'ImageOrCleanupDisks' which is just a wrapper around the
  previous function that cleans up if an exception occurs.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* Extract code, into '_RemoveDegradedDisks', that checks if the disks
  of an instance are degraded and, if so, remove the instance.

* Add a nice comment
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add RPC 'blockdev_image' that uses 'ganeti.backend.BlockdevImage' to
dump an image to an instance's disk device, optionally downloading
that image.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'BlockdevImage' which downloads a file and dumps it to an
instance's disk if the path is a URL, otherwise it dumps the file
directly to the instance's disk.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* Add constant that holds the curl binary.

* Add '_DownloadAndDumpDevice' which downloads a file (e.g., OS image)
  through curl and dumps it to a disk device (e.g., an instance disk).
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* Fix order of imports

* Generalize 'WipeDevice' to 'DumpDevice' which takes an additional
  argument which is the data source.  This will allow this function to
  be reused for both wiping and imaging disks in a future patch.

* Update error messages in order to be consistent with the error
  messages about imaging disks, which will be introduced in a future
  patch.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'GetOSImage' and 'PutOSImage' which handle the OS image key in the
OS parameters dict.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>