GitLab

Verify that, if a lock owner gets notified, he
- had a pending request, and
- the request is fulfilled now.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...from a lock waiting structure. In this way, all the data
describing the behavior can be inspected.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Not only verify that after the blockers for one pending request
have gone, some request was honored, but also verify that it was
not a request by one of the blockers.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

It is not necessarily bad coding stile to use "and" to present
a list of conditions. In fact, a statement like

    and [ foo == foo'
        , bar == bar'
        , baz == baz'
        ]

can be more readable than the equivalent

    (foo == foo')
      && (bar == bar')
      && (baz == baz')
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Jobs may depend on other jobs in the sense that they may only be started
once a given job is finalized. For a job process, however, it is hard to
determine if the status of a different job without a significant overhead.
Therefore, only hand over to execution those jobs where all jobs they depend
on have been finalized.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a function that computes the list of job ids a job depends
on. This will allow to schedule only those jobs for execution,
where all jobs it depends on have been finalized.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a function that extracts the job id, if given in absolute
form, from a job dependency.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

.. in LUClusterSetParams. Since _SetFileStorageDir updates the 'cluster'
variable directly, we need to re-read it and then save later (the other
calls use the configuration directly).
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds a design document detailing the implementation of disks
as new top-level citizens in the config file (just like instances,
nodes etc).

The process is divided in four steps, and currently only the first one
is being described in depth.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

...so that we can notify owners when their pending request got granted.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a basic sanity check for progress on pending requests. If a
request is pending and all owners it is blocked on release their
locks, there is at least one pending request that can be granted
(the said one). So verify that at least one owner gets notified
of the granting of his request.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a basic sanity check on notification: if a request is blocked,
and all old pending owners and blockers give up their resources,
the owner gets notified of his lock being granted.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a basic sanity check for the lock waiting mechanism. If a request
get added to pending, and all blockers and pending owners give up their
resources it is automatically handled.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...so that this predicate can be used for testing the LockWaiting
structure as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a convenience function---definable from the exported
interface---to clean up all the resources of an owner. This
will be needed, e.g., if a job dies.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a function that allows an owner to remove a pending request
it might have. Besides useful for testing, this will also be
needed to clean up the resources for dead owners.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Verify that an owner has a pending request after a waiting request
not fullfilled immediately.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a verify first sanity check for lock waiting: a user that has a pending
request cannot modify his locks. This patch also brings in the necessary
infra structure for having arbitrary waiting structures.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

As lock waiting is also based on locking, export the
test lock/owner structure, so that the tests for lock
waiting can use them.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...as the question of who has pending requests is an import
part of the data structure.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...so that all kind of diagnostic tasks can be carried out without
duplicating the functionality on the waiting structure. Note that
we still do not export the constructor so that we can change the
implementation without breaking the interface.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add functions to update the lock allocation structure and also
compute all the consequences of the update, if it succeeds. Optionally,
requests that cannot be fulfilled immediately, it can be added to
the pendig requests.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

To avoid jobs polling on locks, and also to ensure that the
most important of the waiting jobs obtains a lock, add a
data structure to keep track of who is waiting for which locks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This will allow automatic connection and socket cleanup on command
completion.

We also repeat the Qmp tests using the context manager. For this to be
feasible, we move the test scenario to class variables and modify
QmpStub to not consume its script.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch will do all that is necessary to allow the helper VM to do
whatever it will do with the zeroing image in place - which is,
hopefully, to zero out the disks of the instance.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds the two parameters to gnt-backup export, documenting
their meanings in the manual file as well.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds two parameters controlling the zeroing timeout - one
that is fixed and another that depends on the amount of data (size of
disks) to zero.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch moves towards the actual zeroing by providing a function
that determines the necessary size and creates and destroys a temporary
disk.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

To create a temporary disk, it is necessary to know just how big this
disk must be. This patch adds a function that retrieves this
information for both data sources - a URL and a file.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Due to the need to determine the size of the temporary disk used for
the VM helper, an RPC call capable of supplying this information has
been added. It can be extended as needed to supply additional
information provided by Python's os.stat.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The assert stating that the disk did not have to be activated made
sense once the instance's disks were never completely deactivated. With
zeroing, the assumption no longer holds true, and should be replaced by
a check that the disks of the instance are indeed active.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The instance zeroing requires that a virtual appliance capable of
performing it is used, and short of provisioning another VM, the best
Ganeti can do is reuse the virtualization environment of the instance.
To do this, a disk is needed to host the OS performing the zeroing, and
this patch introduces a utility function that temporarily creates one.

This disk is not meant to last, and almost all Ganeti checks will
scream and flail upon seeing it. This is intentional. It should not
survive any more than needed, and the context manager syntax should
enforce its use as such.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Examining whether an instance is running is useful outside the instance
state check, especially as the check relies on the admin_state recorded
within the configuration, which the code treats as the desired rather
than the actual state. This patch pulls the check out into a separate
function. It also updates the function to account for possible user
shutdowns, if the right option is passed.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds a node lock if zeroing is used, preventing any
operations that might interfere with the amount of available space.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds the option, allowing the CLI client to use the zeroing
option to save space when compressing images.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds a short description of the zeroing-image parameter to
the manual page of gnt-cluster.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds the zeroing-image option to gnt-cluster and the
OpBackupExport params. The many changes are all minor, yet necessary.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The image validity check has been implemented as a part of the OS
installs, yet it could be useful for the zeroing image as well. This
patch factors the utility out.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>