This class is a wrapper around fcntl.flock and abstracts opening and
closing the lockfile. It'll used for the job queue.

(The patch also removes a duplicate import of tempfile into the unittest)

Reviewed-by: iustinp

Add instance tag handling, improved error logging.
...oh, yes adopt instance listing for RAPI2!

Reviewed-by: iustinp

Since we need to compute this from outside utils.py, we change this to a
public function.

Reviewed-by: ultrotter

We cannot depend on all environments to have a start-stop-daemon or
similar tool. We instead implement a KillProcess function that behaves
similar to “start-stop-daemon --retry”.

Note that the attached unittest can hang in foreground if the child
misbehaves (doesn't write to the internal pipe). Since unittest are
either run in the foreground or are run with a timeout from an automated
framework, I think this is an acceptable trade-off (against of using
hardcoded timeouts in the test).

Reviewed-by: imsnah

We already have a function to test if a PID is alive, so it makes more
sense to use function composition that force calling (since we need to
read PIDs from files in other places too). Now IsProcessAlive returns
False for PIDs <= 0, since this is the error return from ReadPidFile.

The patch also adds a unittest for checking that WriteFile raises the
correct exception, and checks that an invalid or missing file causes
ReadPidFile to return zero. The unittest tearDown method will try to
cleanup the temp directory too (otherwise it leaves stuff after it).

Reviewed-by: ultrotter

The RESTHTTPServer module went the way of the dodo.

Reviewed-by: iustinp

Reviewed-by: imsnah

The submitted WritePidFile, RemovePidfile and IsPidFileAlive functions
miss unit tests. Adding a simple one which covers their basic
functionality.

Reviewed-by: iustinp

An implementation mistake from the original design caused nodes to be
locked before instances, rather than after. This patch inverts the level
numbering, changing also the relevant unittests and the recursive
locking function starting point.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: imsnah

Reviewed-by: imsnah

This removes (hopefully) all traces of the old locking functions and
uses.

Reviewed-by: imsnah

This is to discourage even more whoever may think that this requirement
is not really useful and can be lifted, and to at least know where it's
used before trying to break it.

Reviewed-by: imsnah

This test checks the LockSet behaviour when an empty list is passed.
The current behaviour is expected, but since this is a corner case,
we're safer to keep it under a check, and if we need a different one
monitor that everything is as we expect it to be.

Reviewed-by: imsnah

Rather than passing a ConfigWriter to the LUs we'll pass the whole
context, from which a ConfigWriter can be extracted, but we can also
access the GanetiLockManager. This also fixes the places where a FakeLU
is created.

Reviewed-by: iustinp

This will be passed to FakeLUs

Reviewed-by: iustinp

Since the ConfigWriter now handles its own locking it's not necessary to
have a specific level for the config in the Locking Manager anymore.
This patch thus removes it, and all the unittest calls that used it, or
depended on it being present.

Reviewed-by: iustinp

This patch creates a new decorator function ssynchronized in the locking
library, which takes as input a SharedLock, and synchronizes access to
the decorated functions using it. The usual SharedLock semantics apply,
so it's possible to call more than one synchronized function at the same
time, when the lock is acquired in shared mode, and still protect
against exclusive access.

The patch also adds a few unit test to check the basic decorator's
functionality, and to provide an example on how to use it.

Reviewed-by: iustinp

Reviewed-by: ultrotter

This function will return the first unused integer based on a list of
used integers (e.g. [0, 1, 3] will return 2).

Reviewed-by: imsnah

All the tests there used to creare a cfg, a sstore, an opcode and a LU.
Put all the duplicate code in the setUp function.

Reviewed-by: iustinp

Reviewed-by: iustinp

The failure is because in high load, the parent gets to run before the
child has the chance to os._exit(), and therefore it is still running
when the parent does the check.

The fix removes the chance of this happening by waiting to receive a SIGCHLD
(but not calling wait()) before trying to test the pid.

Reviewed-by: imsnah

In cfgupgrade, we need to extract parts of and build new version numbers.

Reviewed-by: iustinp

This change allows us to use cleaner dependencies between
directories. The build system is basically rewritten in large parts
and may contain bugs.

Reviewed-by: iustinp

The path to the filename for drbd8 proc data is not correctly computed
when using distcheck. The patch duplicates it from the other drbd tests.

Reviewed-by: ultrotter

Currently, compute the status of a drbd8 device in GetSyncStatus and
return only the values that we need (and fit in the framework of
GetSyncStatus). However, the full status details are useful (and needed)
in other places, so the patch attempts to improve this situation.

We abstract the status of a device outside in a separate class, that
knows how to parse contents from /proc/drbd and set easily accessible
attributes. We then simplify the GetSyncStatus to use this and return
the values that it needs, and add a separate method that returns the
full status object.

The move to a separate class cleans up a little bit the old
sync-progress computation from GetSyncStatus, but it's still many
regexes.

The patch also adds unittests for a few statuses, and modifies one
BaseDRBD call to accept a custom filename instead of '/proc/drbd' to
ease unittests.

Reviewed-by: imsnah

Reviewed-by: iustinp

Reviewed-by: iustinp

This patch changes the code executed when testing the signal handling
of RunCmd. Since sh does not always point to bash (e.g. on Ubuntu,
where it points to /bin/dash) this test might fail due to the returned
exit code is different so the received signal is not correctly
detected.

Additionally fix the docstring of testSignal.

Reviewed-by: iustinp

The whole Ganeti cluster has a single SSH key. Its fingerprint is
written to Ganeti's known_hosts file, together with an alias. This
allows us to always use that alias instead of the real hostname,
making management of the known_hosts file much easier.

This patch does not handle an upgrade from an earlier version.

Reviewed-by: ultrotter

This patch modifies TcpPing and its callers to make the source address
selection optional. Usually, the kernel will know better what
source address to use, just in some cases we want to enforce a given
source address so it makes sense to make this optional.

Reviewed-by: ultrotter

A LockSet is mostly useful when it has some locks in it. On the other hand
there are cases in which it must function even when empty. For example if a
cluster has no instances in it there's no reason why locking all of them
shouldn't work anyway. This patch adds test code for that situation and
implements the necessary fixes to make it work.

Reviewed-by: imsnah

This check that no operation had been performed before release() was missing in
the test code. Adding it.

Reviewed-by: imsnah

Reviewed-by: ultrotter

This new functionality makes it possible to acquire a whole set, by passing
"None" to the acquire() function as the list of elements. This will avoid new
additions to the set, and then acquire all the current elements. The list of
all elements acquired will be returned at the end.

Deletions can still happen during the acquire process and we'll deal with it by
just skipping the deleted elements: it's effectively as if they were deleted
before we called the function. After we've finished though we hold all the
elements, so no more deletes can be performed before we release them.

Any call to release() will then first of all release the "set-level" lock if
we're holding it, and then all or some of the locks we have.

Some new tests checks that this feature works as intended.

Reviewed-by: imsnah

Lockset's remove() function used to return a list of locks we failed to remove.
Rather than doing this we'll return a list of removed locks, so it's more
similar to how acquire() behaves. This patch also fixes the relevant unit tests.

Reviewed-by: imsnah

In a LockSet acquire() returned True on success. This code changes that to
return a set containing the names of the elements acquired. This is still a
true value if we acquired any lock but is slightly more useful (because if
needed one has access to this data without querying for it). The only change
happens if acquiring no locks, which though is a usage which should not
normally happen because it has no practical use.

The patch also changes a some tests to check that the new format is respected.

Reviewed-by: imsnah

Includes some locking-related constants and explanations on how the
LockManager should be used, the class itself and its test cases.

The class includes:
  - a basic constructor
  - functions to acquire and release lists of locks at the same level
  - functions to add and remove list of locks at modifiable levels
  - dynamic checks against out-of-order acquisitions and other illegal ops

Its testing library checks that the LockManager behaves correctly and that the
external assumptions it relies on are respected.

Reviewed-by: imsnah