Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There were more race conditions. By adding a notify function to
SharedLock.acquire we can prevent them.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

If we're de-offlining or de-draining a node we need to promote it to MC
if we have not enough, or the config will be corrupt.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

During node add we decide whether to self promote to an MC. Abstract
this decision making to a separate function.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Currently during a master candidate removal, when it's possible to
promote another node, the removal operation fails because of a corrupt
config before it's even possible to do the promotion. Fixing this by
doing the promotion before, excluding the current node.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

If --force is used to demote an MC, but then there are not enough MCs in
the cluster, the configuration gets corrupted until a node is promoted.

In order to avoid that we only allow demotion with --force if the node
is offlined or drained at the same time, and we don't have any other
node available to promote.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Other than returning the current number of candidates, and the number of
desired and possible candidates, we also return the maximum possible
number, even if greater than our desires. All callers for now ignore
this third value.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this patch we simplify usage on the SingleActionCondition (which
wasn't a condition at all) by making it a real condition. This way we
can just wait() on it, or notifyAll() as we would on a normal one. The
only catch is that notifyAll can be called only once, and wait can only
be called before notifyAll has, but luckily our PipeCondition, now quite
simplified, takes care of this, by providing a new SingleActionCondition
each time the previous one has been notified.

No Start/StopUsing function are needed anymore, and thus the condition
is a lot more robust, and there's no way file descriptors can be left
open, as they are closed in a finally block, in the same function where
they were opened, by the last thread exiting the class.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This way they can be used to test different condition classes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

All (ok, all but one) _ThreadedTestCase users have a done Queue, so we
move its building in the _ThreadedTestCase setUp

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Each condition has an underlying lock, the acquire and release methods,
and a few helper methods to check that it's called in the proper way.

Abstract them to a separate class so we can have more than one without
duplicating this code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

SharedLock.__acquire_unlocked uses keyword parameters. Just passing
the timeout would set the “shared” parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also remove the “blocking” parameter from LockSet.remove and
GanetiLockManager.remove. There's no point in implementing timeouts on removal
unless we need them.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Our automated test system found a few problems in the new locking
unittests. This patch should fix them, although I wasn't able to
reproduce the problem. All are race conditions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

_PipeCondition is a condition implemented using pipe(2) and poll(2).
It allows the implementation of timeouts without using a busy-wait loop
with time.sleep.

Unlike Python's built-in threading.Condition class and to save file
descriptors and an internal queue, it can only be used to notify
all waiters. Ganeti's use case for this condition class doesn't
require the ability to notify only one waiter.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This class will be used as a basic block for pipe(2)-based
conditions. Upon initialization it creates a pipe and can be
notified once (hence the “single action” in the name). A
callable helper class is used to wait for notifications.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch greatly simplifies the SharedLock code and implements
timeouts for the acquire() and delete() functions. A wrapper around
Python's threading.Condition class must be used to ensure thread
safety when check whether there are any waiters left.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The query now accepts a link parameter.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* origin/master:
  Fix burnin's verbose mode
  Final NEWS update and version increase for 2.0.4
  Encode the actual exception raised by LU execution
  Move the luxi error handling into errors.py
  Fix the confusing ssh/hostname message in node add
  Add man page for ganeti-cleaner
  Update NEWS file for version 2.0.4
  Automatically cleanup _temporary_ids at save
  Separate the computation of all config IDs
  Change config upgrade to be explicit
  Fix unittest breakage due to new test file
  Fix /proc/drbd parsing in presence of gaps
  repair-size: ensure child disks have sane sizes
  Fix yet another bug in LURepairDiskSizes
  Fix a bug in LURepairDiskSizes

Conflicts:
	NEWS          (trivial, the RST change)
	lib/cmdlib.py (trivial, some small change in 2.0)
	lib/config.py (due to the cherry-picks and UUID changes in 2.1)

Signed-off-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Remove variables which are listed at the beginning of the section and variables
which are not declared when building hooks env.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timestamp need special formatting, which was done for the internal
buffer storage but not for the messages logged in verbose mode. This
patch unifies the formatting for these two cases.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

QA passed successfully, let's try to have a release.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Some basic tests for the confd client library

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Until now the peers have to be the same all the time. Adding a new
function to update the list, and call it from the constructor to avoid
duplicating code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is not needed anymore - the original change was more than a year
ago when masterd was in its incipient phase.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It may be handy for upcalls to know which client called them, and call
it back. So we create a new "client" field in the upcall target,
containing the current client instance

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Only install ganeti specific files if they exist. This way we can call
ganeti's devel/upload in another sub-projects (eg. nbma) and have it
uploaded to a host as well, without having to create a new script there.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The HandleExpire function takes the whole "up" structure, and not just
the salt.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Many functionalities of confd have been moved to other classes/modules,
and the main confd daemon doesn't reference these modules directly
anymore.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-confd doesn't need to log anything related to os installations.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the actual exception raised during an LU execution (one of
OpPrereqError, OpExecError, HooksError, etc.) is lost because the
jqueue.py code simply sets that to a str(err), and the code in cli.py
simply passes that string to OpExecError.

This patch moves to encoding the errors as per errors.EncodeError and
changes the cli code to parse and raise that (if possible).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit bcb66fca)

Currently the luxi error handling is hardcoded as special encoding on
the masterd-side and special decoding on the client side. This patch
moves it to errors.py such that other parts of the code can reuse the
same encoding.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 6956e9cd)

Before, it used to say:

  ssh/hostname verification failed node1.example.com -> hostname mismatch, got
  node2

Now it says for wrong hostnames (maybe too verbose):

  ssh/hostname verification failed (checking from node1.example.com): hostname
  mismatch, expected node2.example.com but got node3

And for non-FQDN hostnames:

  ssh/hostname verification failed (checking from node1.example.com): hostname
  not FQDN: expected node2.example.com but got node2

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>