Having a proper name instead of just a number makes debugging
easier.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The warning will be generated if the clocks diverge by more
than 150 seconds. Due to the way the RPC system works, we
cannot get exact time differences, e.g. if one of the
queried nodes is broken. The comparision is done using a
time window.

Confd queries will fail if the clock on the client and server
are more than 300 seconds from each other. This check helps
keeping at least the nodes of a cluster in sync.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It can't work on older distributions where simplejson
doesn't have indentation support.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 6915bc28 based on thread on
ganeti-devel.

Conflicts:

	lib/cmdlib.py (due to the error code classification, trivial)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should fix an issue I've seen exactly once during testing. It might have
been caused by parallel RPC calls to archive jobs.

[…] ganeti-noded:112 ERROR Error in RPC call […]
 File "/usr/lib/python2.4/site-packages/ganeti/backend.py", line 2365, in JobQueueRename
   utils.RenameFile(old, new, mkdir=True)
 File "/usr/lib/python2.4/site-packages/ganeti/utils.py", line 322, in RenameFile
   os.makedirs(os.path.dirname(new), mkdir_mode)
 File "/usr/lib/python2.4/os.py", line 159, in makedirs
   mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/ganeti/queue/archive/0'

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is ok because adding a node or instance cannot happen in a query.

We get the ec id from the LU and pass it to _EnsureUUID, which will
then for now not use it.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The sort_keys argument is supported since simplejson 1.3.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This new function supports two schemes for passwords:
- Old-style cleartext passwords
- Hashed passwords according to RFC2617 (H(A1))

Schemes are differentiated by their prefix, a concept also
used in OpenLDAP. Cleartext passwords can no longer start
with an opening brace ("{") unless they're prefixed with
"{cleartext}" (case insensitive).

Currently there's no documentation for rapi_users at all.
It'll be in a consecutive patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is particularily useful for “gnt-cluster redist-conf”, but
also for all other cases where the configuration files are
rewritten on other nodes.

$ gnt-cluster redist-conf
… Copy of file /var/lib/ganeti/config.data to node … failed: Error while
executing backend function: [Errno 1] Operation not permitted
… Error while uploading ssconf files to node …: Error while executing backend
function: [Errno 1] Operation not permitted

$ gnt-node modify --offline no --force node3.example.com
… - WARNING: Not enough master candidates (desired 10, new value will be 4)
… Copy of file /var/lib/ganeti/config.data to node node8.example.com failed:
Error while executing backend function: [Errno 1] Operation not permitted
Modified node node3.example.com
 - offline -> True
 - master_candidate -> auto-demotion due to offline

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit d22b2999 broke the serializer unittests with certain
versions of simplejson. This patch removes sort_keys again
and implements a slightly more efficient way of detecting
simplejson functionality. The serializer unittests no longer
use a partially broken mock, but rather a function to convert all
tuples to lists before comparing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

With this patch all timeouts are pre-calculated. The interface of
the _LockTimeoutStrategy class is also changed a bit; NextAttempt
now returns a new instance.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timeout is always between ~0.1 and ~10.0 seconds. A small
variation of ±5% is added to prevent different jobs from
fighting each other. After 10 attempts to acquire the locks with
a timeout, a blocking acquire is made.

Lock status reporting will be improved in a separate patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timeout passed to LockSet.acquire() is measured over all lock acquires. If
LockSet.acquire fails to acquire all requested locks within the specified
amount of time, all locks are released again and the acquire fails.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- We never remember to use it (5 uses vs 21 " ,".join())
- It's longer to write than " ,".join()
- The added value of the apostrophe in the string is not very much

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now featuring unit testing, and more deterministic results on some
corner cases.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There were more race conditions. By adding a notify function to
SharedLock.acquire we can prevent them.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this patch we simplify usage on the SingleActionCondition (which
wasn't a condition at all) by making it a real condition. This way we
can just wait() on it, or notifyAll() as we would on a normal one. The
only catch is that notifyAll can be called only once, and wait can only
be called before notifyAll has, but luckily our PipeCondition, now quite
simplified, takes care of this, by providing a new SingleActionCondition
each time the previous one has been notified.

No Start/StopUsing function are needed anymore, and thus the condition
is a lot more robust, and there's no way file descriptors can be left
open, as they are closed in a finally block, in the same function where
they were opened, by the last thread exiting the class.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This way they can be used to test different condition classes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

All (ok, all but one) _ThreadedTestCase users have a done Queue, so we
move its building in the _ThreadedTestCase setUp

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

SharedLock.__acquire_unlocked uses keyword parameters. Just passing
the timeout would set the “shared” parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also remove the “blocking” parameter from LockSet.remove and
GanetiLockManager.remove. There's no point in implementing timeouts on removal
unless we need them.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Our automated test system found a few problems in the new locking
unittests. This patch should fix them, although I wasn't able to
reproduce the problem. All are race conditions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

_PipeCondition is a condition implemented using pipe(2) and poll(2).
It allows the implementation of timeouts without using a busy-wait loop
with time.sleep.

Unlike Python's built-in threading.Condition class and to save file
descriptors and an internal queue, it can only be used to notify
all waiters. Ganeti's use case for this condition class doesn't
require the ability to notify only one waiter.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This class will be used as a basic block for pipe(2)-based
conditions. Upon initialization it creates a pipe and can be
notified once (hence the “single action” in the name). A
callable helper class is used to wait for notifications.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch greatly simplifies the SharedLock code and implements
timeouts for the acquire() and delete() functions. A wrapper around
Python's threading.Condition class must be used to ensure thread
safety when check whether there are any waiters left.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Some basic tests for the confd client library

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

While ‘test1’ matches both ‘test1’ and ‘test1.example’, it has a full,
exact match and we should return it if that is the case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Checking that the fourcc code is indeed a fourcc code, and that request
codes and reply statuses are distinct.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

It should only be used at build-time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The mcpu.py:HooksMaster class needs to have a proc attribute/argument to
init in ordet to call its LogWarning method. However, this is available
from the 'lu' attribute, so we can remove this dependency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>