Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When using pyOpenSSL 0.7 or above, LUClusterVerify will start to show a
warning 30 days before a certificate expires. 7 days before the
certificate expires, the warning becomes an error. Once expired,
LUVerifyCluster will always report an error. The latter is also supported
with pyOpenSSL 0.6.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.
pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and
“notAfter” as strings. Parsing them turned out to be a challenge since
they can be in a variety of formats (YYYYMMDDhhmmssZ, YYYYMMDDhhmmss+hhmm
or YYYMMDDhhmmss-hhmm).

This will be used to verify the validity of cluster certificates in
LUVerifyCluster.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This adds tests for the stacking of objects.Cluster.FillHV to verify
that the override is working as expected.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

To simplify writing unittests, one data structure class in http.server is
also changed. According to the coverage utility, this provides 95%
coverage.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When username and password were sent for a resource not requiring
authentication, it wouldn't be accepted if the user in question had a
hashed password. The reason was that the function GetAuthRealm used to
return None if no authentication was necessary. However, the
authentication realm is necessary to verify hashed passwords. This is
fixed by requiring GetAuthRealm to always return a realm and separating
the decision whether to require authentication or not to a separate
function.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This way we can re-use file objects opened in other places. Also add more
unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This differs slightly from the specification, by allowing names to start
with digits, not checking the length of individual components, and
allowing underscores.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch changes ListVisibleFiles to raise ProgrammerError if it's
passed a non-absolute/non-normalized path name, and adds unittests for
this behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will replace os.path.join since it is not safe for directory
traversal issues.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

According to “coverage”, this covers 99% of the code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function is a generic pythonic version of runparts. We currently
use it in the backend HooksRunner, but we'll use it for running
different directories as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This allows to run a command with only the passed in environment, rather
than just updating the default one with it.

Now with unit testing.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function could be useful in other places and this
way we can easily unittest it.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

On fork, the tempfile module's pseudo random generator is
not reset. If several processes (e.g. two children or parent
and child) try to create a temporary file, they'll conflict.
This function can be used to reset the name generator which
contains the pseudo random generator.

A unittest is included. It is in a separate script because
it changes a variable in the tempfile module to speed up the
test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The recent change to use LogWarning with multiple arguments in
mpcu.py/HooksMaster broke the (simple) mock we have in the tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Every once in a while we saw failures in the locking unittest, but
weren't able to determine the cause. This patch fixes it by using
threading.Event instead of plain threading.Condition to notify another
thread. In most cases, threading.Condition must be used with another
variable to keep the actual state.  threading.Event does this for us.
Otherwise it can happen that the waiter only gets the lock after
condition was notified.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While the name can be looked up in Makefile.am, this
is useful when an automated test just hangs and needs
to be killed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In many cases, where we accept (usually from the command line) a list of
parameters, we remove the use of the separator as an component of any of
the elements.

This patch adds a new function that can split strings of the form
'a,b\,c,d' into ['a', 'b,c', 'd'], with proper un-escaping of
double-backslashes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Having a proper name instead of just a number makes debugging
easier.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The warning will be generated if the clocks diverge by more
than 150 seconds. Due to the way the RPC system works, we
cannot get exact time differences, e.g. if one of the
queried nodes is broken. The comparision is done using a
time window.

Confd queries will fail if the clock on the client and server
are more than 300 seconds from each other. This check helps
keeping at least the nodes of a cluster in sync.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It can't work on older distributions where simplejson
doesn't have indentation support.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 6915bc28 based on thread on
ganeti-devel.

Conflicts:

	lib/cmdlib.py (due to the error code classification, trivial)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should fix an issue I've seen exactly once during testing. It might have
been caused by parallel RPC calls to archive jobs.

[…] ganeti-noded:112 ERROR Error in RPC call […]
 File "/usr/lib/python2.4/site-packages/ganeti/backend.py", line 2365, in JobQueueRename
   utils.RenameFile(old, new, mkdir=True)
 File "/usr/lib/python2.4/site-packages/ganeti/utils.py", line 322, in RenameFile
   os.makedirs(os.path.dirname(new), mkdir_mode)
 File "/usr/lib/python2.4/os.py", line 159, in makedirs
   mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/ganeti/queue/archive/0'

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is ok because adding a node or instance cannot happen in a query.

We get the ec id from the LU and pass it to _EnsureUUID, which will
then for now not use it.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The sort_keys argument is supported since simplejson 1.3.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This new function supports two schemes for passwords:
- Old-style cleartext passwords
- Hashed passwords according to RFC2617 (H(A1))

Schemes are differentiated by their prefix, a concept also
used in OpenLDAP. Cleartext passwords can no longer start
with an opening brace ("{") unless they're prefixed with
"{cleartext}" (case insensitive).

Currently there's no documentation for rapi_users at all.
It'll be in a consecutive patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is particularily useful for “gnt-cluster redist-conf”, but
also for all other cases where the configuration files are
rewritten on other nodes.

$ gnt-cluster redist-conf
… Copy of file /var/lib/ganeti/config.data to node … failed: Error while
executing backend function: [Errno 1] Operation not permitted
… Error while uploading ssconf files to node …: Error while executing backend
function: [Errno 1] Operation not permitted

$ gnt-node modify --offline no --force node3.example.com
… - WARNING: Not enough master candidates (desired 10, new value will be 4)
… Copy of file /var/lib/ganeti/config.data to node node8.example.com failed:
Error while executing backend function: [Errno 1] Operation not permitted
Modified node node3.example.com
 - offline -> True
 - master_candidate -> auto-demotion due to offline

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit d22b2999 broke the serializer unittests with certain
versions of simplejson. This patch removes sort_keys again
and implements a slightly more efficient way of detecting
simplejson functionality. The serializer unittests no longer
use a partially broken mock, but rather a function to convert all
tuples to lists before comparing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

With this patch all timeouts are pre-calculated. The interface of
the _LockTimeoutStrategy class is also changed a bit; NextAttempt
now returns a new instance.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timeout is always between ~0.1 and ~10.0 seconds. A small
variation of ±5% is added to prevent different jobs from
fighting each other. After 10 attempts to acquire the locks with
a timeout, a blocking acquire is made.

Lock status reporting will be improved in a separate patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timeout passed to LockSet.acquire() is measured over all lock acquires. If
LockSet.acquire fails to acquire all requested locks within the specified
amount of time, all locks are released again and the acquire fails.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- We never remember to use it (5 uses vs 21 " ,".join())
- It's longer to write than " ,".join()
- The added value of the apostrophe in the string is not very much

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>