GitLab

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will be used to avoid a race condition between starting a program (dd
for import/export) and sending signals to it.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Some confusion arose handling EINTR on this function: in python 2.6
socket.error is an IOError, and thus:
  - It's an EnvironmentError
  - It has an .errno member

In 2.4 and 2.5 it's not, and so its errno variable must be extracted
from the args tuple. This patch fixes both the function, and the
unittests.

This is a cherry-pick of master commit
965d0e5b

 with the portions not relevant
to 2.1 removed (changes to the RetryOnSignals function).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some confusion arose handling EINTR on those functions: in python 2.6
socket.error is an IOError, and thus:
  - It's an EnvironmentError
  - It has an .errno member

In 2.4 and 2.5 it's not, and so its errno variable must be extracted
from the args tuple. This patch fixes both functions, and the unittests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since the hash functions' changed their module name between python 2.4
and 2.6, and we have to do an try/import/except trick, we'll do it just
once, for both hash functions, and in compat.py. This also fixes a use
of md5 in the utils unittests which didn't use the trick before, and
generated a deprecation warning under 2.6.

In compat we keep both a ganeti-wide non-version-specific version to be
used by other ganeti modules, and a python-version specific that can be
passed to python modules which expect a hash function for their input
but call it differently under different versions of python (hmac, for
example).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

EnsureDirs() should create directories with the exact mode requested
in the arguments, but it currently applies the umask.
This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove duplicate code between a couple of asyncore related function by
having a function in charge of handling EINTR errors. Unittests included.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Read the first non-empty file line. When strict is set, abort if more
than one line is non-empty. Some unittests inspired by the reverted ones
from commit b774bb10

, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This partially reverts commit b774bb10

.
Unittests unrelated to that particular functionality but introduced in
that commit are left untouched. Since the temporary directory is now
only used to check for a non-existing file, it is removed as well, and
a convenient very-very-unlikely-to-exist file is used instead.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Reading only one line is useful for reading passwords from files. This
also adds unittests for utils.ReadFile.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The docstring has been copied by the name, but the name reports a
misspell of the function it is actually testing.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

If Retry has to fail with RetryTimeout, it might be useful to pass the
Retry argument to RetryTimeout, to help debugging outside the Retry
cycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise an
exception passed to RetryAgain. All served with a side of unit tests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Check that Retry actually times out, and actually retries.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We're using salted hashes all over the place.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

HMAC will be used in more places. Centralizing some parts can't hurt.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This function will also be useful for inter-cluster instance
moves for verifying certificates.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This can be very useful if client programs run as non-root.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

utils.py, where they were before, is already huge.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The function's code was mostly copied from Python's documentation
and it's equivalent to “functools.partial” in Python 2.5 and above.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is useful in combination with utils.StartDaemon and will be used for
reading the import/export daemon's PID file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This will be used by the new import/export daemon to split
output by its child processes into lines.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit dfdc4060

 added WaitForFdCondition which uses utils.Retry without
handling timeout exceptions. This breaks any nested retry loops.

This patch fixes the above function, and also changes utils.Retry to
detect and warn future similar cases. In addition, we add a few small
unittests for utils.Retry.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Certificates exchanged via an untrusted third party should be
signed to ensure they haven't been modified.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Seconds since the Epoch are not easily readable by a human. Using a
formatted timestamp makes it easier (e.g.
“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles use
this formatted timestamp.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.
pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and
“notAfter” as strings. Parsing them turned out to be a challenge since
they can be in a variety of formats (YYYYMMDDhhmmssZ, YYYYMMDDhhmmss+hhmm
or YYYMMDDhhmmss-hhmm).

This will be used to verify the validity of cluster certificates in
LUVerifyCluster.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This way we can re-use file objects opened in other places. Also add more
unittests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This differs slightly from the specification, by allowing names to start
with digits, not checking the length of individual components, and
allowing underscores.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch changes ListVisibleFiles to raise ProgrammerError if it's
passed a non-absolute/non-normalized path name, and adds unittests for
this behaviour.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will replace os.path.join since it is not safe for directory
traversal issues.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This function is a generic pythonic version of runparts. We currently
use it in the backend HooksRunner, but we'll use it for running
different directories as well.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This allows to run a command with only the passed in environment, rather
than just updating the default one with it.

Now with unit testing.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The currently available functions in this direction (utils.RunCmd and
utils.Daemonize) both can not be used to start an external daemon
process. This function takes many steps to ensure the daemon is started
properly and does careful error checking. Unittests are included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This saves us one dependency and saves us from complicated handling of
external files if we need key and certificate separated from each other.

At the same time, the number of bits used for RSA keys is increased from
1024 to 2048.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>