* origin/devel-2.1: (116 commits)
  Implement replacing cluster certs and keys via “gnt-cluster renew-crypto”
  cli: Add helper function to stop and start whole cluster
  cfgupgrade: Use new bootstrap function for certs and keys
  bootstrap: Add new function to create cluster certs and keys
  utils.CreateBackup: Use human-readable instead of seconds since Epoch
  Add unittest for daemon-util
  Add support for non-Python unittests
  daemon-util: Generate daemon path in separate function
  daemon-util: Use “return” instead of “exit” in all functions
  daemon-util: Add function to start and stop all daemons
  ganeti.initd: Move all daemon names from init script to daemon-util
  ganeti.initd: Move code checking daemon exit code to daemon-util
  ganeti.initd: Move code checking config to daemon-util
  daemon-util: Require dashes in commands
  Improve ganeti.serializer unittests
  Add unittests for ganeti.errors
  Verify cluster certificates in LUVerifyCluster
  utils: Add function to extract X509 cert validity
  Add constant with cluster X509 certificates
  Release version 2.1.1
  ...

Conflicts:
	lib/backend.py: Trivial
	lib/bootstrap.py: Trivial
	lib/constants.py: Trivial
	lib/http/server.py: Trivial
	lib/utils.py: RunCmd parameter “reset_env”
	test/ganeti.utils_unittest.py: Trivial
	tools/cfgupgrade: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Recent changes to “gnt-cluster verify” made it complain on expiring SSL
certificates. While it was possible to replace the SSL certificates and
other cluster secrets manually before, doing so was cumbersome. Cluster
certificates, keys and secrets can now be replaced easily.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Replacing cluster certificates and keys requires all cluster daemons to be
shut down. This might also be handy for the cluster merger tool, though
the function might need a few more extensions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The code to generate cluster certificates, keys and secrets is currently
spread over several places. It makes sense to move it to a separate
function as we want to provide the user with the ability to automatically
replace all cluster certificates and keys.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Seconds since the Epoch are not easily readable by a human. Using a
formatted timestamp makes it easier (e.g.
“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles use
this formatted timestamp.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This test doesn't cover everything, but it's better than nothing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is important if they're called directly within daemon-util.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is, to some degree, duplicated code from the init script. However,
the init script has to conform to standards of the underlying Linux
distributions, while these functions will be called by Ganeti itself. By
moving more code into daemon-util, the amount of duplication has been
reduced.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The list of daemon names will be used in daemon-util, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is again for re-using code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This allows for more code re-use. daemon-util will also be used to start
all daemons.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Even though the script uses underscores (_) internally, the external
commands are supposed to be written using dashes (-).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When using pyOpenSSL 0.7 or above, LUClusterVerify will start to show a
warning 30 days before a certificate expires. 7 days before the
certificate expires, the warning becomes an error. Once expired,
LUVerifyCluster will always report an error. The latter is also supported
with pyOpenSSL 0.6.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.
pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and
“notAfter” as strings. Parsing them turned out to be a challenge since
they can be in a variety of formats (YYYYMMDDhhmmssZ, YYYYMMDDhhmmss+hhmm
or YYYMMDDhhmmss-hhmm).

This will be used to verify the validity of cluster certificates in
LUVerifyCluster.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.1:
  Release version 2.1.1
  Update NEWS file for the 2.1.1 release
  Validate the os-specific hypervisor parameters

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In case the hypervisor has issues on one node, currently
backend.VerifyNode will exit via an exception (two exit paths possible,
one via HypervisorError from hypervisor.Verify(), and one via RPCFail
from GetInstanceList). This is bad as it invalidates all other checks of
that node.

This patch catches these two errors and allows the rest of the
VerifyNode function to run. This leads to a more complete verify cluster
run, for example now only real missing LVs are reported, not all of
them.

The cluster verify is not perfect as it will skip some tests even if it
has data, but this will require a more complete rewrite (see issue 90).

Also, the patch fixes and improves some error messages in cmdlib.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Sorry…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds basic qa tests for gnt-os modify

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Burnin has a custom job executor, because of its need to retry some job
series.

While we cannot replace all of it, at least the execution we can switch
to cli.JobExecutor, to take advantage of the recently-introduced
out-of-order waiting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… rather than submission order. The results are still returned in the
submission order, and for this we needed to track internally the index
of the submission.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is similar to the Haskell function, except that the signature is
reverse to match the 'any' and 'all' Python functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This makes the Log function able to take multiple args for simplified
message construction, similar to the ToStdout one.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In simulate errors mode, the test "ntime_diff is not None" will be
ignored, and thus a None value will try to be formatted as %.01f. We
workaround this by formatting it before, and then only using %s, which
can format a 'None' value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

We don't need the pwentry when checking if a username exists, just to be
sure the KeyError is not returned. Remove the variable, and thus shut up
lint.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds a validation similar to the one for cluster-wide hypervisor
paramters.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Initially we only support the "user" model (in which the user running
the virtual machine can be specified as an additional parameter).

We use usernames rather than uids in this mode, because the kvm -runas
flag doesn't support uids anyway, and we check the passed username for
validity.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These constants add two new kvm hypervisor parameters, specifying the
security model (user/pool) and the security domain, within that model.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This also adds a large piece of pseudo code for explanatory purposes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds tests for the stacking of objects.Cluster.FillHV to verify
that the override is working as expected.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Passing <quote> rather than ' avoids having special characters at the
beginning of the line, which man doesn't like.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>