This patch fixes the error handling in the add OS to instance function
with regard to invalid OSes. Previously, we didn't handle any such
errors, with the end result that the user would have to look in the node
daemon log.

The patch also renames the name of the function to match the RPC call
name.

Reviewed-by: ultrotter

Currently the return value or errors from the block device Open() method
are ignored. This patch catches any BlockDeviceErrors and returns a
well-formatted result.

Reviewed-by: ultrotter

By using the RemoteFailMsg() or the payload field of RpcResult, we can
simplify a few functions in cmdlib.

Reviewed-by: ultrotter

For results which use the (status, payload) response type, it's easier
to define a ‘payload’ field on the result holding the payload than to
extract it using “data[1]” in the caller code.

Reviewed-by: ultrotter

In lockless queries, it's better if we see the instance in ADMIN_down
rather than ERROR_down during the time it's installed. As such, we
change the LU to only mark the instance 'up' at the time we are ready to
start it.

Reviewed-by: ultrotter

Apparently it's not supported. Also add -boot command line parameters
to kvm, since they seem to help booting from the right place. Everything
will still only work when not using a kernel, but well... :)

Reviewed-by: iustinp

Reviewed-by: iustinp

The cdrom image has the same meaning than in Xen HVM, and so does
boot_order, even though it has a slightly different syntax, and uses the
value 'disk' too boot from disk and 'cdrom' to boot from cdrom.

Reviewed-by: iustinp

Confusingly, as a leftober from 1.2, there was a
constants.HT_HVM_DEFAULT_BOOT_ORDER constant, with a value opposite to
the default HV_BOOT_ORDER hv param that got enabled only if
HV_BOOT_ORDER was set to None. Since setting it to None is very
hard/impossible for the user, and we didn't handle other "empty" values
(False, ''), we'll just force the parameter to have a valid value (after
all we have a default, and that's the way we use hvparams) and get rid
of the old constant altoghether.

Reviewed-by: iustinp

Since we by default now use SSL for RAPI, we need to switch the QA
tests to SSL too.

Reviewed-by: amishchenko

This patch fixes a couple of issues with the job listing:
  - in case of a non-existing job, nicely raise 404 instead of 500
  - in the job detail listing, also list the job log, the job
    timestamps, etc.
  - the opcode migrate instance was missing its description field

Reviewed-by: imsnah

This patch fixes the SSL mode (by actually constructing SSL parameters
from the command line options) and enables SSL by default; the old “-S”
option which enabled SSL is now changed to “--no-ssl”. The certificate
and key are by default pointing to the Ganeti auto-generated certificate
for rapi.

Reviewed-by: imsnah

The start_action function is changed so that it can be called with
arguments - this could be used to parse a defaults file, etc.

Reviewed-by: imsnah

With this parameters VNC for KVM is able to be protected by tls,
optionally with an x509 certificate, and optionally verifying the
client as well. Additionally in this patch we limit the bind address to
being a directory, rather than a file or a directory, for simplicity, as
it allows for the same level of control anyway.

Reviewed-by: iustinp

Before we forced the VNC_BIND_ADDRESS to be an ip. Now we also accept a
path, and bind the instance to it, or to a file in it if it's a
directory.

Reviewed-by: iustinp

This patch converts some more jobs with only queries into cheaper luxi
queries (no job created), and fixes some fallout from the lockless
queries changes.

Reviewed-by: ultrotter

This patch fixes the RAPI QA tests to work with today's RAPI code and
also does some other minor improvements:
  - QA: only create the cluster if so configured (‘create-cluster’ key),
    this allows running parts of the QA suite against existing clusters
  - export the “hvparams” for instances in RAPI

Reviewed-by: imsnah

This patch fixes the 'bulk' parameter (before any non-empty
specification was considered True, in conflict with the documentation,
i.e. bulk=0 still did bulk queries).

The patch also adds optional locking on the instance/node listing (does
not have effect when we only list names).

Reviewed-by: imsnah

This patch cleans up and updates the RAPI interface:
  - queries are changes to luxi queries instead of jobs, where possible
  - since we changed the API version, we remove the old-style attributes
    (sda_size, ip, etc.) and replace them with 2.0 style
  - a small optimization in the instance and node list, don't query
    twice the names in bulk output
  - switch the instance and node lists to no locking

Reviewed-by: imsnah

Similar to the instance list, this patch enables lockless node queris.
“gnt-node list” accepts now the “--sync” flag which enables locking, the
default is lockless.

Reviewed-by: imsnah

For queries, we don't want to require authentication. We fix this by adding an
override GetAuthRealm in the rapi daemon.

We also fix a method name.

Reviewed-by: imsnah

This is the last query that RAPI executes via opcodes and is purely
static (config values only). As such, we can convert it safely to a
query instead of job.

Reviewed-by: imsnah

This patch adds the online node list and instance list to the ssconf
keys. In order to do distribute correctly the instance list, we need to
update the cluster serial number on instance additions and removals.

The patch also changes the permissions on the ssconf files to be 0444:
  - no write for root, in order to signal that these file should not be
    modified
  - read for everyone since the files don't contain sensitive data
    anymore (and permissions can be controlled via the parent directory
    if needed)

The patch also fixes a small typo on gnt-cluster.

Reviewed-by: ultrotter

This patch adds the framework for, and enables lockless OpQueryInstances. This
means that instances will be shown in ERROR_up or ERROR_down state, even though
this is not an error (but just an in-progress job).

The framework is implemented as follows:
  - the OpQueryInstances, OpQueryNodes and OpQueryExports opcodes take
    an additional “use_locking” flag which will denote whether to lock
    or not; this patch only implements this for LUQueryInstances
  - the luxi query functions take an additional argument use_locking
    which is passed to the master daemon, and then passed to the above
    opcodes
  - cli.py export a new SYNC_OPT command line options which implement
    setting this flag to true
  - except for gnt-instance list, which uses this option, and for
    name-only queries (e.g. QueryNodes(fields=["names"])), all other
    callers are setting this flag to True
  - RAPI also sets the flag to True

The patch was tested with a continuous (0.2s sleep in-between)
gnt-instance list during a burnin, and no problems were observed.

Reviewed-by: ultrotter

Or actually more so. If this function gets called while instances get
shut down, it might try to report information on instances which don't
exits. Try to fail gracefully if that happens, by just skipping an
instance which has disappeared in the meantime.

Reviewed-by: iustinp

Reviewed-by: iustinp

Due to the fact that we sanitize now the output from environment
scripts, the unittest needs to be adjusted. My bad for not checking it.

Reviewed-by: imsnah

This is a partial implementation of fully automated node evacuation:
we allow passing an iallocator and all instance replace-disks will be
execute via that iallocator.

The individual OpReplaceDisks opcodes are submitted in a single job,
which causes them to be executed serially and thus keeps the iallocator
runs consistent. This also changes the behaviour so that the first
reallocation that failed will stop all the reallocations.

Reviewed-by: ultrotter

This is a (modified) forward-port of commit 1190 on the 1.2 branch:

  This is the same as gnt-node failover, and is also a cut&paste of its
  code (almost). It will be really really useful to quickly empty a
  healthy node. I can be persuaded to merge MigrateNode and FailoverNode
  in a common codebase, but could also forget about it and submit it if
  nobody cares.

  Reviewed-by: iustinp

The original MigrateNode function has been converted to the 2.0 style
(cli.JobExecutor). Also commit 2076 has been added that fixes a missing
opcode parameter.

Original-author: ultrotter
Reviewed-by: ultrotter

This patch unifies the hardcoded re-encoding attempts into a single
function in utils.py. This function is used to take either an unicode or
str object and convert it to a ASCII-only str object which can be safely
displayed and transmitted.

We replace then the current manual re-encodings with this function. In
mcpu we stop re-encoding the hooks output and instead we do it right at
the hook generation in backend.py.

This passes on my 'custom' lvs output with non-ASCII chars. But there
are probably other places we will need to fix.

Reviewed-by: ultrotter

For testing or just in case a device is exported by a bad driver with
the 'removable' flag set, this patch adds a flag to lvmstrap that allows
it to use these devices too.

Reviewed-by: ultrotter

This patch updates the gnt-os man page and the common footer page for
ganeti 2.0.

Reviewed-by: ultrotter

This small path hopefully fixes the handling of ssh verify errors in
node add (note: untested).

Reviewed-by: ultrotter

In case we fail without output from the ssh command, we should at least
add the exit code or any other failure reason to the error message, and
log it and the cmdline used to the node daemon log.

Reviewed-by: imsnah

Reviewed-by: iustinp

This patch correctly exports the mode of disks (rw/ro) and also exports
the instance OS.

Reviewed-by: imsnah

This patch only changes some indentation in constants.py.

Reviewed-by: imsnah

When the rpc call node_add fails, we don't have any error message. This
patch changes the call to return (status, data) so that the user can see
the correct error message.

Reviewed-by: imsnah

Since parameters get set to False if a no_ is prefixed don't try to
interpret those boolean values, and pass them unchanged.

Reviewed-by: iustinp

We don't need to show hvparams for hypervisors which are not enabled on
the cluster.

Reviewed-by: iustinp