This patch fixes a couple of issues with the job listing:
  - in case of a non-existing job, nicely raise 404 instead of 500
  - in the job detail listing, also list the job log, the job
    timestamps, etc.
  - the opcode migrate instance was missing its description field

Reviewed-by: imsnah

This patch fixes the SSL mode (by actually constructing SSL parameters
from the command line options) and enables SSL by default; the old “-S”
option which enabled SSL is now changed to “--no-ssl”. The certificate
and key are by default pointing to the Ganeti auto-generated certificate
for rapi.

Reviewed-by: imsnah

The start_action function is changed so that it can be called with
arguments - this could be used to parse a defaults file, etc.

Reviewed-by: imsnah

With this parameters VNC for KVM is able to be protected by tls,
optionally with an x509 certificate, and optionally verifying the
client as well. Additionally in this patch we limit the bind address to
being a directory, rather than a file or a directory, for simplicity, as
it allows for the same level of control anyway.

Reviewed-by: iustinp

Before we forced the VNC_BIND_ADDRESS to be an ip. Now we also accept a
path, and bind the instance to it, or to a file in it if it's a
directory.

Reviewed-by: iustinp

This patch converts some more jobs with only queries into cheaper luxi
queries (no job created), and fixes some fallout from the lockless
queries changes.

Reviewed-by: ultrotter

This patch fixes the RAPI QA tests to work with today's RAPI code and
also does some other minor improvements:
  - QA: only create the cluster if so configured (‘create-cluster’ key),
    this allows running parts of the QA suite against existing clusters
  - export the “hvparams” for instances in RAPI

Reviewed-by: imsnah

This patch fixes the 'bulk' parameter (before any non-empty
specification was considered True, in conflict with the documentation,
i.e. bulk=0 still did bulk queries).

The patch also adds optional locking on the instance/node listing (does
not have effect when we only list names).

Reviewed-by: imsnah

This patch cleans up and updates the RAPI interface:
  - queries are changes to luxi queries instead of jobs, where possible
  - since we changed the API version, we remove the old-style attributes
    (sda_size, ip, etc.) and replace them with 2.0 style
  - a small optimization in the instance and node list, don't query
    twice the names in bulk output
  - switch the instance and node lists to no locking

Reviewed-by: imsnah

Similar to the instance list, this patch enables lockless node queris.
“gnt-node list” accepts now the “--sync” flag which enables locking, the
default is lockless.

Reviewed-by: imsnah

For queries, we don't want to require authentication. We fix this by adding an
override GetAuthRealm in the rapi daemon.

We also fix a method name.

Reviewed-by: imsnah

This is the last query that RAPI executes via opcodes and is purely
static (config values only). As such, we can convert it safely to a
query instead of job.

Reviewed-by: imsnah

This patch adds the online node list and instance list to the ssconf
keys. In order to do distribute correctly the instance list, we need to
update the cluster serial number on instance additions and removals.

The patch also changes the permissions on the ssconf files to be 0444:
  - no write for root, in order to signal that these file should not be
    modified
  - read for everyone since the files don't contain sensitive data
    anymore (and permissions can be controlled via the parent directory
    if needed)

The patch also fixes a small typo on gnt-cluster.

Reviewed-by: ultrotter

This patch adds the framework for, and enables lockless OpQueryInstances. This
means that instances will be shown in ERROR_up or ERROR_down state, even though
this is not an error (but just an in-progress job).

The framework is implemented as follows:
  - the OpQueryInstances, OpQueryNodes and OpQueryExports opcodes take
    an additional “use_locking” flag which will denote whether to lock
    or not; this patch only implements this for LUQueryInstances
  - the luxi query functions take an additional argument use_locking
    which is passed to the master daemon, and then passed to the above
    opcodes
  - cli.py export a new SYNC_OPT command line options which implement
    setting this flag to true
  - except for gnt-instance list, which uses this option, and for
    name-only queries (e.g. QueryNodes(fields=["names"])), all other
    callers are setting this flag to True
  - RAPI also sets the flag to True

The patch was tested with a continuous (0.2s sleep in-between)
gnt-instance list during a burnin, and no problems were observed.

Reviewed-by: ultrotter

Or actually more so. If this function gets called while instances get
shut down, it might try to report information on instances which don't
exits. Try to fail gracefully if that happens, by just skipping an
instance which has disappeared in the meantime.

Reviewed-by: iustinp

Reviewed-by: iustinp

Due to the fact that we sanitize now the output from environment
scripts, the unittest needs to be adjusted. My bad for not checking it.

Reviewed-by: imsnah

This is a partial implementation of fully automated node evacuation:
we allow passing an iallocator and all instance replace-disks will be
execute via that iallocator.

The individual OpReplaceDisks opcodes are submitted in a single job,
which causes them to be executed serially and thus keeps the iallocator
runs consistent. This also changes the behaviour so that the first
reallocation that failed will stop all the reallocations.

Reviewed-by: ultrotter

This is a (modified) forward-port of commit 1190 on the 1.2 branch:

  This is the same as gnt-node failover, and is also a cut&paste of its
  code (almost). It will be really really useful to quickly empty a
  healthy node. I can be persuaded to merge MigrateNode and FailoverNode
  in a common codebase, but could also forget about it and submit it if
  nobody cares.

  Reviewed-by: iustinp

The original MigrateNode function has been converted to the 2.0 style
(cli.JobExecutor). Also commit 2076 has been added that fixes a missing
opcode parameter.

Original-author: ultrotter
Reviewed-by: ultrotter

This patch unifies the hardcoded re-encoding attempts into a single
function in utils.py. This function is used to take either an unicode or
str object and convert it to a ASCII-only str object which can be safely
displayed and transmitted.

We replace then the current manual re-encodings with this function. In
mcpu we stop re-encoding the hooks output and instead we do it right at
the hook generation in backend.py.

This passes on my 'custom' lvs output with non-ASCII chars. But there
are probably other places we will need to fix.

Reviewed-by: ultrotter

For testing or just in case a device is exported by a bad driver with
the 'removable' flag set, this patch adds a flag to lvmstrap that allows
it to use these devices too.

Reviewed-by: ultrotter

This patch updates the gnt-os man page and the common footer page for
ganeti 2.0.

Reviewed-by: ultrotter

This small path hopefully fixes the handling of ssh verify errors in
node add (note: untested).

Reviewed-by: ultrotter

In case we fail without output from the ssh command, we should at least
add the exit code or any other failure reason to the error message, and
log it and the cmdline used to the node daemon log.

Reviewed-by: imsnah

Reviewed-by: iustinp

This patch correctly exports the mode of disks (rw/ro) and also exports
the instance OS.

Reviewed-by: imsnah

This patch only changes some indentation in constants.py.

Reviewed-by: imsnah

When the rpc call node_add fails, we don't have any error message. This
patch changes the call to return (status, data) so that the user can see
the correct error message.

Reviewed-by: imsnah

Since parameters get set to False if a no_ is prefixed don't try to
interpret those boolean values, and pass them unchanged.

Reviewed-by: iustinp

We don't need to show hvparams for hypervisors which are not enabled on
the cluster.

Reviewed-by: iustinp

Reviewed-by: iustinp

We'll also enable a tablet usb device, as suggested by the kvm man page.

Reviewed-by: iustinp

Reviewed-by: iustinp

Before we used to copy the file if xen-hvm was enabled on the cluster,
no we'll do that if any enabled hypervisor is in the new HTS_USE_VNC
group.

Reviewed-by: iustinp

HT_KVM doesn't technically require a port, but if it has one it can give
vnc displays to instances.

Reviewed-by: iustinp

Under KVM we don't strictly need a kernel and initrd. If some are passed
we'll use them, otherwise the guest OS will need to behave as fully
native, and have its own boot loader and kernel. 
The root_path hypervisor parameter becomes mandatory only if a kernel is
specified.

Reviewed-by: iustinp

Up until now a KVM instance was forced to have a serial port.
With this change this is no longer mandatory, by default we'll use one,
but if the HV_SERIAL_CONSOLE parameter is set to False we'll do without.

Reviewed-by: iustinp

Sometimes the hypervisor will use the instance hv and/or be parameters
to determine the best shell command. This is not possible, though,
currently, as the instance hv/beparams are not filled, so we have to
pass the filled versions separately.

Reviewed-by: iustinp

Currently the LUVerifyCluster only reports the protocol version changes,
not software ones. This is useful to know/monitor, so we add this too as
a warning.

Reviewed-by: ultrotter

Currently gnt-instance list will refuse to take arguments, and always
return the full list of instances. This patch allows it to pass names to
LUQueryInstances, so that we restrict the input to a given set of
instances.

Reviewed-by: ultrotter