This patch adds the online node list and instance list to the ssconf
keys. In order to do distribute correctly the instance list, we need to
update the cluster serial number on instance additions and removals.

The patch also changes the permissions on the ssconf files to be 0444:
  - no write for root, in order to signal that these file should not be
    modified
  - read for everyone since the files don't contain sensitive data
    anymore (and permissions can be controlled via the parent directory
    if needed)

The patch also fixes a small typo on gnt-cluster.

Reviewed-by: ultrotter

This patch adds the framework for, and enables lockless OpQueryInstances. This
means that instances will be shown in ERROR_up or ERROR_down state, even though
this is not an error (but just an in-progress job).

The framework is implemented as follows:
  - the OpQueryInstances, OpQueryNodes and OpQueryExports opcodes take
    an additional “use_locking” flag which will denote whether to lock
    or not; this patch only implements this for LUQueryInstances
  - the luxi query functions take an additional argument use_locking
    which is passed to the master daemon, and then passed to the above
    opcodes
  - cli.py export a new SYNC_OPT command line options which implement
    setting this flag to true
  - except for gnt-instance list, which uses this option, and for
    name-only queries (e.g. QueryNodes(fields=["names"])), all other
    callers are setting this flag to True
  - RAPI also sets the flag to True

The patch was tested with a continuous (0.2s sleep in-between)
gnt-instance list during a burnin, and no problems were observed.

Reviewed-by: ultrotter

Or actually more so. If this function gets called while instances get
shut down, it might try to report information on instances which don't
exits. Try to fail gracefully if that happens, by just skipping an
instance which has disappeared in the meantime.

Reviewed-by: iustinp

Reviewed-by: iustinp

Due to the fact that we sanitize now the output from environment
scripts, the unittest needs to be adjusted. My bad for not checking it.

Reviewed-by: imsnah

This is a partial implementation of fully automated node evacuation:
we allow passing an iallocator and all instance replace-disks will be
execute via that iallocator.

The individual OpReplaceDisks opcodes are submitted in a single job,
which causes them to be executed serially and thus keeps the iallocator
runs consistent. This also changes the behaviour so that the first
reallocation that failed will stop all the reallocations.

Reviewed-by: ultrotter

This is a (modified) forward-port of commit 1190 on the 1.2 branch:

  This is the same as gnt-node failover, and is also a cut&paste of its
  code (almost). It will be really really useful to quickly empty a
  healthy node. I can be persuaded to merge MigrateNode and FailoverNode
  in a common codebase, but could also forget about it and submit it if
  nobody cares.

  Reviewed-by: iustinp

The original MigrateNode function has been converted to the 2.0 style
(cli.JobExecutor). Also commit 2076 has been added that fixes a missing
opcode parameter.

Original-author: ultrotter
Reviewed-by: ultrotter

This patch unifies the hardcoded re-encoding attempts into a single
function in utils.py. This function is used to take either an unicode or
str object and convert it to a ASCII-only str object which can be safely
displayed and transmitted.

We replace then the current manual re-encodings with this function. In
mcpu we stop re-encoding the hooks output and instead we do it right at
the hook generation in backend.py.

This passes on my 'custom' lvs output with non-ASCII chars. But there
are probably other places we will need to fix.

Reviewed-by: ultrotter

For testing or just in case a device is exported by a bad driver with
the 'removable' flag set, this patch adds a flag to lvmstrap that allows
it to use these devices too.

Reviewed-by: ultrotter

This patch updates the gnt-os man page and the common footer page for
ganeti 2.0.

Reviewed-by: ultrotter

This small path hopefully fixes the handling of ssh verify errors in
node add (note: untested).

Reviewed-by: ultrotter

In case we fail without output from the ssh command, we should at least
add the exit code or any other failure reason to the error message, and
log it and the cmdline used to the node daemon log.

Reviewed-by: imsnah

Reviewed-by: iustinp

This patch correctly exports the mode of disks (rw/ro) and also exports
the instance OS.

Reviewed-by: imsnah

This patch only changes some indentation in constants.py.

Reviewed-by: imsnah

When the rpc call node_add fails, we don't have any error message. This
patch changes the call to return (status, data) so that the user can see
the correct error message.

Reviewed-by: imsnah

Since parameters get set to False if a no_ is prefixed don't try to
interpret those boolean values, and pass them unchanged.

Reviewed-by: iustinp

We don't need to show hvparams for hypervisors which are not enabled on
the cluster.

Reviewed-by: iustinp

Reviewed-by: iustinp

We'll also enable a tablet usb device, as suggested by the kvm man page.

Reviewed-by: iustinp

Reviewed-by: iustinp

Before we used to copy the file if xen-hvm was enabled on the cluster,
no we'll do that if any enabled hypervisor is in the new HTS_USE_VNC
group.

Reviewed-by: iustinp

HT_KVM doesn't technically require a port, but if it has one it can give
vnc displays to instances.

Reviewed-by: iustinp

Under KVM we don't strictly need a kernel and initrd. If some are passed
we'll use them, otherwise the guest OS will need to behave as fully
native, and have its own boot loader and kernel. 
The root_path hypervisor parameter becomes mandatory only if a kernel is
specified.

Reviewed-by: iustinp

Up until now a KVM instance was forced to have a serial port.
With this change this is no longer mandatory, by default we'll use one,
but if the HV_SERIAL_CONSOLE parameter is set to False we'll do without.

Reviewed-by: iustinp

Sometimes the hypervisor will use the instance hv and/or be parameters
to determine the best shell command. This is not possible, though,
currently, as the instance hv/beparams are not filled, so we have to
pass the filled versions separately.

Reviewed-by: iustinp

Currently the LUVerifyCluster only reports the protocol version changes,
not software ones. This is useful to know/monitor, so we add this too as
a warning.

Reviewed-by: ultrotter

Currently gnt-instance list will refuse to take arguments, and always
return the full list of instances. This patch allows it to pass names to
LUQueryInstances, so that we restrict the input to a given set of
instances.

Reviewed-by: ultrotter

Currently LUQueryInstances keeps the ordering of instances only in some cases,
and in others it will reorder the list. This patch fixes this by more clearly
separating the various cases (names passed or not and locking or not locking),
so that the output list is in the same order as always.

Of course, this disables the sorting when arguments are passed.

Reviewed-by: ultrotter

Currently LockSet.acquire() sorts in place it's input argument if it's a
list. This is not good, since callers might depend on a specific
ordering of the input data, and this is a 'hidden' modification.

We fix it by simply using a sorted copy, instead of sorting in place.

Reviewed-by: ultrotter

This non-code change rewraps some lines in locking.py to keep them under
80 chars.

Reviewed-by: ultrotter

Currently we ask the user for confirmation, and only after (try to)
remove, failover or migrate the instance. This doesn't work nicely if
the instance doesn't exist, so we make a query for the instance before
the prompt, which will throw an error in case it doesn't exist.

Side-note: the way the query works today is not really nice. It would be
better if we could query explicitly for a missing instance name, so that
this is done cleaner (explicit check) instead of side-effect (throw
exception). We do add code for this explicit check, except that today it
won't be used actually.

Reviewed-by: ultrotter

Generalize tag work for instances/nodes/cluster tag management.

Reviewed-by: iustinp

The resources we still need moved to rlib2.

Reviewed-by: iustinp

Reviewed-by: iustinp

It is impossible to keep backward compatibility due to
significant changes in the Ganeti core.

Reviewed-by: iustinp

Reviewed-by: ultrotter

This patch correctly marks the drives as read-only for Xen, and raises
and exception for KVM since it doesn't support read-only drives.

Reviewed-by: ultrotter

This patch fixes two issues with the cancel mechanism:
  - cancelled jobs show as such, and not in error state (we mark them as
    OP_STATUS_CANCELED and not OP_STATUS_ERROR)
  - queued jobs which are cancelled don't raise errors in the master (we
    treat OP_STATUS_CANCELED now)

Reviewed-by: imsnah

Also raise HypervisorError rather than OpExecError.

Reviewed-by: iustinp