Reviewed-by: iustinp

We'll also enable a tablet usb device, as suggested by the kvm man page.

Reviewed-by: iustinp

Reviewed-by: iustinp

Before we used to copy the file if xen-hvm was enabled on the cluster,
no we'll do that if any enabled hypervisor is in the new HTS_USE_VNC
group.

Reviewed-by: iustinp

HT_KVM doesn't technically require a port, but if it has one it can give
vnc displays to instances.

Reviewed-by: iustinp

Under KVM we don't strictly need a kernel and initrd. If some are passed
we'll use them, otherwise the guest OS will need to behave as fully
native, and have its own boot loader and kernel. 
The root_path hypervisor parameter becomes mandatory only if a kernel is
specified.

Reviewed-by: iustinp

Up until now a KVM instance was forced to have a serial port.
With this change this is no longer mandatory, by default we'll use one,
but if the HV_SERIAL_CONSOLE parameter is set to False we'll do without.

Reviewed-by: iustinp

Sometimes the hypervisor will use the instance hv and/or be parameters
to determine the best shell command. This is not possible, though,
currently, as the instance hv/beparams are not filled, so we have to
pass the filled versions separately.

Reviewed-by: iustinp

Currently the LUVerifyCluster only reports the protocol version changes,
not software ones. This is useful to know/monitor, so we add this too as
a warning.

Reviewed-by: ultrotter

Currently gnt-instance list will refuse to take arguments, and always
return the full list of instances. This patch allows it to pass names to
LUQueryInstances, so that we restrict the input to a given set of
instances.

Reviewed-by: ultrotter

Currently LUQueryInstances keeps the ordering of instances only in some cases,
and in others it will reorder the list. This patch fixes this by more clearly
separating the various cases (names passed or not and locking or not locking),
so that the output list is in the same order as always.

Of course, this disables the sorting when arguments are passed.

Reviewed-by: ultrotter

Currently LockSet.acquire() sorts in place it's input argument if it's a
list. This is not good, since callers might depend on a specific
ordering of the input data, and this is a 'hidden' modification.

We fix it by simply using a sorted copy, instead of sorting in place.

Reviewed-by: ultrotter

This non-code change rewraps some lines in locking.py to keep them under
80 chars.

Reviewed-by: ultrotter

Currently we ask the user for confirmation, and only after (try to)
remove, failover or migrate the instance. This doesn't work nicely if
the instance doesn't exist, so we make a query for the instance before
the prompt, which will throw an error in case it doesn't exist.

Side-note: the way the query works today is not really nice. It would be
better if we could query explicitly for a missing instance name, so that
this is done cleaner (explicit check) instead of side-effect (throw
exception). We do add code for this explicit check, except that today it
won't be used actually.

Reviewed-by: ultrotter

Generalize tag work for instances/nodes/cluster tag management.

Reviewed-by: iustinp

The resources we still need moved to rlib2.

Reviewed-by: iustinp

Reviewed-by: iustinp

It is impossible to keep backward compatibility due to
significant changes in the Ganeti core.

Reviewed-by: iustinp

Reviewed-by: ultrotter

This patch correctly marks the drives as read-only for Xen, and raises
and exception for KVM since it doesn't support read-only drives.

Reviewed-by: ultrotter

This patch fixes two issues with the cancel mechanism:
  - cancelled jobs show as such, and not in error state (we mark them as
    OP_STATUS_CANCELED and not OP_STATUS_ERROR)
  - queued jobs which are cancelled don't raise errors in the master (we
    treat OP_STATUS_CANCELED now)

Reviewed-by: imsnah

Also raise HypervisorError rather than OpExecError.

Reviewed-by: iustinp

Also raise HypervisorError rather than OpExecError.

Reviewed-by: iustinp

This patch adds a simple check that the 'mode' attribute of top-level disks is
correct. It does not recurse over children.

The framework could be extended with other checks in the future.

Reviewed-by: imsnah

Currently, only the LUSetInstanceParams correctly sets up the mode
attribute via a manual operation. We remove this and instead do the
correct setting in the generic _GenerateDiskTemplate function, so that
we set the mode correctly for all disk creations.

Reviewed-by: ultrotter

This patch changes the multi-instance gnt-* commands (gnt-instance
start/stop, gnt-node evacuate/failover) such that the individual
operations are submitted in parallel, ideally improving the speed of the
execution.

The patch does this by abstracting the job set functionality into a new
class in cli.py, that takes care of the job submit, job poll and error
handling.

Reviewed-by: ultrotter

This is a simply typo from the conversion to multi-job archiving.

Reviewed-by: imsnah

This parameter allows a different path to be passed to the instance
kernel. The new parameter is mandatory, and by default has the value of
the old hardcoded value for both kvm and xen.

Beta1 clusters will need to have this parameter added for their
instances to be able to boot.

Reviewed-by: iustinp

This is a class method, because it calls _InstanceSerial, which is
another class method. The patch changes it to classmethod for all the
hypervisor classes.

Reviewed-by: iustinp

Those methods need nothing from the instantiated class, and just
manipulate strings, and fetch some class global variables, so they can
be classmethods.

Reviewed-by: iustinp

Even though alpha started at 0, we release beta 1 first as we did for
1.2.

Reviewed-by: imsnah, ultrotter

Also import the NEWS entries from the 1.2 branch which were added since
we created it.

Reviewed-by: ultrotter

A missing 'be' was present in the error string for both xen and kvm,
when the kernel or initrd path was not absolute.

Reviewed-by: imsnah

In case we submit multiple instances via batcher, it's nicer to have the
sorted nicely.

Reviewed-by: imsnah

This patch fixes the gnt-instance batch-create command, and in doing so
also slightly changes two other functions:
  - we change utils.ParseUnit so that it accepts integer values also
    (both ParseUnit(5) and ParseUnit("5") return the same value)
  - a bridge 'None' in LUCreateInstance will be converted to the default
    bridge; currently only missing bridges will be accepted to mean the
    default one

The main changes to batcher were the change to variable number of disks
and NICs.

The patch also adds a batcher-instances.json example file copied from
the 1.2 branch and properly modified.

Reviewed-by: imsnah, killerfoxi

This patch changes the iallocator framework to work with and properly
export to plugins offline nodes. It does this by only exporting the
static configuration data for those nodes, and not attempting to parse
the runtime data.

The patch also fixes bugs in iallocator related to the RpcResult
conversion, changes the should_run to admin_up attribute name (as per
the internals change), and adds “-I” as a short option for
“--iallocator” in gnt-instance, gnt-backup and burnin.

Reviewed-by: ultrotter

Currently the DRBD code checks that the metadata devices are valid
before creation, initial disk attachment and add children.

However, the process for checking validity requires a free DRBD minor,
and this conflict with parallel checking.

There are at least three possible solutions:
  - serialize all checks, which means we reduce parallelism and need
    extra locks
  - don't pass a valid minor number, but one like “/dev/drbd256” (which
    is invalid); this works for current version of DRBD, but since it's
    not guaranteed to remain so it doesn't look nice
  - don't do the checking at all, and rely on “drbdsetup ... disk ...”
    to fail by itself

The reason for checking metadata was that in 1.2, this was much cheaper
than trying to activate devices (and the subsequent iteration over the
minors). However, in 2.0, they have the same cost, so we can choose
option 3: just remove the explicit checking and rely on drbdsetup and
the kernel to fail.

Since DRBD8._InitMeta still requires a minor number, the two places
where this is run are handled as follows:
  - Create: we just use our own (unused currently) minor number
  - AddChildren: we keep using FindUnusedMinor, with the caveat that
    this function (used by replace-disks -n ...) cannot be yet
    parallelized

Reviewed-by: ultrotter

This patch changes (significantly) the execution model in burnin:
  - for all runs, (almost) all instance mods in a single Burn* procedure
    are done as part of a job; so for example add disk, stop, remove
    disk, start are no longer done as separate jobs but as a single job
    consisting of four opcodes
  - for parallel runs, all Burn* procedures except the rename (which
    uses a single target name) run in parallel; before, only the
    creation was done in parallel
  - due to the single-job execution and also parallel execution, the
    logging messages are no longer happening synchronously with the
    execution, so they are more informative than an actual execution log

The end result is that burnin now tests properly multi-opcode jobs and
also tests all opcodes (except rename) for parallel execution.

Note: On a test cluster, parallelization reduces burnin time from 23m to
15m.

Reviewed-by: ultrotter

Currently the restrictions are too harsh: there is a time interval
between an instance gets a new disk and before it is added to the
configuration in which the restriction is not met. We solve this by
allowing temporary DRBD minors to match existing minors (for the same
instance), such that parallel creations/minor allocations are OK.

The change is done by moving the add of temporary minors to the
minor map after the instance minors are computed, and only considering
them as duplicate if the instance name doesn't match.

Reviewed-by: ultrotter

This patch enhances the duplicate DRBD minors checks (currently just a
few) and adds automatic checks of configuration consistency at
configuration file writing time.

In order to do so and show meaningful error messages, the
_UnlockedComputeDRBDMap function is changed to not raise errors in case
of duplicates, but instead return both the minors map and the duplicate
list, and its callers now raise the error. This allows the VerifyConfig
function to return a complete list of duplicates.

The new checks required some small updates to the unittests for the
config module.

Reviewed-by: ultrotter