Update the design about cluster parameters and provide
some example of backend parameters. By now this includes
all supported parameters.

Reviewed-by: iustinp, ultrotter

This adds the set/reset in the jqueue and luxi modules, and a way to
query it in OpQueryConfigValues, and also the comand line interface for
it:
$ gnt-cluster queue info
The drain flag is unset
$ gnt-cluster queue drain
$ gnt-cluster queue info
The drain flag is set
$ gnt-cluster queue undrain
$ gnt-cluster queue info
The drain flag is unset

The choice of making the setting via luxi and not an opcode is that
opcodes can't be executed when drained, but we don't query via luxi
since in the future it might become a cluster property as opposed to a
node one.

Reviewed-by: imsnah

This change is part of the integration of tools/batcher from
Ganeti 1.2 into Ganeti 2.0 core code. It has a compatible
submission interface to the version from 1.2 with these differences:

* it's integrated into Ganeti directly: gnt-instance batch-create
* iallocator is now provided by the instance specs
* log is now held within Ganeti
* no force needed
* no sleep needed
* memory and vcpus goes now into beparams
* missing stuff from the TODO (see below)

Open TODOs:

* Implement notification about creation status/interactive part (top like)
* Backup instance allocation specs?
* Document usage and spec format

Reviewed-by: iustinp

A new multi-node call is added that sets/resets the drain flag.

Reviewed-by: imsnah

We add a (per-node) queue drain flag that blocks new job submission.
There is not yet an interface to add/remove the flag (will come in next
patches).

Reviewed-by: imsnah

This patch adds a generic method to identify the ganeti error given its
class name, and implements this across the luxi protocol.

Reviewed-by: imsnah

Currently this is not derived from GenericError, but there's no reason
for it to be so.

Reviewed-by: imsnah

Reviewed-by: ultrotter

Reviewed-by: ultrotter

- add backend and hypervisor parameters
- fix beparams validation/passing
- pass hypervisor and hvparams
- remove deprecated flags

Reviewed-by: iustinp

ValidateBeParams does not return, but its return value is assigned to a
variable which is never used. Avoid this assignment.

Reviewed-by: iustinp

kernel, initrd, hvm_boot_order and vnc_bind_address are now hypervisor
parameters and should not have their own flag. Moreover querying of
vnc_bind_address should of course pass through the hv/ namespace.

Reviewed-by: iustinp

The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similar
to the one in instance add, and changes the default to wait.

This is cleaner as at the moment when the command returns, we either
have a fully synced disk or there is an error.

This is a forward-port of rev 1183 on the 1.2 branch.

Reviewed-by: ultrotter

With this patch both the os and the swap disk are grown during
burnin. You can pass an increase size of 0 to skip this operation.

and:

burnin: don't try to grow diskless instances

    When burnin is run on a diskless instance, it fails when trying
    GrowDisk, because a non-existant disk cannot be grown. This patch
    disables the test for that disk template.

This is a forward-port of revisions 1181 and 1606 on the 1.2 branch.

Original author: ultrotter
Reviewed-by: ultrotter

I'm not sure which parameters it should use, I took all I saw in the
code. Update: ultrotter says the list is OK for now.

Reviewed-by: ultrotter

Change the constant name to match the value (autobalance ->
auto_balance).

Also add the auto_balance header so that gnt-instance can list it.

Reviewed-by: ultrotter

The secondary nodes should not be checked for free memory in case of non
auto-balanced instances.

This is a forward-port from rev 1567 on the 1.2 branch.

Reviewed-by: ultrotter

This patch changes the cluster verify in two ways:
  - do not use instances marked with auto_balance=False in N+1 memory
    checks
  - report, if any, the number of instances with auto_balance=False

Forward port from rev 1562 on the 1.2 branch.

Reviewed-by: ultrotter

After the locking changes, and the partial locking of nodes, the
instance/node list is not ordered anymore. Reorder so that it behaves as
before.

Reviewed-by: ultrotter

Reviewed-by: ultrotter

This big patch changes the master code to use the beparams. Errors might
have crept in, but it passes a small burnin.

Reviewed-by: ultrotter

The backend.FinalizeExport function is changed to use the beparams
instead of the instance attributes. Future enhancements should be done
in order to export and import/reuse the whole be/hv params.

Reviewed-by: ultrotter

This patch changes the hypervisor functions to use the beparams dict
instead of the instance attributes.

Reviewed-by: ultrotter

As for the hvparams, we need to replace the sent instance beparams.

Reviewed-by: ultrotter

This is used to validate the parameters in both “gnt-instance add” and
“gnt-backup import”.

Reviewed-by: ultrotter

Memory is renamed for easier use.

Reviewed-by: ultrotter

When creating an instance, we need to check not the opcode hvparams, but
the final, filled hvparams against validity. While we do this for the
remote node calls (i.e. ValidateParams), we didn't do this for
CheckParameterSyntax.

Reviewed-by: imsnah

This patch adds a new '-s' parameter to ‘gnt-instance info’ that makes
it return only 'static' information. This is much faster, especially for
drbd instances.

This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending
due to some conflicts.

Reviewed-by: imsnah

Some informations are not printed nicely (e.g. “virtual CDROM: False”),
but this is the first step.

Reviewed-by: imsnah

Reviewed-by: imsnah

This is just a change of the various hvm_ and pvm parameters to the hv
model. Parameters are queried via hv/$name or via the whole dict as
returned by hvparams.

Reviewed-by: ultrotter,imsnah

This big patch changes instance create to the new hvparams structure.
Old parameters are removed, so old jobs or old instances file will break
current clusters.

Reviewed-by: ultrotter

We have a problem with the current model of combining instance lists
from multiple hypervisors: we don't allow duplicates, but "xm list"
gives the same output for both pvm and hvm. This is a lack in the actual
xen hypervisor implementation/split between pvm and hvm, but for now we
implement a weak workaround: identical instance params will be allowed,
and merged. This breaks because there is a delta in listing, and should
be treated as temporary workaround only.

Note that there are two cases for duplicate instance: the above one (xen
is the same, whether pvm or hvm), and the other case, the real error,
when we have two different hypervisors reporting the same instance name.
The latter case needs to be handled better (not by refusing to list the
instances in the backend).

Reviewed-by: ultrotter

We fill the instance hvparams with cluster defaults, and send a modified
dict over the wire to the node methods - they don't know anything about
cluster defaults versus instance parameters.

Reviewed-by: ultrotter

The newly-added node-specific ValidateParams hypervisor method is
exported over RPC, using the semi-standard (success, message) return
value. Multi-node call, so that we call on both primary and secondary at
once.

Reviewed-by: ultrotter

This patch adds the new methods CheckParameterSyntax and the
ValidateParameters in the base and xen hypervisors. It also changes the
xen hypervisor to use the new hvparams field on instance.

Reviewed-by: imsnah

This was a leftover from the early days of the remote API in Ganeti 1.2.

Reviewed-by: ultrotter

This patch adds paramter removal in SplitKeyVal, by prefixing a
value-less key with "-"; this is needed in resetting parameters back to
cluster defaults, but care must be applied now that None can come from
the parser.

Reviewed-by: imsnah

This patch adds the instance and cluster be/hvparams attributes, and the
Fill* methods on the cluster object. Nothing else is changes, except
that these attributes exist now on the objects.

Reviewed-by: ultrotter

Since we don't want the string values of the parameters (e.g.
“kernel_path”) spread over the code, we introduce constants for these.

Reviewed-by: ultrotter,schreiberal