This is just a change of the various hvm_ and pvm parameters to the hv
model. Parameters are queried via hv/$name or via the whole dict as
returned by hvparams.

Reviewed-by: ultrotter,imsnah

This big patch changes instance create to the new hvparams structure.
Old parameters are removed, so old jobs or old instances file will break
current clusters.

Reviewed-by: ultrotter

We have a problem with the current model of combining instance lists
from multiple hypervisors: we don't allow duplicates, but "xm list"
gives the same output for both pvm and hvm. This is a lack in the actual
xen hypervisor implementation/split between pvm and hvm, but for now we
implement a weak workaround: identical instance params will be allowed,
and merged. This breaks because there is a delta in listing, and should
be treated as temporary workaround only.

Note that there are two cases for duplicate instance: the above one (xen
is the same, whether pvm or hvm), and the other case, the real error,
when we have two different hypervisors reporting the same instance name.
The latter case needs to be handled better (not by refusing to list the
instances in the backend).

Reviewed-by: ultrotter

We fill the instance hvparams with cluster defaults, and send a modified
dict over the wire to the node methods - they don't know anything about
cluster defaults versus instance parameters.

Reviewed-by: ultrotter

The newly-added node-specific ValidateParams hypervisor method is
exported over RPC, using the semi-standard (success, message) return
value. Multi-node call, so that we call on both primary and secondary at
once.

Reviewed-by: ultrotter

This patch adds the new methods CheckParameterSyntax and the
ValidateParameters in the base and xen hypervisors. It also changes the
xen hypervisor to use the new hvparams field on instance.

Reviewed-by: imsnah

This was a leftover from the early days of the remote API in Ganeti 1.2.

Reviewed-by: ultrotter

This patch adds paramter removal in SplitKeyVal, by prefixing a
value-less key with "-"; this is needed in resetting parameters back to
cluster defaults, but care must be applied now that None can come from
the parser.

Reviewed-by: imsnah

This patch adds the instance and cluster be/hvparams attributes, and the
Fill* methods on the cluster object. Nothing else is changes, except
that these attributes exist now on the objects.

Reviewed-by: ultrotter

Since we don't want the string values of the parameters (e.g.
“kernel_path”) spread over the code, we introduce constants for these.

Reviewed-by: ultrotter,schreiberal

This fixes:
  - whitespace change, double lines between methods
  - duplication of call_upload_file, introduced by mistake in rev 1795
    and which went undetected because of the many changes in that ref
    (only diff -b shows it clearly)
  - call_instance_info didn't pass the hypervisor name parameter, but
    the backend requires it

Reviewed-by: ultrotter

For the new 2.0-style command line options, we need to parse strings of
the type:
  ident:key=val[,...]
and
  key=val[,...]

This patch adds two new option builders for these two, which return
(ident, {key=val,}) and {key=val,} for the above two formats. It also
handles specially constructs of type “key” (val is set to True) and
“no_key” (val is set to False, and the ‘no_’ prefix is stripped).

Reviewed-by: imsnah

Reviewed-by: imsnah

As discussed, we update the design doc with the three classes of
parameters and other changes.

Reviewed-by: imsnah

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to
change the implementation.

The patch abstracts this into a separate utils.OwnIpAddress(addr)
function. We add a rpc call for it, which we use instead of the
(single-use of) call_node_tcp_ping. We leave node_tcp_ping in, as seems
useful and eventually it should be removed in a separate patch.

Reviewed-by: imsnah

Reviewed-by: iustinp

Allow multiple api versions in an OS. This is according to the OS API
changes design doc, by which an OS can support multiple versions of the
Ganeti API and if one is supported by Ganeti it will work. Since up to
version 5 of the API mandates an OS could support only one version, this
change is retrocompatible with it and requires no version bump up.

Reviewed-by: iustinp

When calling node_verify leads to an error _VerifyNodes tries to iterate
over a non-sequence. Catch the error before and avoid this from
happening.

Reviewed-by: iustinp

Reviewed-by: iustinp

This patch adds another implementation of an HTTP server. It's
based on code of Python's BaseHTTPServer, from both version
2.4 and 3k. In the future we can write code to decide whether
we should fork for a request or not. Keep-alive is not supported.

Reviewed-by: iustinp

This mainloop can be used in daemons like ganeti-noded.

Reviewed-by: iustinp

This clarifies the job storage and the reason for choosing it.

Reviewed-by: imsnah

With change 1773 a new status WAITLOCK was introduced if a job/opcode is
waiting for a lock. This change updates the document about the job-queue
accordingly.

Reviewed-by: iustinp

This big patch changes the call model used in internode-rpc from
standalong function calls in the rpc module to via a RpcRunner class,
that holds all the methods. This can be used in the future to enable
smarter processing in the RPC layer itself (some quick examples are not
setting the DiskID from cmdlib code, but only once in each rpc call,
etc.).

There are a few RPC calls that are made outside of the LU code, and
these calls are left as staticmethods, so they can be used without a
class instance (which requires a ConfigWriter instance).

Reviewed-by: imsnah

This patch is a cleanup of the standalone functions in cmdlib. Many of
them too as argument a ConfigWriter instance, but some also took other
parameters from the lu (e.g. proc), and in the future, if we want to
also pass the RpcRunner, we would have to add yet another parameter.

One option is to make all these methods of top-level LogicalUnit class.
I took another approach, and made (almost) all these functions take as
first parameter the lu instance. It's like methods, just not declared
under LogicalUnit.

Reviewed-by: imsnah

Indentation in bootstrap was wrong and some names in cmdlib.py were not
right.

Reviewed-by: imsnah

This check can be done earlier, in ExpandNames, and is needed here for
the hypervisor parameter check.

Reviewed-by: ultrotter

Reviewed-by: imsnah

Reviewed-by: iustinp

Reviewed-by: ultrotter

Reviewed-by: iustinp

Since in 2.0 the user will possibly have more interaction with the
hypervisor names, we sanitize them by removing the version numbers
(the version can be a prerequisite for the ganeti installation, we
shouldn't document it in variable names).

Reviewed-by: schreiberal

The idea is that if the OSes will support multiple version (e.g. both
1.2 and 2.0), then Ganeti should be able to talk to it using version
2.0, but then the script needs to be told nicely what version Ganeti is
using.

Reviewed-by: imsnah

Reviewed-by: iustinp

This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains
the ‘enable_hypervisors’ attribute, and instances can be created with
any of the enabled ones (no provision yet for changing that attribute).

The many many changes in the rpc/backend layer are due to the fact that
all backend code read the hypervisor from the local copy of the config,
and now we have to send it (either in the instance object, or as a
separate parameter) for each function.

The node list by default will list the node free/total memory for the
default hypervisor, a new flag to it should exist to select another
hypervisor. Instance list has a new field, hypervisor, that shows the
instance hypervisor. Cluster verify runs for all enabled hypervisor
types.

The new FIXMEs are related to IAllocator, since now the node
total/free/used memory counts are wrong (we can't reliably compute the
free memory).

Reviewed-by: imsnah

This patch changes formatting and the DRBD shared secret details, and
adds master daemon socket details to the security doc.

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

This is just some additions of not-yet-mentioned docs.

Reviewed-by: ultrotter

Merged r1777 from branches/ganeti/ganeti-1.2

Reviewed-by: imsnah