Reviewed-by: amishchenko

Reviewed-by: amishchenko

Reviewed-by: iustinp

Remove unused vars

Reviewed-by: amishchenko

Reviewed-by: imsnah

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

This patch adds a new node parameter called offline that will be used to
mark nodes which should be touched by commands.

We also add this flag at cluster init, node add, and export it to
iallocator scripts.

Reviewed-by: ultrotter

After the cluster is ready we'll load the ConfigWriter and force a
writeout of all config files.

Reviewed-by: imsnah

Reviewed-by: imsnah

- Add a new command line option, defaulting to the constant value
- Pass the value to bootstrap.InitCluster
- Use it to init the new Cluster object

Reviewed-by: imsnah

For a long time we had the problem that both RPC-layer errors and
results from the remote node share the same "valuespace". This is
because we shouldn't raise an exception when only one node failed
(and lose the results from the other nodes).

This patch attempts to address this problem by returning a special
object from RPC calls, which separates the rpc-layer status and the
remote results into different attributes.

All the users of rpc (mainly cmdlib, but also bootstrap and the
HooksMaster in mcpu) have been converted to this new model. The code has
changed from, e.g. for boolean return types:

  if not self.rpc.call_...

to

  result = self.rpc.call_
  if result.failed or not result.data:
     ^ rpc-layer error    |
                          - result payload

While this is slightly more complicated, it will allow cleaner checks in
the future; right now the code is just a plain port, without
optimizations.

There's also a "result.Raise()" which raises an OpExecError if the
rpc-layer had errors.

One side-effect of the patch is that now all return types from the
rpc.call_ functions are of either RpcResult (single-node) or dicts of
(node name, RpcResult); previously, some functions were returning
different object types based on error status.

The code passes burnin (after many retries :).

Reviewed-by: imsnah

Where we used/forgot to validate beparams we now use the new common function.

Reviewed-by: imsnah

The ssconf files were not updated by the master failover. We need to
push them, and since we already have RPC initialized, we can use the
standard ConfigWriter to do so - this will take care of both the config
file and the ssconf files.

Reviewed-by: imsnah

Reviewed-by: imsnah

Currently it was left alone, and thus its value was "null".

Reviewed-by: imsnah

Currently, the node add operation uses a job to query the node name and
the bootstrap function directly reads the config file for the cluster
name.

This patch changes to that both the cluster name and the verification of
the node is done via queries to the master.

Reviewed-by: ultrotter

With the new SSL client certificate stuff it's no longer needed.

Reviewed-by: iustinp

Reviewed-by: iustinp

"from ganeti.rpc import RpcRunner" does not conform to the style guide.

Reviewed-by: iustinp

During cluster init, set the default hypervisor to be used for instances.
Ensure that the default hypervisor belongs to the set enabled hypervisors
for this cluster. Also fix a small bug with setting the default enabled
hypervisor list.

Reviewed-by: imsnah

We no longer use a single, cluster-wide hypervisor, but configure the
actual to be used hypervisor on the instance level.

Reviewed-by: imsnah

Reviewed-by: iustinp

If we don't actually know the current master (as determined via voting),
we prevent the failover.

The patch also changes some messages (capitalization, typos).

Reviewed-by: ultrotter

In order to account for future improvements to master failover, we move
the actual data gathering capabilities from ganeti-masterd into
bootstrap.py, and we leave only the verification into masterd.

The verification procedure is then changed to retry multiple times (up
to one minute) in case most nodes do not respond, and also the algorithm
is changed to require at least half (but not half+1) votes, since our
vote also should count (and we vote for ourselves).

Example for consistent (config-wise) cluster:
  - 5 node cluster, 2 nodes down: still start
  - 4 node cluster, 2 nodes down: retry for one minute, abort

Reviewed-by: ultrotter

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to
change the implementation.

The patch abstracts this into a separate utils.OwnIpAddress(addr)
function. We add a rpc call for it, which we use instead of the
(single-use of) call_node_tcp_ping. We leave node_tcp_ping in, as seems
useful and eventually it should be removed in a separate patch.

Reviewed-by: imsnah

This big patch changes the call model used in internode-rpc from
standalong function calls in the rpc module to via a RpcRunner class,
that holds all the methods. This can be used in the future to enable
smarter processing in the RPC layer itself (some quick examples are not
setting the DiskID from cmdlib code, but only once in each rpc call,
etc.).

There are a few RPC calls that are made outside of the LU code, and
these calls are left as staticmethods, so they can be used without a
class instance (which requires a ConfigWriter instance).

Reviewed-by: imsnah

Indentation in bootstrap was wrong and some names in cmdlib.py were not
right.

Reviewed-by: imsnah

Since in 2.0 the user will possibly have more interaction with the
hypervisor names, we sanitize them by removing the version numbers
(the version can be a prerequisite for the ganeti installation, we
shouldn't document it in variable names).

Reviewed-by: schreiberal

Reviewed-by: iustinp

More places actually use the SshRunner than just the gnt-cluster
commands.

Reviewed-by: ultrotter

Remove leftovers from ssconf.

Reviewed-by: iustinp

Replace ssconf with configuration.

Reviewed-by: iustinp

Get rid of ssconf and convert to configuration instead.

Reviewed-by: iustinp

ssconf will become write-only from ganeti-masterd's point of view,
therefore all settings in there need to go into the main configuration
file.

Reviewed-by: iustinp

Future patches will add even more variables to the cluster config.
Adding more parameters wouldn't make the function easier to use and
it doesn't make sense to pass them to another function, as it's
only done once in bootstrap.py on cluster initialization.

Reviewed-by: iustinp

The bootstrap code needs a pseudo-secret and this is currently generated
inside the InitGanetiServerSetup function. Since more users will need
this, move it to utils.py

Reviewed-by: ultrotter

This option will be used to add nodes to the cluster without
asking the user to confirm the key. Together with key based
authentication this can be used in the QA tests.

Reviewed-by: ultrotter

If a node hasn't been part of the cluster before being added it'll not
have the cluster's SSH key. This patch makes sure to accept those by
not aliasing the machine name to the cluster name.

Reviewed-by: ultrotter

With the recent startup/shutdown changes (and with the master daemon in
place), the cluster destroy needs some fixing.

This patch moves the finalization of the destroy out from cmdlib into
bootstrap, so we can nicely shutdown the rapi and master daemons.

Reviewed-by: ultrotter

With the recent changes, I forgot the extra parameter to this rpc call.
Also the rpc call needs to be done after we setup the config data, for
the master daemon to be able to start, so we move it after all other
init steps.

Reviewed-by: ultrotter