Currently many error handling code paths in bdev.py log only
result.fail_reason (i.e. exit code or signal that killed the command)
but not its output. This makes debugging very hard.

The patch changes all places where we only log fail_reason to also log
result.output.

Reviewed-by: ultrotter

DRBD8 requires that we pass ‘--create-device’ to the first command that
wants to activate a new DRBD minor. We do this currently when we run the
“drbdsetup ... disk” command which we run before the network setup.

But if the LVs are missing, we skip the ‘disk’ subcommand and run only
the ‘net’ one, so it might be that the activation fails because the
minor we selected was never created in the first place.

The patch adds the required parameter to the DRBD8._AssembleNet() call.
Since it's a no-op for existing minors, it should not create any
problems (tested and works both with configured and unconfigured
minors).

Reviewed-by: ultrotter

There are a couple of reasons for doing so:
- /proc is not OS independent, it's only supported by Linux (there are
  emulations on other systems, but those might differ from the way
  Linux represents data).
- Checking a daemon's state doesn't necessarily mean it's usable.
  Connecting to the socket using “xm info” is much safer.
- Reduce code size.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

The algorithm for attaching to existing DRBD devices is not trivial. It
has four alternatives, and there is a bug in the last one when we have
diskless devices.

The last case (local disk info matches but remote/network configuration
doesn't match) we disconnect from the network and reattach with the
correct info. We do this because correct local device has higher
priority than remote device.

However, the test we use (self._MatchesLocal) can succeed in two cases:
  - we have a disk and it's the same as the one attached
  - we don't have a disk and the drbd is in diskless mode

But this creates problems for the fourth case as when we already have
one diskless DRBD, activating then next one will do:
  - _MatchesLocal? yes, because both config data and system have no
    disks (with the effect that all diskless devices are identical)
  - _MatchesRemote? no, because this disk is configured to its current
    remote peer, not to our new one

The fix is trivial, although the algorithm not: we only allow overriding
the network configuration when the disk information matches and we are
not diskless, by adding the <"local_dev" in info'> test.

Reviewed-by: ultrotter

Reviewed-by: iustinp

Upgrades will be handled in future patches.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

- Use variable with prefix instead of grep and sed
- Always run with /bin/bash

Reviewed-by: ultrotter

Now that we have the number of cpus available from the hypervisors, we
can export this to the iallocator scripts.

Reviewed-by: ultrotter

This shortens the help output in gnt-node so that the output looks
nicer, and improves the manual page for gnt-instance with the new
'status' field.

Reviewed-by: ultrotter

This patch allows the '-o' option to the list subcommands to add more
fields to the default list instead of replacing the default list by
prefixing the fields list with '+'.

The patch also moves the listing (in the help output) of the default
field list from hardcoded to built at runtime from the actual list.

Reviewed-by: ultrotter

This patch adds the backend and frontend changes needed for being able
to list the cpu count.

Reviewed-by: ultrotter

nodelist.remove(X) could potentially raise a ValueError (even if the chance
that the current node is not in the list are pretty scarce, and its absence
should raise a red flag anyway). If this happens let things go on, as that's
what the code which previously distributed the config did.

Reviewed-by: iustinp

Currently we get the list of nodes, and for each one extract all its info, and
just to exclude it if the name matches ours. Since the list of nodes is a list
of names just use .remove() to exclude ourself from it, and use that list
directly.

Reviewed-by: iustinp

SetKey is used, other than for adding new nodes, in another few cases. Update
the docstring to reflect this, so we don't mislead people reading it.

Reviewed-by: iustinp

This completes the changes in r898 by actually getting rid of the old unused
hypervisor.py code which was left in the code tree.

Reviewed-by: iustinp

- Add a docstring to IOServer's constructor
- Add argument description to PoolWorker's and JobRunner's ones

Reviewed-by: iustinp

Reviewed-by: iustinp

Since local_raid1 and remote_raid1 are deprecated they are removed
from the docs. This patch removes some old documentation sections
and bumps the documented version from 1.2 to 1.3.

Reviewed-by: iustinp

Reviewed-by: iustinp

Only post-hooks are run on cluster verify, and then their output is sent back
to the LU, which upon failure displays it to the user and changes the result of
the execution to a failure.

Reviewed-by: iustinp

Previously LUs could be failed by pre-hooks, and post-hooks just had effects by
themselves. This patch allows a LU to define the HooksCallBack function if it
wants to know about its hooks' results and alter its results in response.

The ChainOpCode execution path contains some commented out hooks code, which
this patch modifies to run the HooksCallBack function, so this is not forgot if
it ever gets uncommented out.

Reviewed-by: iustinp

Right now the hooks output is propagated from the nodes all the way up to
HooksMaster.RunPhase, which uses it for debugging PRE hooks, but then silently
discards them. We'll now propagate it up to the Processor.ExecOpCode function,
where they can be handled for other purposes (or discarded again, of course).
This patch also improves a bit the HooksMaster.RunPhase docstring.

Reviewed-by: iustinp

Since NoHooksLU defines HPATH as None, BuildHooksEnv will never be called (as
the LogicalUnit.BuildHooksEnv docstring correctly states). Removing the
function altogether, to avoid having dead code lying around, and to make sure
the non-implemented one in LogicalUnit will be called if someone mistakenly
tries to call it, so it's immediately clear that something is wrong.

Reviewed-by: iustinp

The LogicalUnit.BuildHooksEnv docstring used to say that the node list should
not include the master node. This is obviously not the case checking the
relevant code, and double-checking with iustin he confirmed it just document
some obsolete behaviour. This patch removes those remarks making the docstring
shorter, easier, and, last but not least, correct! ;)

Reviewed-by: iustinp

A fewdocstrings in the HooksRunner backend class list arguments the relevant
functions do not take.  Clean them up.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

This patch adds a mechanism to disable utils.RunCmd in selected
programs. This is needed in the master daemon unless we confirm
threading doesn't pose any problems.

This makes cluster init fail, but creating new trunk clusters is anyway
broken at the moment.

Reviewed-by: ultrotter

This patch adds a check to LUExportInstance.CheckPrereq to raise an
error when an instance with file disks is exported.

Reviewed-by: ultrotter

This patch adds the file storage options to gnt-backup import which
would otherwise fail since the values of file_storage_dir and
file_driver are accessed in LUCreateInstance.

Reviewed-by: ultrotter

Before it was a huge file with more than 700 lines. No code changes were
made except for changing imports and class names where needed. In the
future, we could even think about only importing the hypervisors we
actually need. The patch finished QA scripts and distcheck successfully.

Reviewed-by: iustinp

This patch removes the lock and the lock options from cli.py and moves
them to the master.

Later during development we can remove it completely, but for now it's
good to protect any other tool that uses the lock directly.

Reviewed-by: imsnah,ultrotter

This patch converts the cli.py SubmitOpCode method to use the unix
protocol and thus execute the opcodes via the master.

The patch allows a partial burnin to work with the master. Currently the
query opcodes, since they are executed via the SubmitOpCode, are
executed inside a job too, which is suboptimal, but they work fine.

The cmd lock has been removed from the master, but the cli.py still
takes the lock. This is ok for this in-progress patch (since the master
still has only one executor thread). This will be fixed in a future
patch.

Reviewed-by: ultrotter

bdev.FileStorage does not have major or minor fields, so gnt-instance
info for file backed instances fails with a TypeError. This patch
adds checks to prevent that and print 'N/A' instead.

Reviewed-by: imsnah

This patch changes the code executed when testing the signal handling
of RunCmd. Since sh does not always point to bash (e.g. on Ubuntu,
where it points to /bin/dash) this test might fail due to the returned
exit code is different so the received signal is not correctly
detected.

Additionally fix the docstring of testSignal.

Reviewed-by: iustinp

Currently the iallocator execution takes place in the master, which is a
violation of the current architecture, and will create problems with a
threaded master daemon.

This patch moves the execution to the backend, similar to the hooks
runner, by:
  - introducing a new class that handles the execution in the backend
    (and could be used also for listing the allocators, etc.)
  - introducing a new rpc call
  - replacing the actual execution in IAllocator.Run() with a rpc call

This passes burnin with the dumb allocator

Reviewed-by: imsnah