Reviewed-by: ultrotter

Adding a locking.py file for the ganeti locking library. Its first component is
the implementation of a non-recursive blocking shared lock complete with a
testing library.

Reviewed-by: imsnah, iustinp

This patch adds some documentation about the proposed client API. It's
not yet complete, but should be a usable draft for the initial
implementation.

Reviewed-by: ultrotter

This can be used for testing purposes.

Reviewed-by: ultrotter,imsnah

This patch is a first step in reducing the chance of causing DRBD
activation failures when the primary node has not-perfect data.

This issue is more seen with DRBD8, which has an 'outdate' state (in
which it can get more often). But it can (and before this patch, usually
will) happen with both 7 and 8 in the case the primary has data to sync.

The error comes from the fact that, before this patch, we activate the
primary DRBD device and immediately (i.e. as soon as we can run another
shell command) we try to make it primary. This might fail - since the
primary knows it has some data to catch up to - but we ignored this
error condition. The failure was visible later, in either md failing to
activate over a read-only storage or by instance failing to start.

The patch has two parts: one affecting bdev.py, which changes failures
in BlockDev.Open() from returning False to raising
errors.BlockDeviceError; noone (except a generic method inside bdev.py)
checked this return value and we logged it but the master didn't know
about it; now all classes raise errors from Open if they have a failure.

The other part, affecting cmdlib.py, changes the activation sequence
from:
  - activate on primary node as primary and secondary as secondary, in
    whatever order a function returns the nodes
to the following:
  - activate all drives as secondaries, on both the primary and the
    secondary nodes of the instance
  - after that, on the primary node, re-activate the device stack as
    primary

This is in order to give the chance to DRBD to connect and make the
handshake. As noted in the comments, this just increases the chances of
a handshake/connect, not fixing entirely the problem. However, it is a
good first step and it passes all tests of starting with stale (either
full or partial) primaries, with both drbd 7 and 8, and also passes a
burnin.

Note that the patch might make the device activation a little bit
slower, but it is a reasonable trade-off.

Reviewed-by: imsnah

Reviewed-by: imsnah

This patch completes the change introduced in r566 (trunk) and r568
(branch-1.2).

Reviewed-by: imsnah

Currently just the bridge and ip address are passed. Add an environment
variable for the mac address.

Reviewed-by: iustinp

Reviewed-by: imsnah

gnt-backup export used to export the ip and mac of each nic, but not which
bridge it was connected to. Adding this information.

Reviewed-by: iustinp

The help string missed drbd as a disk template option. Adding it.

Reviewed-by: iustinp

Reviewed-by: ultrotter

Reviewed-by: iustinp

The gnt-node and gnt-instance list commands have a customizable list of
output fields, but the list is not up to date (in the man page) and not
easily understandable from the ‘--help’ output.

This patch updates the man pages and adds the available fields and
default fields in the ‘--help’ output, as part of the description.

Example:
Usage
=====
  gnt-node list

Lists the nodes in the cluster. The available fields are (see the man page for
details): name, pinst_cnt, pinst_list, sinst_cnt, sinst_list, pip, sip,
dtotal, dfree, mtotal, mnode, mfree, bootid. The default field list is (in
order): name, dtotal, dfree, mtotal, mnode, mfree, pinst_cnt, sinst_cnt.

Reviewed-by: imsnah,ultrotter

Files are uploaded to $prefix/sbin, not $prefix/bin

Reviewed-by: iustinp

This patch adds QA tests for most of the possible parameters in the
instance modify operation (exception being the MAC), and modifies the
sample QA file to run this test.

It also tests the no-modification test, but that is a weak one: we only
test that the exit code is one, not that the command gave a proper
response ("... please give at least one parameter") as opposed to a
traceback.

Reviewed-by: imsnah

Reviewed-by: ultrotter

This patch allows the selection of the hypervisor type for the QA
process; this is useful when testing hypervisor-independent changes that
don't require a Xen setup.

The patch also fixes the OS name in the sample QA config file provided.

Reviewed-by: imsnah

The new QA tests for instance modify uncovered a bug in the modify
initrd operation when setting the initrd to none.

Reviewed-by: imsnah

As reported by hypnoce@gmail.com, this is missing a check on None. As we don't
care about uppercase, we keep the check simple by removing the lower() call.

Reviewed-by: ultrotter

It was wrongly deleted when converting

if a in dict.keys():
to
if a in dict:

Reviewed-by: imsnah

Currently, the tools are installed under $prefix/share/ganeti. This
prevents installing other things in a nice way under share/ganeti (like
arch-independent OS definitions), therefore we want the tools to live
under share/ganeti/tools.

A second change is that since these are programs, they would better live
under libdir than datadir - we might have to change them later to
binaries in which case 'share' is definitely not the way to go.

This patch therefore changes the install directory for the tools to
$prefix/lib/ganeti/tools.

Reviewed-by: imsnah

Those tests were added in the wrong place. This patch removes them.  One day
we'll implement proper command line regression testing and they should go in
there.

Reviewed-by: iustinp

This tests both that those two aliases have not been removed and also that
aliases handling hasn't been broken.

Reviewed-by: iustinp

These aliases are widely used to think of these operations and save some typing
too. Even though there is some thought to make start/stop the default operation
name I don't think this should happen for 1.2, for now adding it as an alias is
fine.

Reviewed-by: iustinp

Alias activate_block_devs to activate-disks, for ganeti 1.1 compatibility.

Reviewed-by: iustinp

Passing a new aliases dict to generic main we can easily support aliases for
compatibility reasons or simply useability.

Reviewed-by: iustinp

The upload tool can be used to submit the current code to an arbitrary list of
nodes. This helps developers in easily testing their changes before submitting
them.

Reviewed-by: iustinp

Before iterating over the list of exports present on a node, check that what
ganeti returned is actually a list. This solves the case when one of the nodes
is down, and an error value is returned.

This fixes issue 21

Reviewed-by: imsnah

LVM code sometimes adds an extra separator at the end of the field list.
Make the code strip it if exists.

Reviewed-by: imsnah

Currently, the function backend._GetVGInfo only checks for errors via
the exit code of the 'vgs' command. However, there are other ways of
failure so we need to also check for valid output before parsing.

Furthermore, the checks on the exit code were reported via a 'raise
LVMError', however this exception is not handled anywhere and so the
remote caller will not get reasonable data.

This patch does two main things:
  - change the calling protocol for this function to not raise an error,
    and instead return the same type of argument always (dict) with the
    requested keys but values changed into None; this allows in the
    parent rpc call node_info to have valid memory information but
    "error" value for disk space, if there's an error with disks
  - check the validity of the output so that in case we fail to parse
    it, we don't abort with a backtrace in the node daemon but instead
    return the default result value (containing errors), and log these
    cases in the node daemon log file

We also bump the protocol version to 11.

Reviewed-by: ultrotter

This patch does two things:
  - checks that the result values from call_node_info are valid integer
    values and aborts otherwise
  - skips disk space computation for the DT_DISKLESS case

The most important point of the patch is the verification of results
from the rpc call, as it prepares for a patch that allows failures to be
better reported from the remote node.

Reviewed-by: ultrotter

The checking of a node's free memory (via rpc.call_node_info) is done in
both start instance an failover. This patch abstracts this call,
together with the appropriate error handling, into a separate function
called _CheckNodeFreeMemory.

The patch also has some related changes:
  - the check is done in prereq and not in exec for start instance
  - the redundant check in exec for failover has been removed

Reviewed-by: ultrotter

The function backend.UploadFile still uses "/etc/hosts" directly instead
of the existing constant; this patch fixes this.

Reviewed-by: ultrotter

Even if this doesn't have any practical use for actually creating
instances, it can be used for very fast burnin and testing just the
add/start/stop/remove functionality.

This has also revealed a bug in export/import related to diskless
instances, so it's educational value is proved.

Reviewed-by: ultrotter

Currently the fake hypervisor has hardcoded ‘/var/run’ as a base
directory for its store. This patch adds a constant RUN_DIR that is used
for both the fake hypervisor and for BDEV_CACHE_DIR.

Reviewed-by: ultrotter

The script (which is geared towards Debian) is actually not fully
compliant, as lintian generates a warning on it - the S runlevel is not
a valid one in the "Stop" stanza. This patch removes "S" from the stop
list.

Reviewed-by: imsnah

In revision 459 I added a bug in the make dist rule in the sense that
the archive will include *all* of test/data directory, including the
.svn directory if it exists.

This patch fixes that problem and adds a distcheck hook that tests for
such errors in the future (files/directories matching the .svn and .git
patterns).

It also fixes a typo in the NEWS file.

Reviewed-by: imsnah

This a merge to trunk of revision 494.

Reviewed-by: imsnah

This is a merge from the 1.2 branch

Reviewed-by: imsnah