Currently, the actual exception raised during an LU execution (one of
OpPrereqError, OpExecError, HooksError, etc.) is lost because the
jqueue.py code simply sets that to a str(err), and the code in cli.py
simply passes that string to OpExecError.

This patch moves to encoding the errors as per errors.EncodeError and
changes the cli code to parse and raise that (if possible).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit bcb66fca)

Currently the luxi error handling is hardcoded as special encoding on
the masterd-side and special decoding on the client side. This patch
moves it to errors.py such that other parts of the code can reuse the
same encoding.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 6956e9cd)

Before, it used to say:

  ssh/hostname verification failed node1.example.com -> hostname mismatch, got
  node2

Now it says for wrong hostnames (maybe too verbose):

  ssh/hostname verification failed (checking from node1.example.com): hostname
  mismatch, expected node2.example.com but got node3

And for non-FQDN hostnames:

  ssh/hostname verification failed (checking from node1.example.com): hostname
  not FQDN: expected node2.example.com but got node2

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We don't bump up the version yet, pending more QA tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 34d657ba)

We will need this in another place, so we abstract the 'compute all
current IDs' functionality into a separate function. We also change the
name of the _ComputeAllLVs to _AllLVs to match the other _All*s
functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 34e54ebc)

Currently the config upgrade is done at each object instantiation, that
means that ganeti-noded will run UpgradeConfig on all objects received
remotely (instances, disks, nics). This is not so good, so this patch
changes it so that only the ConfigWriter runs this method at
configuration load time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit 90d726a8)

* commit 'origin/next': (74 commits)
  Fix gnt-node modify online help
  Fix gnt-job info entry in gnt-job(8)
  locking: Don't swallow exceptions
  Add check for duplicate MACs in instance add
  scripts/gnt-node: fix a help string
  Optimise multi-job submit
  Extend gnt-debug with more debugging options
  Return cluster tags from LUQueryClusterInfo
  Add script to clean archived jobs after 21 days
  rapi: export more static node information
  Pass the correct signal to handlers
  cli: Use ToStdout/ToStderr instead of print
  Fix small typo in gnt-node
  Simplify handling of boolean args in rapi
  Fix checks in LUSetNodeParms for the master node
  Improve the example startup script
  Fix insserv dependencies
  Fix a typo in InitCluster
  Ignore results from drained nodes in iallocator
  Ship the ethers hook
  ...

This completes what was began in commit
862b1b2b

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It currently reports "cancel" in its syntax, which is obviously a
cut&paste error.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is an indentation bug.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently LUAddInstance doesn't check for duplicate MACs, and it fails
during the Exec() phase when trying to add the instance to the config
(ConfigWriter checks for this). This patch copies the code from
LUModifyInstance (which already does it).

This fixed issue #70.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

gnt-node modify says it wants an "<instance>". Changing it to
"<node_name>" as for the other commands.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, on multi-job submits we simply iterate over the
single-job-submit function. This means we grab a new serial, write and
replicate (and wait for the remote nodes to ack) the serial file, and
only then create the job file; this is repeated N times, once for each
job.

Since job identifiers are ‘cheap’, it's simpler to simply grab at the
start a block of new IDs, write and replicate the serial count file a
single time, and then proceed with the jobs as before. This is a cheap
change that reduces I/O and reduces slightly the CPU consumption of the
master daemon: submit time seems to be cut in half for big batches of
jobs and the masterd cpu time by (I can't get consistent numbers)
between 15%-50%.

Note that this doesn't change anything for single-job submits and most
probably for < 5 job submits either.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch extends gnt-debug to be able to submit multiple copies of the
input jobs and job contents, in order to simplify testing. It also adds
a timing mode, and splits the execution into separate submit and
execution stages (for timing purposes).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently not all static node information is exported over rapi, and
this makes it less complete than gnt-node list. This patch attempts to
reduce the difference.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In case there are gaps in /proc/drbd related to the minors sequence,
this can lead to empty lines (see the attached data file). In this case,
we currently fail to parse the file correctly.

The patch fixes this by skipping empty lines in
bdev.BaseDRBD._MassageProcData and adds a data file and unittest for
this condition.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Mainloop currently calls all handlers with the signal number harcoded to
sigchild. Fixing this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

While this patch doesn't do a full match of on-disk size to config-size
for child devices, it does a sanity check (for DRBD only) that the child
size is not less than the DRBD size. While this would be a strange
occurrence (failed growth and masterd crash or similar), it does prevent
replace-disks from working.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 03298ebe)

The iallocator option is '-I' not '-i'.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch replaces hardcoded boolean-type args with
bool(_checkIntVariable). There should be no other cases of this left, I
think.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

There was a check already in the LU for the master node, however is
wasn't correct. This patch disallows any role changes on the master node
via LUSetNodeParms (and as this LU can't change anything else, it
practically prevents it from touching the master node).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the supplised script has two issues:
  - it doesn't use start-stop-daemon --start correctly, leading to
    messages like "ganeti.errors.GenericError:
    /var/run/ganeti/ganeti-rapi.pid contains a live process" in the logs
  - it doesn't allow start/stop/restart of a single daemon, which leads
    to manual launch, which is bad because we don't reuse the settings
    from the defaults file

For the first one, we change from ‘--exec …’ to ‘--startas …’, which is
the actual option used for start, whereas exec is a test (that also
supplies the default to startas). We also add ‘--oknodo’ as per recent
Debian policy changes.

For the second, we do a bigger change; we basically remove the full-path
and pid variables, and construct these two from the daemon name. We then
check if we are given a daemon name (in which case we only do that)
otherwise we do the requested action on all daemons.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

(import of a Debian patch)

This patch removes xend from the list of dependencies.

Ganeti doesn't need xend running to startup, it will only need it later
(and only if xen is used as virtualisation technology). It also removes
'Xen' from the description in the init script.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 022c3a0b)

Since drained nodes could be (partially or fully) broken in iallocator,
we ignore results from these nodes when building the cluster map in
preparation for sending it to the script.

This is a cheap change for the stable branch; ideally we should not
query them at all.

The patch also fixes a typo in iallocator.rst.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

doc/examples/hooks/ethers has been added without being shipped in the
released tarball. Putting a stop to this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is a result of broken copy-paste, and because needed_locks is not a
dict right here it will error out.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Remove "-l" option since some ancient systems ship a version of lockfile-progs
not supporting it.

Signed-off-by: Luca Bigliardi <shammash@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>

The same old story: disks need SetDiskID before being sent to remote
nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This moves yet another hardcoded value to a hypervisor parameter. I
removed the 64/32 difference as it doesn't seem valid to me - it's more
of a local site config rather than arch config.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit 09ea8710)

For the xen-hvm hypervisor, the KERNEL_PATH parameter is needed but
today is hardcoded to a constants in the xen hypervisor library (argh!).

This patch moves this to a hypervisor constant with the default value
being the current hardcoded path. This will allow cluster/instance
customisation based on the installed xen version.

This should fix Debian bug #528618.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit e2ee1cea)

From time to time we're adding new be or hv parameters. With this patch
missing parameters get set to the default value when loading the cluster
object. This patch version also considers the case when hv/be params
don't exist at all, and fixes a broken unit test triggered in that
case.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit c1b42c18)

If --no-etc-hosts is passed in at cluster init time we set a new
parameter in the cluster's object to false, and avoid adding nodes to
the hosts file. The UpgradeConfig function is used to set the value to
True, when upgrading from an old configuration version.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit b86a6bcd)