The first can be problematic if poll(2) returns POLLHUP|POLLERR on a
socket. Before it would be only be respected for SOCKOP_RECV, but since
they can also occur on other socket operations, esp. in combination with
OpenSSL, letting the socket functions handle POLLHUP|POLLERR seems to be
the right thing.

The second is a typo leading to an endless loop if the first line of an
HTTP connection is empty (simply "\r\n"). Instead of removing the empty
line, it would remove anything after it.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is from Debian Bug#568105.

ganeti-cleaner in one case erroneously calls "echo 0" instead of "exit
0". As ganeti-cleaner is called in the daily cronjob, this leads to a
daily mail containing just "0" with an unconfigured default install.

Signed-off-by: Elmar Hoffmann <elho@elho.net>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>

The credit goes again to Lintian.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The recent change to use LogWarning with multiple arguments in
mpcu.py/HooksMaster broke the (simple) mock we have in the tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

LURemoveNode runs under the BGL, which means we're guaranteed that the
list of nodes as retrieved in CheckPrereq is still valid in
BuildHooksEnv. However, we can make Ganeti handle failures in case the
locking is broken (or the node list has been modified otherwise) easily,
which is better than crashing hard in such a case.

This will also fix issue 79, even though that is due to an out-of-tree
patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This might fix issue 84; in any case, the current situation is that we
have a potentially unsafe formatting, which should be fixed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

int()/float() can raise either ValueError (in case of int("a")), or
TypeError (in case of int(None)). We had many bugs over time due to
this, and a recent one was just diagnosed, so we go over the codebase
and replace all 'except ValueError' with 'except (TypeError,
ValueError)' that protect such conversions (there were no 'except
TypeError' cases that needed a ValueError added).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per pylint warnings.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This just adds a stub function that raises an assertion error; this
accomplishes two things:

- silences many pylint warnings
- if we ever stumble upon this, a specific assertion error is
  (hopefully) clearer than just a not implemented error

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This removes/updates some module-wide pylint disables.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

New version of pylint, new bugs found!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Since the current pylintrc is in now way good for our style, we update
it:

- remove docstring checks, since we have too many cases where a
  docstring doesn't make sense
- relax naming rules to cover more alternatives (but not break the
  naming rules)
- increase many of the limits

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This patch adds targeted pylint disables, where it makes sense (either
due to limitations in pylint or due to historical usage), and also a few
blanket ones in rapi where all the names are… “different”.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This cherry-picks the utils.FieldSet.Matches changes and the significant
jqueue.py change. These are stable in the 2.1 branch and therefore make
sense to backport to 2.0 (are basically cleanups).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Currently releases are done via the manual procedure on
http://code.google.com/p/ganeti/wiki/ReleaseProcess

, but that is not
very reliable, and breaks for rc releases. The 1.2.9/2.0.5/2.1.0~rc2
releases were done with this new script that eases the process.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

MaybeRaise in lib/errors.py had a typo in the doc string

Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch unifies the search for external script to always go through
utils.FindFile and implements in that function a restriction on valid
chars in file names and (additionally) that the passed name is the
basename of the final (absolute) name.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will allow reuse of the same mask for multiple validations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In most uses of LUQueryNodes, we don't take a lock. This means that the
instance data is not protected across GetInstanceList and
GetInstanceInfo, and this can lead to instances not existing anymore.

Switching to GetAllInstanceInfo means that we get a single,
semi-consistent snapshot (since instances can still be modified, but
it's much better).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Patch 4352bf6d changed RAPI docs to RST from the previous SGML format
(the last file to be converted to RST).

However, it didn't add rapi.rst to the docrst Makefile variable, and as
such rapi.html was neither build nor shipped in the archive. We fix this
and we also distribute doc/rapi-resources.gen since otherwise “make
distcheck” has issues (since this file is built and thus lives in _built
whereas rapi.rst is shipped and lives in topsrcdir).

Note: yes, this means that no 2.0 release until now had a rapi.html file
(in the .tar.gz archive).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The timestamp need special formatting, which was done for the internal
buffer storage but not for the messages logged in verbose mode. This
patch unifies the formatting for these two cases.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

QA passed successfully, let's try to have a release.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the actual exception raised during an LU execution (one of
OpPrereqError, OpExecError, HooksError, etc.) is lost because the
jqueue.py code simply sets that to a str(err), and the code in cli.py
simply passes that string to OpExecError.

This patch moves to encoding the errors as per errors.EncodeError and
changes the cli code to parse and raise that (if possible).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit bcb66fca)

Currently the luxi error handling is hardcoded as special encoding on
the masterd-side and special decoding on the client side. This patch
moves it to errors.py such that other parts of the code can reuse the
same encoding.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 6956e9cd)

Before, it used to say:

  ssh/hostname verification failed node1.example.com -> hostname mismatch, got
  node2

Now it says for wrong hostnames (maybe too verbose):

  ssh/hostname verification failed (checking from node1.example.com): hostname
  mismatch, expected node2.example.com but got node3

And for non-FQDN hostnames:

  ssh/hostname verification failed (checking from node1.example.com): hostname
  not FQDN: expected node2.example.com but got node2

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We don't bump up the version yet, pending more QA tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 34d657ba)

We will need this in another place, so we abstract the 'compute all
current IDs' functionality into a separate function. We also change the
name of the _ComputeAllLVs to _AllLVs to match the other _All*s
functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 34e54ebc)

Currently the config upgrade is done at each object instantiation, that
means that ganeti-noded will run UpgradeConfig on all objects received
remotely (instances, disks, nics). This is not so good, so this patch
changes it so that only the ConfigWriter runs this method at
configuration load time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit 90d726a8)

* commit 'origin/next': (74 commits)
  Fix gnt-node modify online help
  Fix gnt-job info entry in gnt-job(8)
  locking: Don't swallow exceptions
  Add check for duplicate MACs in instance add
  scripts/gnt-node: fix a help string
  Optimise multi-job submit
  Extend gnt-debug with more debugging options
  Return cluster tags from LUQueryClusterInfo
  Add script to clean archived jobs after 21 days
  rapi: export more static node information
  Pass the correct signal to handlers
  cli: Use ToStdout/ToStderr instead of print
  Fix small typo in gnt-node
  Simplify handling of boolean args in rapi
  Fix checks in LUSetNodeParms for the master node
  Improve the example startup script
  Fix insserv dependencies
  Fix a typo in InitCluster
  Ignore results from drained nodes in iallocator
  Ship the ethers hook
  ...

This completes what was began in commit
862b1b2b

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It currently reports "cancel" in its syntax, which is obviously a
cut&paste error.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is an indentation bug.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently LUAddInstance doesn't check for duplicate MACs, and it fails
during the Exec() phase when trying to add the instance to the config
(ConfigWriter checks for this). This patch copies the code from
LUModifyInstance (which already does it).

This fixed issue #70.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

gnt-node modify says it wants an "<instance>". Changing it to
"<node_name>" as for the other commands.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, on multi-job submits we simply iterate over the
single-job-submit function. This means we grab a new serial, write and
replicate (and wait for the remote nodes to ack) the serial file, and
only then create the job file; this is repeated N times, once for each
job.

Since job identifiers are ‘cheap’, it's simpler to simply grab at the
start a block of new IDs, write and replicate the serial count file a
single time, and then proceed with the jobs as before. This is a cheap
change that reduces I/O and reduces slightly the CPU consumption of the
master daemon: submit time seems to be cut in half for big batches of
jobs and the masterd cpu time by (I can't get consistent numbers)
between 15%-50%.

Note that this doesn't change anything for single-job submits and most
probably for < 5 job submits either.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch extends gnt-debug to be able to submit multiple copies of the
input jobs and job contents, in order to simplify testing. It also adds
a timing mode, and splits the execution into separate submit and
execution stages (for timing purposes).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>