The job queue is now updated through its own RPC functions.

Reviewed-by: iustinp

Reviewed-by: iustinp

jobqueue_update: Uploads a job queue file's content to a node. The
most common operation is to upload something that we already have
in a string. Unlike in the upload_file function, the file is not
read again when distributing changes, but content has to be passed
as a string.

jobqueue_purge: Removes all queue related files from a node.

Reviewed-by: iustinp

JobQueuePurge() will be used by an RPC function.

Reviewed-by: iustinp

Reviewed-by: iustinp

feedback_fn wasn't passed to it.

Reviewed-by: iustinp

The job queue maintains its own node list and must be notified
when nodes are added/removed.

Reviewed-by: iustinp

By doing this we've a central place which coordinates what needs to be
done when adding or removing nodes. Another patch will add calls into
the job queue.

Two log messages move to config.py.

When removing a node, node_leave_cluster is now called after it has
been removed from the configuration and job manager. That way we're
sure not to access the node again after files have been removed.

Reviewed-by: iustinp

These functions will be used to notify the queue about newly added
or removed nodes.

Reviewed-by: iustinp

The job queue now maintains its own list and is updated when
nodes are added or removed from the cluster.

Reviewed-by: iustinp

The code makes sure not to include the master in the list.

Reviewed-by: iustinp

The job queue must be called from cmdlib when adding or removing
nodes to the cluster. Moving it to the context objects makes
this possible.

Reviewed-by: iustinp

Old job files shouldn't be left on nodes removed from a cluster.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

Queries don't create jobs and are more efficient. Log messages
are not yet stored anywhere.

Reviewed-by: iustinp

Newly added nodes are not yet taken care of. Queue locking on
non-master nodes is not yet correct.

Reviewed-by: iustinp

Reviewed-by: iustinp

This will be used to move common code out of jqueue.

Reviewed-by: iustinp

This patch adds the infrastructure for executing a job in background,
instead of foreground, via a new “--submit” option. The behaviour is
that the job ID is printed and the script will immediately exit.

The patch also converts gnt-node list to this model (yes, this will be a
query in the future).

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

This reduces code duplication. A later patch will modify the job queue
a bit more and will need a change of this assert. The assertion is
also removed from all class-internal functions.

Reviewed-by: iustinp

The current SubmitOpCode function is not flexible enough to be used for
submitters that don't want to wait for the job finish.

The patch splits this in two, a SendJob function and a PollJob one, and
the old SubmitOpCode becomes a wrapper. Note that the new SendJob takes
a list of opcodes (and not a single opcode anymore).

Reviewed-by: imsnah

This is needed for job queue replication.

Reviewed-by: iustinp

This class is a wrapper around fcntl.flock and abstracts opening and
closing the lockfile. It'll used for the job queue.

(The patch also removes a duplicate import of tempfile into the unittest)

Reviewed-by: iustinp

The job queue will need to access to configuration, which is provided
through the context object, to get a list of nodes.

Reviewed-by: iustinp

Reviewed-by: imsnah

Add instance tag handling, improved error logging.
...oh, yes adopt instance listing for RAPI2!

Reviewed-by: iustinp

With the recent startup/shutdown changes (and with the master daemon in
place), the cluster destroy needs some fixing.

This patch moves the finalization of the destroy out from cmdlib into
bootstrap, so we can nicely shutdown the rapi and master daemons.

Reviewed-by: ultrotter

It's python, isn't it?

Reviewed-by: iustinp

With the recent changes, I forgot the extra parameter to this rpc call.
Also the rpc call needs to be done after we setup the config data, for
the master daemon to be able to start, so we move it after all other
init steps.

Reviewed-by: ultrotter

This patch adds a check that we are on the master after failing to
connect to the socket, and log nicely the master name.

Reviewed-by: ultrotter

Reviewed-by: iustinp

Prevent mistakes with an assert.

Reviewed-by: iustinp

This is my fault, sorry.

Reviewed-by: imsnah

This is mostly:
  - whitespace fix (space at EOL in some files, not all, broken
    indentation, etc)
  - variable names overriding others (one is a real bug in there)
  - too-long-lines
  - cleanup of most unused imports (not all)

Reviewed-by: ultrotter

Reviewed-by: imsnah

The 'old-style' info, error, debug logs do not make much sense. This
patch unifies the SetupLogging and SetupDaemon functions. As a result,
all the commands logs to a 'commands.log' file.

The patch also changes the log setup to keep going if there's an error
in setting up the file logging but we're logging to stderr.

Also, burnin now logs to its own file (burnin.log).

Reviewed-by: ultrotter