- s/GanetiQuitException/QuitGanetiException/
- Look for the arguments in err.args, not err itself

Reviewed-by: iustinp

Accoring to the usage documented in the QuitGanetiException docstring,
if we receive such an exception we'll set the global _EXIT_GANETI_NODED
variable to True, and then return either a valid value or an error
message to the user. This will be the last request we serve, though,
because the main loop will be interrupted and the daemon will terminate.

Reviewed-by: iustinp

Rather than calling httpd.serve_forever() in ganeti-noded we'll call
httpd.handle_request() but just while a global variable, which we'll
call _EXIT_GANETI_NODED, remains false.

Reviewed-by: iustinp

If an uncaught exception is thrown currently it destroys the calling
thread. This patch changes the behaviour to failing the current job,
logging a message, but trying to keep the daemon up.

Reviewed-by: imsnah

This patch adds rpc layer calls (in rpc.py and the equivalent in
ganeti-noded) to close a list of block devices, and the wrapper in
backend.py that takes a list of Disk objects, identifies them and
returns correctly formatted results.

The reason why this very basic call was missing until now from the rpc
layer is that we usually don't care about device closes (though we
should, and will do so in the future) as only drbd has a meaningful
Close() operation; right now we directly do Shutdown().

The patch is clean enough that it's actually independent of the live
migration implementation.

Reviewed-by: imsnah

This change allows us to use cleaner dependencies between
directories. The build system is basically rewritten in large parts
and may contain bugs.

Reviewed-by: iustinp

Reviewed-by: iustinp

This is the safest way to detect changes and the amount of data
is small, so keeping a copy around is cheap enough.

Reviewed-by: iustinp

Cleanup: _data is private and should not be modified from outside
of this class.

Reviewed-by: iustinp

Reviewed-by: iustinp

- Lock it before renaming
- Code cleanup; close() automatically unlocks it

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: ultrotter

- Log timestamp for all messages
- Write everything to logfile and optionally to stderr
- Log messages are no longer buffered, allowing a user to see progress

Reviewed-by: ultrotter

This simple patch exposes the block device grow operation at the rpc
layer. It does not increase the protocol version as it has been recently
changed by the live failover rpc call.

Reviewed-by: imsnah

This patch adds the migration rpc call and its implementation in the
backend. The patch does not deal with the correct activation of disks.

Because of the new RPC, the protocol version is increased.

Reviewed-by: imsnah

- Shorter code
- Reorder arguments to logger.SetupLogging calls to make more sense

Reviewed-by: iustinp

When a Job raises a ganeti exception a message is printed but nothing is
reported in the job itself. It's better to update the job status, thus
notifying the client, possibly polling for the job result, of what went
wrong.

Reviewed-by: iustinp

Currently the watcher runs first the instance startup and then the
boot-id method of disk reactivation. However, irrelevant of the fact
that a node has rebooted or not, if we just started an instance, there's
no need for its disks to be activated again, since the start instance
has done that (if it is at all possible).

The patch modifies the watcher to remember all started instances and not
run activate-disks for them.

Reviewed-by: ultrotter

Currently the watcher does activate disks (via bootid mechanisms) even
for admin_down instances.  This patch logs and skips over these
instances.

Reviewed-by: ultrotter

- Add a docstring to IOServer's constructor
- Add argument description to PoolWorker's and JobRunner's ones

Reviewed-by: iustinp

This patch adds a mechanism to disable utils.RunCmd in selected
programs. This is needed in the master daemon unless we confirm
threading doesn't pose any problems.

This makes cluster init fail, but creating new trunk clusters is anyway
broken at the moment.

Reviewed-by: ultrotter

This patch removes the lock and the lock options from cli.py and moves
them to the master.

Later during development we can remove it completely, but for now it's
good to protect any other tool that uses the lock directly.

Reviewed-by: imsnah,ultrotter

This patch converts the cli.py SubmitOpCode method to use the unix
protocol and thus execute the opcodes via the master.

The patch allows a partial burnin to work with the master. Currently the
query opcodes, since they are executed via the SubmitOpCode, are
executed inside a job too, which is suboptimal, but they work fine.

The cmd lock has been removed from the master, but the cli.py still
takes the lock. This is ok for this in-progress patch (since the master
still has only one executor thread). This will be fixed in a future
patch.

Reviewed-by: ultrotter

Currently the iallocator execution takes place in the master, which is a
violation of the current architecture, and will create problems with a
threaded master daemon.

This patch moves the execution to the backend, similar to the hooks
runner, by:
  - introducing a new class that handles the execution in the backend
    (and could be used also for listing the allocators, etc.)
  - introducing a new rpc call
  - replacing the actual execution in IAllocator.Run() with a rpc call

This passes burnin with the dumb allocator

Reviewed-by: imsnah

This patch changes the definition of a job and introduces per-opcode
results.

First, the result and status fields of a job are condensed into a single
'status' attribute. Then, we introduce an opcode status and one result
list, that allow jobs to return values.

The gnt-job script is also modified to allow these new fields to be
queried.

Note that the patch changes the opcode field to op_list, and it changes
its return value from string to a list of (serialized) opcodes.

Reviewed-by: ultrotter

This patch adds checks for the master role and daemonize support to
ganeti-masterd.

The patch modifies the startup/shutdown of the server because:
  - we want bind()/listen() to the master socket to occur before forking
    so that we can return a correct exit code and write messages to
    stderr
  - but we want thread startup to occur after fork(), otherwise python
    threading gets confused

The patch also has some small cleanups:
  - remove the unix socket after closing it, so we don't need to remove
    it manually
  - instead of just telling the threads to terminate via the new_queue,
    we also join() them so that the logs show what thread clinging to
    life
  - the daemon logs to its own logfile now
  - there is command line parameter support :)

Reviewed-by: imsnah

Reviewed-by: ultrotter

This patch adds a very basic gnt-job script that allows job querying.
This goes on top of the previous master daemon patches.

Currently, because of the not-changed cmd lock, you can't query the jobs
as long as a job is running - you have to rm the cmd lock and then you
can query the jobs.

Reviewed-by: imsnah

Currently, in ganeti-noded we have the createDaemon function. Since
we'll need the same in other daemons, we move this function to utils.py

With the move, a few changes were also done:
  - change the name to Daemonize()
  - add a parameter, logfile, as different daemons will want to log to
    different files
  - remove the try.. except.. around the fork calls, since they were
    only re-raising the OS exception with less data; unless we want to
    actually handle fork error (not just re-raising), these try blocks
    are not useful
  - change the return style at the end of the function

Reviewed-by: imsnah

This patch adds a very in-progress master daemon. This needs to be
launched manually, does not background itself, but can be used for
opcode execution.

Also parts of this code should be moved to luxi.py.

Reviewed-by: ultrotter

This patch just removes an extraneous \n from the log message making it
nicer to view.

Reviewed-by: schreiberal

Currently in order to deal with tmpfs /var/run, we create the
BDEV_CACHE_DIR in the init script. However, that does not cover all the
cases, and it's not a proper place to deal with it: for example, dealing
with not initialized clusters and the master node is more complicated.

Therefore, this patch does:
  - make ganeti-noded create the directory automatically
  - make ganeti-noded error out if it can't create it or it's already
    there but not a directory
  - remove the creation from the init.d script

Reviewed-by: ultrotter

This patch modifies TcpPing and its callers to make the source address
selection optional. Usually, the kernel will know better what
source address to use, just in some cases we want to enforce a given
source address so it makes sense to make this optional.

Reviewed-by: ultrotter

This patch switches from the twisted usage for inter-node protocol to
simple BaseHTTPServer/httplib. The patch has more deletions because we
use no authentication, no encryption at all.

As such, this is just for trunk, and only for testing. What it brings is
the ability to use the rpc library from within multiple threads in
parallel (or it should so).

Since the changes are very few and non-intrusive, they can be reverted
without impacting the rest of the code.

This passes burnin. QA was not tested.

Reviewed-by: imsnah

This can be used for testing purposes.

Reviewed-by: ultrotter,imsnah

This patch modifies the watcher to run the ‘gnt-cluster verify-disks’
command and to log its output (if any).

Reviewed-by: imsnah

- When line wrapping is needed, move spaces to the next line.
- Remove embedded line breaks from error messages.

Reviewed-by: schreiberal

This patch adds logging of command failures to the debug log in case the
user either started the command (gnt-*) or the node daemon with the
debug flag.

Reviewed-by: imsnah

- Use constants for keys.
- Fix bug through which automatic instance restarts wouldn't be limited

Reviewed-by: iustinp