The function to stop a worker pool is TerminateWorkers(), not Shutdown().

Reviewed-by: iustinp

Reusing threads instead of starting one for each request is more efficient.

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

Apparently I forgot to this code when removing the rest.

Reviewed-by: iustinp

Reviewed-by: iustinp

Currently, in debug mode, both the logfile handler and the stderr
handler will log debug messages. Since the stderr is redirected to the
same logfile (to catch non-logged errors), it means log entries are
doubled.

The patch adds an extra parameter to the logger.SetupDaemon() function
that allows disabling of the stderr logging. The master and node daemon
will use this to enable stderr logging only when running in foreground.

Reviewed-by: imsnah

The patch adds some more logging to the node daemon:

- log methods at beggining not only at the end
- log method parameters (they are very verbose, but useful)

A separate change is to initialize the global variable in the global
scope, not inside main().

Reviewed-by: imsnah

This removes (hopefully) all traces of the old locking functions and
uses.

Reviewed-by: imsnah

Reviewed-by: iustinp

- Introduce abstraction class on client side
- Use constants for method names
- Adopt legacy function SubmitOpCode to use it

Reviewed-by: iustinp

- Use constants for dict entries
- Handle exceptions on server side
- Rename client function to CallMethod to match server side naming

Reviewed-by: iustinp

Reviewed-by: iustinp

Rather than just creating BDEV_CACHE_DIR we loop through the
SUB_RUN_DIRS list and create all its childs.

Reviewed-by: iustinp

This patch fixes two bugs:
  - the state file is not saved because we use the method for checking
    for udpated data
  - in two places 'Error' was used instead of 'Exception', which breaks
    error handling

Additionally:
  - the unused 're' import has been removed
  - a variable named 'id' which collides with a builtin function has
    been renamed

Note that comparing the serialized forms might create false negatives
(due to the dicts being reordered) but that will just cause an extra
write of the file, which is sub-optimal but harmless.

Reviewed-by: ultrotter

It's better for daemons if:
  - they log only to one log file
  - the log level is included
  - for debug runs, the filename/line number is included

This patch moves the custom formatter from the watcher to the logging
module and generalizes it; then it changes the master daemon to use this
function instead of the generic logging (which might be deprecated
anyway in the future).

Reviewed-by: imsnah

Reviewed-by: iustinp, ultrotter

Reviewed-by: ultrotter, iustinp

Make the GanetiLockManager instance of GanetiContext lowercase

Reviewed-by: imsnah

Now that we use the locking library to make sure running opcodes cannot
step on each other toes we can have a bigger thread size, and
potentially process many opcodes in a parallel manner.

Reviewed-by: iustinp

The processor used to create a new ConfigWriter when it was initialized.
We now have one in the context, so we'll just recycle it. First of all
we'll pass the context in when creating a new Processor object, then
we'll just use context.cfg, which is granted to be initialized, wherever
we used self.cfg, and stop checking whether the config is already
initialized or not.

In the future the Processor will be able to use the context also to
acquire the BGL for LUs that require it, and to push the context down to
LUs that don't in order for them to manage their own locking.

Reviewed-by: iustinp

This patch creates a new GanetiContext class, which is used to hold
context common to all ganeti worker threads. As for the
GanetiLockingManager class it is paramount that there is only one such
class throughout the execution of Ganeti, so the class checks for that,
and also forbids its own modification after it's been initialized. The
context for now contains a ConfigWriter and a GanetiLockingManager and
is created by the daemon and propagated to PoolWorker(s) and
JobRunner(s).

Reviewed-by: iustinp

- s/GanetiQuitException/QuitGanetiException/
- Look for the arguments in err.args, not err itself

Reviewed-by: iustinp

Accoring to the usage documented in the QuitGanetiException docstring,
if we receive such an exception we'll set the global _EXIT_GANETI_NODED
variable to True, and then return either a valid value or an error
message to the user. This will be the last request we serve, though,
because the main loop will be interrupted and the daemon will terminate.

Reviewed-by: iustinp

Rather than calling httpd.serve_forever() in ganeti-noded we'll call
httpd.handle_request() but just while a global variable, which we'll
call _EXIT_GANETI_NODED, remains false.

Reviewed-by: iustinp

If an uncaught exception is thrown currently it destroys the calling
thread. This patch changes the behaviour to failing the current job,
logging a message, but trying to keep the daemon up.

Reviewed-by: imsnah

This patch adds rpc layer calls (in rpc.py and the equivalent in
ganeti-noded) to close a list of block devices, and the wrapper in
backend.py that takes a list of Disk objects, identifies them and
returns correctly formatted results.

The reason why this very basic call was missing until now from the rpc
layer is that we usually don't care about device closes (though we
should, and will do so in the future) as only drbd has a meaningful
Close() operation; right now we directly do Shutdown().

The patch is clean enough that it's actually independent of the live
migration implementation.

Reviewed-by: imsnah

This change allows us to use cleaner dependencies between
directories. The build system is basically rewritten in large parts
and may contain bugs.

Reviewed-by: iustinp

Reviewed-by: iustinp

This is the safest way to detect changes and the amount of data
is small, so keeping a copy around is cheap enough.

Reviewed-by: iustinp

Cleanup: _data is private and should not be modified from outside
of this class.

Reviewed-by: iustinp

Reviewed-by: iustinp

- Lock it before renaming
- Code cleanup; close() automatically unlocks it

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: ultrotter

- Log timestamp for all messages
- Write everything to logfile and optionally to stderr
- Log messages are no longer buffered, allowing a user to see progress

Reviewed-by: ultrotter

This simple patch exposes the block device grow operation at the rpc
layer. It does not increase the protocol version as it has been recently
changed by the live failover rpc call.

Reviewed-by: imsnah

This patch adds the migration rpc call and its implementation in the
backend. The patch does not deal with the correct activation of disks.

Because of the new RPC, the protocol version is increased.

Reviewed-by: imsnah

- Shorter code
- Reorder arguments to logger.SetupLogging calls to make more sense

Reviewed-by: iustinp

When a Job raises a ganeti exception a message is printed but nothing is
reported in the job itself. It's better to update the job status, thus
notifying the client, possibly polling for the job result, of what went
wrong.

Reviewed-by: iustinp