Apparently I forgot to this code when removing the rest.

Reviewed-by: iustinp

Reviewed-by: iustinp

Right now, the queued opcode doesn't have a lock, and instead relies on
the parent QueuedJob's lock.

This is not good for logging feedback, so it's better to have a lock for
each queuedopcode.

Reviewed-by: ultrotter

This is just a simple, hardcoded decorator for object methods needing
synchronization on the _lock instance attribute.

Reviewed-by: ultrotter

Reviewed-by: iustinp

Currently, in debug mode, both the logfile handler and the stderr
handler will log debug messages. Since the stderr is redirected to the
same logfile (to catch non-logged errors), it means log entries are
doubled.

The patch adds an extra parameter to the logger.SetupDaemon() function
that allows disabling of the stderr logging. The master and node daemon
will use this to enable stderr logging only when running in foreground.

Reviewed-by: imsnah

The patch adds some more logging to the node daemon:

- log methods at beggining not only at the end
- log method parameters (they are very verbose, but useful)

A separate change is to initialize the global variable in the global
scope, not inside main().

Reviewed-by: imsnah

Reviewed-by: imsnah

... as it was intended from the beggining, but by mistake left in the
top run dir.

Reviewed-by: ultrotter

Currently, the 'public' functions of bdev (FindDevice and
AttachOrAssemble) will call the Attach() method right after class
instantiation.

But the constructor itself calls this function, and therefore we have
duplicate Attach() calls (which are not cheap at all).

The patch introduces a new 'attached' instance attribute that tells if
the last Attach() was successful. The public functions reuse this so
that we only do the minimum required number of calls.

Reviewed-by: imsnah

This does not enhance in any way the messages; it just switches to the
new module.

Reviewed-by: imsnah

The patch also logs all commands executed from RunCmd when we are at
debug level.

Reviewed-by: imsnah

This removes (hopefully) all traces of the old locking functions and
uses.

Reviewed-by: imsnah

Reviewed-by: iustinp

- Introduce abstraction class on client side
- Use constants for method names
- Adopt legacy function SubmitOpCode to use it

Reviewed-by: iustinp

- Use constants for dict entries
- Handle exceptions on server side
- Rename client function to CallMethod to match server side naming

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

This is to discourage even more whoever may think that this requirement
is not really useful and can be lifted, and to at least know where it's
used before trying to break it.

Reviewed-by: imsnah

In order to do so:
  - We set REQ_BGL to False
  - We implement ExpandNames

That's it, really.

Reviewed-by: iustinp

If we're running in a "new style" LU we may need some locks, as required
by the ExpandNames function, to be able to run. We'll walk up the lock
levels present in the needed_locks dictionary and acquire them, then run
the actual LU. LUs can release some or all the acquired locks, if they
want, before terminating, provided they update their needed_locks
dictionary appropriately, so that we know not to release a level if they
have already done so.

Reviewed-by: iustinp

New concurrent LUs will need to call ExpandNames so that any names
passed in by the user are canonicalized, and can be used by hooks,
locking and other parts of the code. This was done in CheckPrereq
before, but it's now splitted out, as it's needed for locking, which in
turn CheckPrereq needs. Old LUs can be converted gradually.

Reviewed-by: iustinp

This makes the try...finally code simplier, and helps adding a more
complex locking structure before the actual execution. It also fixes a
concurrency bug caused by the fact that write_count was read before
acquiring the BGL, and thus spurious config update hooks run could have
been triggered. This doesn't solve the issue of running config update
hooks for concurrent LUs.

Reviewed-by: iustinp

This test checks the LockSet behaviour when an empty list is passed.
The current behaviour is expected, but since this is a corner case,
we're safer to keep it under a check, and if we need a different one
monitor that everything is as we expect it to be.

Reviewed-by: imsnah

Reviewed-by: iustinp

Workers have to notify their pool if they finished a task to make
the WorkerPool.Quiesce function work. This is done in the finally:
clause to notify even in case of an exception. However, before
we notified on each run, even if there was no task, thereby creating
some sort of an endless loop of notifications. In a future patch
we should split the single condition object into several to
produce less spurious notifications.

While we're at this, this patch also adds two new functions to
BaseWorker to query whether it's currently running a task and then
uses one of these functions in the WorkerPool instead of querying
the internal variable directly.

Reviewed-by: iustinp

Rather than just creating BDEV_CACHE_DIR we loop through the
SUB_RUN_DIRS list and create all its childs.

Reviewed-by: iustinp

This patch creates a base RUN_GANETI_DIR and then moves the other run
dir constants to use that (even if just setting BDEV_CACHE_DIR as equal
to it, rather than putting it deeper, for now).

Also we create a constant list of all the subdirs we need in RUN_DIR to
work properly, which we'll use when creating them in ganeti-noded.

Reviewed-by: iustinp

The DISK_LINKS_DIR points to the RUN_DIR/ganeti/instance-disks
directory, which will contain symlinks to the instances' disks. These
provide a stable name accross all nodes for them, and permit
live-migration to happen.

Unfortunately RUN_DIR/ganeti/instance-disks happens to be below ganeti
1.2's BDEV_CACHE_DIR, which will we need to address at some point
(possibly in 2.0).

Reviewed-by: iustinp

Reviewed-by: iustinp

If the simplejson module supports indentation, it's always used. There
are cases where we might not want to use it or enable it only for
debugging purposes, such as in RPC.

Reviewed-by: iustinp

cmdlib uses some constants from locking (ie. locking levels) but doesn't
import it. This patch fixes the issue.

Reviewed-by: iustinp

Since the context is passed to LogicalUnit, rather than the cfg, we can
only access the cfg as self.cfg, self.context.cfg, or context.cfg (in
the constructor). cfg is not valid anymore.

Reviewed-by: iustinp

Whenever we add an instance or node to the cluster (i.e. to the config
and whenever we remove them we should add/remove locks as well). In the
future we may want to optimize this so that the configwriter does it, or
it's handled at the context level, but till we're adding/removing
instances and nodes with the BGL held it doesn't matter too much.

Reviewed-by: iustinp

Rather than passing a ConfigWriter to the LUs we'll pass the whole
context, from which a ConfigWriter can be extracted, but we can also
access the GanetiLockManager. This also fixes the places where a FakeLU
is created.

Reviewed-by: iustinp

This will be passed to FakeLUs

Reviewed-by: iustinp

Reviewed-by: iustinp

Since the ConfigWriter now handles its own locking it's not necessary to
have a specific level for the config in the Locking Manager anymore.
This patch thus removes it, and all the unittest calls that used it, or
depended on it being present.

Reviewed-by: iustinp

Since we share the ConfigWriter we need somehow to make sure that
accessing it is properly synchronized. We'll do it using the
locking.ssynchronized decorator and a module-private shared lock.

This patch also renames a few functions, which were called inside the
ConfigWriter, to a private version _UnlockedFunctionName, and exports
the synchronized public ones. The internal callers, which are already
synchronized, are then changed to use the _Unlocked version, to prevent
double locking.

Reviewed-by: iustinp

This patch creates a new decorator function ssynchronized in the locking
library, which takes as input a SharedLock, and synchronizes access to
the decorated functions using it. The usual SharedLock semantics apply,
so it's possible to call more than one synchronized function at the same
time, when the lock is acquired in shared mode, and still protect
against exclusive access.

The patch also adds a few unit test to check the basic decorator's
functionality, and to provide an example on how to use it.

Reviewed-by: iustinp