With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be
removed again, unless some action is taken.

Reviewed-by: imsnah

This patch bans add() on a half-acquired set. This behavior was
previously possible, but created a deadlock if someone tried to acquire
the set-lock in the meantime, and thus is now forbidden. The
testAddRemove unit test is fixed for this new behavior, and includes a
few more lines of testing and a new testConcurrentSetLockAdd function
tests its behavior in the concurrent case.

Reviewed-by: imsnah

Reviewed-by: imsnah

Now that is_owned is public we don't need to play games at the end of an
LU. If we're still owning anything we just release it.

Reviewed-by: imsnah

This is a public version of the private function we already had.
We don't just change the previous version because it had lots of users
in the library itself and in the testing code.

Reviewed-by: imsnah

If the set-lock is acquired, currently, the _names function will fail on
a double acquire of a non-recursive lock. This patch fixes the behavior,
and some lines of code added to the testAcquireSetLock test check that
this and other functioins behave properly.

Reviewed-by: imsnah

Finish the --submit changes with these two, which (because they are
multi-opcode commands) require special handling.

Reviewed-by: ultrotter

We didn't decide yet what exactly it should do with failed nodes.

Reviewed-by: ultrotter

This patch is similar to the node patch (rev 1650). We disable locking
of instance (and nodes) if we only query static information.

Reviewed-by: ultrotter

In order to be able to query instance without locking them, we need the
same atomic query of multiple instances as for nodes.

Reviewed-by: ultrotter

This patch splits the GetInstanceInfo and GetInstanceList methods into
two parts, one locked one _Unlocked similar to the way nodes are
queried.

Reviewed-by: ultrotter

In the gnt-instance script, _ExpandNames() uses jobs to query instance
names. This is not optimal, so we change it to use queries.

Reviewed-by: ultrotter

This patch adds support for the “--submit” parameter in the gnt-instance
script, for the commands where it makes sense.

Reviewed-by: ultrotter

The "sys.exit(0)" was not nice as you couldn't differentiate it from
other exit codes. We change this to a specially defined exception for
this, so that multi-opcode commands can handle this nicely.

Reviewed-by: imsnah

Currently, OpQueryNodes is locking all nodes (in shared mode), which
will also block the special case of querying only for the node names
(this is needed for gnt-cluster command, for example). There is no
logical requirement to not give the administrator enough power if she/he
knows what to do, so it would be logical that querying the node names
works without a lock.

The patch changes the LUQuerytNodes.ExpandNodes to only request locking
when the selected fields contain dynamic attributes, and a small change
in the Exec() method to use the new, atomic query method from
ConfigWriter.

Reviewed-by: ultrotter

The patch adds a new function to export all node information at once
(i.e. atomically with respect to the configuration lock).

Reviewed-by: ultrotter

Reviewed-by: iustinp

Otherwise, corruption could occur in some corner cases. E.g. when
LeaveNode is running in a child and is in the process of removing
queue files, the main process gets killed, started again and gets
a request to update the queue. This is rather extreme corner case,
but we should opt for safety.

Reviewed-by: iustinp

This is an initial version of the master startup checks. It's a very
rudimentary change, however in normal usage (an old master was started,
the rest of the cluster is functioning normally) it will succeed in
preventing wrong startups.

Reviewed-by: imsnah

We create a multi-node call so that querying all nodes for agreement
will be fast.

Reviewed-by: imsnah

The _GetMasterInfo() function needs to export the master name too to be
useful in master safety checks. This patch makes it a public (no _)
function and adds a third element in the return tuple. Its callers are
modified too.

Reviewed-by: imsnah

Reviewed-by: iustinp

These are two easy querying LUs which require shared access to all
nodes/instances.

Reviewed-by: iustinp

This is the most complex parallelization so far. We have to lock one
instance (and its nodes) plus one more node if doing a remote replace,
or all nodes if doing a remote replace with iallocator.

Reviewed-by: iustinp

This will be used to lock the instance's nodes in addition to some more.

Reviewed-by: iustinp

This function was incompatible with the new locking system, and its
usage has been removed from the code. For now LUs share code by calling
common module-private functions in cmdlib.py, in the future they will
use tasklets (when those will be implemented).

Reviewed-by: iustinp

Now that they are not used in other opcodes by chaining,
this can easily be done.

Reviewed-by: iustinp

The calls to OpActivateInstanceDisks and OpDeactivateInstanceDisks has
been replaced by _StartInstanceDisks and _SafeShutdownInstanceDisks
respectively. This is the last usage of ChainOpCode.

Reviewed-by: iustinp

This new function checks whether an instance is running, before shutting
down its disks. This is what the Exec() of LUDeactivateInstanceDisks
did, so that is replaced by a call to this function.

Reviewed-by: iustinp

s/locking.LEVEL_INSTANCES/locking.LEVEL_INSTANCE/

Reviewed-by: iustinp

This constant replaces what we used to write in recalculate_locks, and
represents the lock recalculation mode. It lives in constants.py because
it's used only in cmdlib, and thus doesn't deal with the locking library
by itself.

Reviewed-by: iustinp

self._RunAllocator() sets self.op.remote_node, but doesn't return the
new remote node. If we set it to the return value of the function we
basically reset it to None, and iallocator is never run.

Reviewed-by: imsnah

This helps to prevent complete deadlocks.

Reviewed-by: iustinp

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

Fixing the check in gnt-cluster, or gnt-cluster verify-disks is broken.
Since the version in 1.2 used to return a tuple we'll accept both.

Reviewed-by: iustinp

Unfortunately for the first version we need to lock all nodes. The patch
discusses why this is and discuss ways to improve this in the future.

Reviewed-by: iustinp

Reviewed-by: iustinp

When rebooting an instance and we're not changing it's disks status (all
the cases except in a "full" reboot) we can lock just its primary node.

Reviewed-by: iustinp

As the name says when the flag is on (the default is off) only the
primary nodes are locked, as opposed to all of them.

Reviewed-by: iustinp