port forward of patch from revision 1690 with following message:

Patch on revision 1686 used the wrong field: ial.name, which is the instance
name and not the iallocator name. self.op.iallocator is the right field.

Sorry for this inconvenience.

Reviewed-by: imsnah

This patch fixes a broken format string. It's expecting 3 parameters, but only
gets 2. This change will add the missing parameter. This is a forward-port
of the fix in Ganeti 1.2

Reviewed-by: imsnah

It's not complete, but I hope it's up to date. It's restructured text,
but no make rules or such for html output are needed, it's readable
as-is.

Reviewed-by: imsnah

A couple of more modules are using the obsolete logger functions, config
being one of them.

Reviewed-by: imsnah

With some todos remaining, this patch switches the DRBD devices to use
the passed minors, and the cmdlib code (add instance and replace disks)
to request and assign minors to the DRBD disks.

Todos:
  - look at the disk RPC calls to see which can be optimized away, since
    we now know the minor beforehand
  - remove the _FindUnusedMinor usage from the few places it's still
    used (not for actual disks, but for temporary use in meta devs) and
    eventually replace with _CheckMinorUnused or such

Of course, this and/or the previous two patches break existing clusters.
Again.

Reviewed-by: imsnah

This patch adds support for allocating static minors.

Like for the LVM uuids, we add a new cache for the temporarily allocated
requests, and the users of the new methods must manually clear the
cache. If this doesn't happen, at worst we lose some minors.

Todos remaining:
  - implement duplicate check at configuration load, and at instance
    add/instance update
  - investigate automatically cleaning the cache on instance add/update
    if the minors/instance/node pairs match

Reviewed-by: imsnah

The code in 'updating instance configuration' section of the replace
disks with change secondary node was setting a wrong new logical_id for
the drbd devices (only set the new node, not the new minor). The patch
fixes this by remembering the new logical_id that we use for activating
the drbd and reusing it (instead of recomputing).

Since the minors are not needed to be allocated before the LVs are
created, we also move the minor allocation step (currently as None)
after this step.

Reviewed-by: imsnah

This patch converts the DRBD disks to contain also a minor (per each
node) attribute. This minor is not yet used and is always initialized
with None, so the patch does not have any real-world impact - except for
automatically upgrading config files (it adds the minors as None, None).

Reviewed-by: imsnah

Currently when not locking all nodes/instances are returned, regardless
if the user asked only for some of them. With this patch we return to
the previous behaviour:
  - if no names are specified return info on all current ones
  - if some names are specified and are not found give an error
  - otherwise return only info on the specified names

Reviewed-by: iustinp

There is no such feature in trunk yet.

Reviewed-by: iustinp

It abstracts exception handling and is like a complement to
utils.WriteFile.

Reviewed-by: iustinp

The previous description was unclear. Reported by Guido Trotter.

Reviewed-by: ultrotter

Merged from branches/ganeti/ganeti-1.2 r1648
Use static values for new hvm instance flags

Reviewed-by: iustinp

GetAllInstancesInfo used "node" as an iterator name. Change it to
instance to make it less confusing.

Reviewed-by: iustinp

It used to refer to "nodes", which was confusing.

Reviewed-by: iustinp

For now we lock the instance/node for adding/deleting tags from it, but
we could probably in the future do without, with more support from the
config for atomic operations.

Reviewed-by: iustinp

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Using the new add/remove infrastructure this becomes pretty easy! :)

Reviewed-by: imsnah

Finally, instance create on different node, without iallocator, can run
in parallel. Iallocator usage still needs all nodes to be locked,
unfortunately. As a bonus most checks which could have been moved to
ExpandNames, before any locking is done.

Reviewed-by: imsnah

With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be
removed again, unless some action is taken.

Reviewed-by: imsnah

This patch bans add() on a half-acquired set. This behavior was
previously possible, but created a deadlock if someone tried to acquire
the set-lock in the meantime, and thus is now forbidden. The
testAddRemove unit test is fixed for this new behavior, and includes a
few more lines of testing and a new testConcurrentSetLockAdd function
tests its behavior in the concurrent case.

Reviewed-by: imsnah

Reviewed-by: imsnah

Now that is_owned is public we don't need to play games at the end of an
LU. If we're still owning anything we just release it.

Reviewed-by: imsnah

This is a public version of the private function we already had.
We don't just change the previous version because it had lots of users
in the library itself and in the testing code.

Reviewed-by: imsnah

If the set-lock is acquired, currently, the _names function will fail on
a double acquire of a non-recursive lock. This patch fixes the behavior,
and some lines of code added to the testAcquireSetLock test check that
this and other functioins behave properly.

Reviewed-by: imsnah

Finish the --submit changes with these two, which (because they are
multi-opcode commands) require special handling.

Reviewed-by: ultrotter

We didn't decide yet what exactly it should do with failed nodes.

Reviewed-by: ultrotter

This patch is similar to the node patch (rev 1650). We disable locking
of instance (and nodes) if we only query static information.

Reviewed-by: ultrotter

In order to be able to query instance without locking them, we need the
same atomic query of multiple instances as for nodes.

Reviewed-by: ultrotter

This patch splits the GetInstanceInfo and GetInstanceList methods into
two parts, one locked one _Unlocked similar to the way nodes are
queried.

Reviewed-by: ultrotter

In the gnt-instance script, _ExpandNames() uses jobs to query instance
names. This is not optimal, so we change it to use queries.

Reviewed-by: ultrotter

This patch adds support for the “--submit” parameter in the gnt-instance
script, for the commands where it makes sense.

Reviewed-by: ultrotter

The "sys.exit(0)" was not nice as you couldn't differentiate it from
other exit codes. We change this to a specially defined exception for
this, so that multi-opcode commands can handle this nicely.

Reviewed-by: imsnah

Currently, OpQueryNodes is locking all nodes (in shared mode), which
will also block the special case of querying only for the node names
(this is needed for gnt-cluster command, for example). There is no
logical requirement to not give the administrator enough power if she/he
knows what to do, so it would be logical that querying the node names
works without a lock.

The patch changes the LUQuerytNodes.ExpandNodes to only request locking
when the selected fields contain dynamic attributes, and a small change
in the Exec() method to use the new, atomic query method from
ConfigWriter.

Reviewed-by: ultrotter

The patch adds a new function to export all node information at once
(i.e. atomically with respect to the configuration lock).

Reviewed-by: ultrotter

Reviewed-by: iustinp

Otherwise, corruption could occur in some corner cases. E.g. when
LeaveNode is running in a child and is in the process of removing
queue files, the main process gets killed, started again and gets
a request to update the queue. This is rather extreme corner case,
but we should opt for safety.

Reviewed-by: iustinp

This is an initial version of the master startup checks. It's a very
rudimentary change, however in normal usage (an old master was started,
the rest of the cluster is functioning normally) it will succeed in
preventing wrong startups.

Reviewed-by: imsnah