Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Even for snapshot we looked at the biggest free pv space even though
the vg might have fit the snapshot we aborted if one of the pvs was too
small. This patch fixes this by looking at the vg size instead of the pv
size.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Don't abort failover if instance shutdown doesn't work on a node marked
offline. The node is offline, so the instances living on it are too. Before
you had to use --ignore-consistency to archieve that.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

So far the “--dry-run” option is not documented at all. This also adds
documentation for the new “--priority” option.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In order to support priorities, the processing of jobs needs to be
changed. Instead of processing jobs as a whole, the code is changed to
process one opcode at a time and then return to the queue. See the
Ganeti 2.3 design document for details.

This patch does not yet use priorities for acquiring locks.

The enclosed unittests increase the test coverage of jqueue.py from
about 34% to 58%. Please note that they also test some parts not added
by this patch, but testing them became only possible with some
infrastructure added by this patch. For the first time, many
implications and assumptions for the job queue are codified in these
unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A small helper function is added to make this easier. Priorities are not
yet used in all necessary places.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was forgotten in commit 099b2870.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch bails out in early stage, if the user invoking the daemon
doesn't match the user set at configure time.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* devel-2.2:
  QA: Test more tag operations via RAPI
  qa_rapi: Fix indentation error
  opcode summary: improve display for list summaries
  cmdlib: Fix type of “name” parameter for tag operations
  rlib2: Set tag operation param “name” to None for cluster tags
  Fix OS_VARIANT variable setting

Conflicts:
	lib/backend.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This ensures tag operations work via RAPI. Also print requested URI to
stdout for debugging.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If no function to verify the result was specified, qa_rapi._DoTests
wouldn't return the result.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.2:
  cmdlib: Fix type of “name” parameter for tag operations
  rlib2: Set tag operation param “name” to None for cluster tags
  Fix OS_VARIANT variable setting

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, opcodes like NODE_EVAC_STRATEGY look bad:

  89684 error   NODE_EVAC_STRATEGY([u'node3'])

With this patch, we try to render list arguments a little bit better:

  89684 error   NODE_EVAC_STRATEGY(node3)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

By enabling all log levels, we ensure all calls are fully evaluated.
There was one case in the workerpool where a call to “logging.debug”
was wrong, but not caught in unittests because debug logging was
disabled.

The optional environment variable “LOGTOSTDERR” can be set to
write all log messages to stderr instead of /dev/null.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A call to logging.debug was missing an argument, leading to complaints
on stderr at runtime.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Moving the internals of this function will allow it to be used from
unittests in the future. Splitting this into a pure, side-effect free
function and an impure one makes the pure function easily testable.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.2:
  Fix case of MAC special-values
  Remove mcpu's ReportLocks callback
  Revert "jqueue: Resume jobs from “waitlock” status"

(no conflicts, took LGTM from original commit)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* devel-2.1:
  Fix case of MAC special-values

(no conflicts, took LGTM from original commit)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This fixes Debian bug #596347. An alternative would be to also allow
the uppercase form, but it's not nice I think.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The parameter “name” is be None for cluster tags.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Otherwise parameter verification in the master daemon fails.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Please note that if the pid file is broken or missing we'll not catch
the process (if any is running) and it's up to the user to fix this state

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since the nodegroups dict is indexed by uuid duplicate names might
happen as a result of bugs. Add a check to prevent them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch also ensures that the initial configuration has all the
needed UUIDs and that they are unique, by using a
TemporaryReservationManager inside InitConfit to generate them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We autocomplete both by nodegroup name and uuid.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If a node doesn't have a node group we'll upgrade the config making it
the cluster default. Also the node add and removal operations are
changed to set/clear the node group correctly. Finally we populate the
"members" list of nodegroups on config load with  the value from the
nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows a node group to be looked up by name or uuid.
If no nodegroup is specified and only one exists, that one is returned.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since the uuid is immutable the probability of it getting out of sync
between the object and the dict key is very low. Still, checking is
cheap, so we do it to be more sure nothing is wrong.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If no node groups exist we'll create a new default empty one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also:
  - reformat the __slots__ declaration of ConfigData
  - call UpgradeConfig on each node groups to handle future upgrades
  - add nodegroups special case in {To,From}Dict
  - add nodegroups to _AllUUIDObjects

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When _ContainerFromDicts is called on some element which doesn't exist
in the config, because it is yet to be upgraded, it will receive its
value as None. We take care of this case by using an empty element of
the required target type.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The "members" slot of this object is not serialized, and is discarded
when deserializing, initializing it explicitly to [].

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Quoting the design document: “Submitted opcodes can have one of the priorities
listed below. Other priorities are reserved for internal use”. Submitting jobs
at priority -20 should not be allowed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows clients to submit opcodes with a priority. Except for being
tracked by the job queue, it is not yet used by any code.

Unittests for jqueue._QueuedOpCode and jqueue._QueuedJob are provided for
the first time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>