Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Even for snapshot we looked at the biggest free pv space even though
the vg might have fit the snapshot we aborted if one of the pvs was too
small. This patch fixes this by looking at the vg size instead of the pv
size.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the mac checker regex could match a corner case of
11:22:33:44:55:66: (one extra colon at the end). We fix this, and we
also move the regex compilation outside of this function, at module
level.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Don't abort failover if instance shutdown doesn't work on a node marked
offline. The node is offline, so the instances living on it are too. Before
you had to use --ignore-consistency to archieve that.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In order to support priorities, the processing of jobs needs to be
changed. Instead of processing jobs as a whole, the code is changed to
process one opcode at a time and then return to the queue. See the
Ganeti 2.3 design document for details.

This patch does not yet use priorities for acquiring locks.

The enclosed unittests increase the test coverage of jqueue.py from
about 34% to 58%. Please note that they also test some parts not added
by this patch, but testing them became only possible with some
infrastructure added by this patch. For the first time, many
implications and assumptions for the job queue are codified in these
unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A small helper function is added to make this easier. Priorities are not
yet used in all necessary places.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

New KVMs (0.12.1.2-el6 and 0.13.5 tested) exit immediately after
unsuccessful network connection when they are in "-incoming" mode. The
simple check netutils.TcpPing causes remote kvm to exit so the migration
will always fail. This check is also redundant by the way as if the
network port is closed migration will fail with guarantee. This commit
removes this check.

Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

They'll be used for job priorities. Also add an empty line to
gnt-os where it's missing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was forgotten in commit 099b2870.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch bails out in early stage, if the user invoking the daemon
doesn't match the user set at configure time.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, opcodes like NODE_EVAC_STRATEGY look bad:

  89684 error   NODE_EVAC_STRATEGY([u'node3'])

With this patch, we try to render list arguments a little bit better:

  89684 error   NODE_EVAC_STRATEGY(node3)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

A call to logging.debug was missing an argument, leading to complaints
on stderr at runtime.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Moving the internals of this function will allow it to be used from
unittests in the future. Splitting this into a pure, side-effect free
function and an impure one makes the pure function easily testable.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The parameter “name” is be None for cluster tags.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Otherwise parameter verification in the master daemon fails.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Please note that if the pid file is broken or missing we'll not catch
the process (if any is running) and it's up to the user to fix this state

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since the nodegroups dict is indexed by uuid duplicate names might
happen as a result of bugs. Add a check to prevent them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch also ensures that the initial configuration has all the
needed UUIDs and that they are unique, by using a
TemporaryReservationManager inside InitConfit to generate them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We autocomplete both by nodegroup name and uuid.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If a node doesn't have a node group we'll upgrade the config making it
the cluster default. Also the node add and removal operations are
changed to set/clear the node group correctly. Finally we populate the
"members" list of nodegroups on config load with  the value from the
nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows a node group to be looked up by name or uuid.
If no nodegroup is specified and only one exists, that one is returned.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since the uuid is immutable the probability of it getting out of sync
between the object and the dict key is very low. Still, checking is
cheap, so we do it to be more sure nothing is wrong.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If no node groups exist we'll create a new default empty one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also:
  - reformat the __slots__ declaration of ConfigData
  - call UpgradeConfig on each node groups to handle future upgrades
  - add nodegroups special case in {To,From}Dict
  - add nodegroups to _AllUUIDObjects

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When _ContainerFromDicts is called on some element which doesn't exist
in the config, because it is yet to be upgraded, it will receive its
value as None. We take care of this case by using an empty element of
the required target type.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The "members" slot of this object is not serialized, and is discarded
when deserializing, initializing it explicitly to [].

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Quoting the design document: “Submitted opcodes can have one of the priorities
listed below. Other priorities are reserved for internal use”. Submitting jobs
at priority -20 should not be allowed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows clients to submit opcodes with a priority. Except for being
tracked by the job queue, it is not yet used by any code.

Unittests for jqueue._QueuedOpCode and jqueue._QueuedJob are provided for
the first time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is no longer needed with the new lock monitor. One callback is kept to
check for cancelled jobs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 4008c8ed.

While it worked in my initial tests, I've now found cases where this doesn't
work properly as it is. More work is needed and will be done as part of the
Ganeti 2.3 job queue changes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was introduced in efaa9b06.

 in OSCoreEnv:
  inst_os.name is pure operating system name (without variant) as variant is stripped
   in OSFromDisk(). So we always get variant = inst_os.supported_variants[0] (first
   variant in variants list).
  Adding argument os_name with full name (including variant) solves this problem.

Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
[modified by iustin to handle the call to OSCoreEnv from ValidateOS too]
Reviewed-by: Michael Hanselmann <hansmi@google.com>

My bad for not seeing this before:
R0201:614:HttpBase.GetSslCiphers: Method could be a function

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Users of this class, such as the RAPI server, might want to override or adjust
the default SSL cipher defined in a constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>