Use new NormalizeAndValidateThreeOctetMacPrefix() util function in
LUNetworkAdd/LUNetworkSetParams to validate network's MAC prefix.
Additionally, move the check in CheckArguments() in the case of
LUNetworkAdd.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

All node locks are acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This opcode acquires all node resource locks, which conflicts with
instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

See comment in code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

See comment in code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

If the lock was acquired in the first place (only when an iallocator is
used), it is released as soon as possible.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Changing instances' groups shouldn't conflict with instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

If locking is used (usually by ganeti-watcher), node allocations must be
temporarily blocked.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

In the future instance creations might have a lock on all nodes as was
the case until the implementation of opportunistic locking. Nodes for
which the lock is not held will be shown to the iallocator plugin as if
they were marked offline.

This patch adds a new parameter named “node_whitelist” to
“IAReqInstanceAlloc”. If set to a list, only nodes contained within are
shown as online.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

In some cases it is useful to ignore the output of and avoid mentioning
successful commands. One would be when looking for a certain string in
a file:

  $ gnt-cluster command egrep -q '^testing$' /etc/...

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Comments with a colon after the hash sign (“#:”) show up in the
epydoc output.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The node resource locks were not set correctly on instance import.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If TIsLength is applied to a non-container item, it will fail (type
error) due to invalid application of len(). Since this can happen on
user-supplied data, we add an explicit TList/TTuple check (the TTuple
test is a new one).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

All resources are acquired and opportunistic instance creations would
fail. Also add a TODO.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Opportunistic locks are not yet used. This patch changes
LUInstanceCreate to acquire the node allocation lock to avoid conflicts
with other opcodes acquiring many node locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

See code for an explanatory comment. The lock is released as soon as
possible.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUNetworkConnect/Disconnect looks up a nodegroup's instances for
conflicting IPs. To do so, use GetNodeGroupInstances() and
GetMulitInstanceInfo().

Additionally, check if the correct locks were acquired.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The network management code needs to verify a MAC address prefix.
Instead of (ab)using NormalizeAndValidateMac, clean code should be used.
Unit tests for NormalizeAndValidateMac are updated and new ones for
NormalizeAndValidateThreeOctetMacPrefix are added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This code is going to be used by a new utility for setting up the node
daemon. Unit tests are updated/added.

Additionally, the certificate and key stored in “server.pem” are
verified, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Similar to “share_locks”, a new dictionary containing booleans for each
locking level is added to “cmdlib.LogicalUnit”. Logical units wanting to
make use of opportunistic locks will be able to configure this
dictionary accordingly.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Just forwarding the parameter, nothing more.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds a new parameter to “LockSet.acquire” named
“opportunistic”. When enabled the lockset will try to acquire as many
locks as possible, but it won't wait for them (with the exception of the
lockset-internal lock in case the whole set is acquired). This is
implemented by using a timeout of 0 seconds when acquiring individual
locks. Commit 03c5291c made such acquisitions significantly cheaper.

The most complicated code included in this patch is probably the helper
function used to determine which mode to use and which timeout functions
are needed.

Full unit tests are included for the new and changed code
(“locking.py”'s overall coverage is at 97%).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Configuring a node daemon on a newly added node will need all ssconf
values.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 224ff0f7 modified the node SSH setup to use the system's SSH
client. Before that Paramiko was used. It's not entirely clear whehter
the latter ever supported IPv6 properly, but with this patch
“bootstrap.RunNodeSetupCmd” is changed to use it if configured. The code
for doing so is taken from “bootstrap.SetupNodeDaemon”, which is going
to use “bootstrap.RunNodeSetupCmd”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Part of the code used for running “prepare-node-join” can be re-used for
running a tool to configure the node daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

A new utility for configuring the node daemon will support a dry-run
mode. This patch adds the necessary functionality to
“ssconf.SimpleStore” and provides comprehensive tests for
“SimpleStore.WriteFiles”. To enable the latter, a testing-only parameter
specifying the lockfile was added to the class.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The new utility for configuring the node daemon will have to check
whether it received valid ssconf names.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In case conflicts are checked, log warnings if nodes' IPs cannot
be reserved.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Ensure that locks are held only if needed.

Add conflicts_check in OpNetworkAdd. This is needed if we want to
check whether nodes/master IPs are included in network.

Depending on conflicts_check value, we have to hold node/instance locks
during OpNetworkAdd/OpNetworkConnect or not.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This makes the OpCode more consistent with the other opcodes. The
downside is incompatibility when upgrading from 2.6, but since this is
a test opcode it shouldn't be problematic.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This field is used with just changed description in about 10 opcodes,
so unifying it makes things simpler for future potential changes to
the field type.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This also corrects a docstring in OpBackupExport on the Python side.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Commit 12e62af5 (“Adding the new opcode for multi-allocation”)
introduced a "bad" default value; while porting this to Haskell, I
realised this is wrong.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The order of the calls to “ctx.use_privatekey” and “ctx.use_certificate”
was wrong, leading to an exception being thrown.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This code will also be used by the node daemon setup utility.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

TValueNone checks if a value is "none" and TMaybeValueNone is a wrapper
of TOr(TValueNone, x). This is used by OpNetworkSetParam in order to
reset a network value (e.g. mac_prefix, gateway, etc.)

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also remove a superfluous empty line in test file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds descriptors to the “_CheckCIDR*” functions in opcodes
and improves the descriptions generated by “ht.TInstanceOf”, thereby
indirectly fixing bad type descriptions in the RAPI documentation.

Before this patch:
- (String and (<function _CheckCIDRAddrNotation at 0x2f…>))
- (Instance of (<class 'ganeti.opcodes.OpInstanceCreate'>))

After this patch:
- (String and (IPv4 address))
- (Instance of (ganeti.opcodes.OpInstanceCreate))

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This bug was introduced by my commit "Check ispecs against ipolicy on
instance modify", because I did not filter for the 'add' action. Thus it
would try to read the size parameter also when removing disks.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>