Catch the StopIteration exception and return None to the temporary_ips
reservation manager, so that it fails at the 64-attempt boundary.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

The retry counter used in TemporaryReservationManager.Generate() was not
decremented for each failed retry, as presumably intended. This patch
decrements the retry counter after each failed attempt.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Call cluster.SimpleFillNIC to get the default link when link is None.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Change the semantics of ReleaseIp, so that it can be called from outside
without knowing the exact network UUID, by passing only node and link.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add a new method, TemporaryReservationManager.GetECReserved(), which returns all
reservations held by a given ec_id.

Also add a new method, ConfigWriter._UnlockedCommitReservedIps(), which can be
called to commit all temporarily reserved IP addresses to the respective
address pools. For this to work, the IP reservation manager must hold both, the
address and the network UUID of the respective address pool.

Finally, we automatically commit IP address reservations during
ConfigWriter.Update() of Instance objects. For this to work, we add an optional
ec_id keyword, which must hold the calling job's unique id to get the
relevant reservations.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Prototype implementation of IP address management during instance
addition/removal.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add helper methods to ganeti.config.ConfigWriter to handle Network addition and
removal, IP address reservation and commitment to the address pools.

Also add the “networks” ssconf key to make network names/UUIDs available to all
nodes via ssconf.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

This patch introduces the following changes to lib.config and lib.objects to
facilitate IP pool management.

- Add a "networks" key to objects.cluster to hold link -> subnet relations.
  Each link is assumed to be associated with at most one IPv4 and at most one
  IPv6 subnet. Currently only IPv4 subnets are used.
- Add a TemporaryReservationManager for IP address reservations.
- Add config.{_UnlockedCommitIp,CommitIp,_UnlockedReleaseIp,ReleaseIp} to write
  IP pool changes to the config file.
- Add config.{ReserveIp,GenerateIp} for use with the _temporary_ips
  TemporaryReservationManager.
- Commit succesful reservations/releases to the IP pools in config.AddInstance
  and config.RemoveInstance

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

ConfigWriter.ReserveLV() and Configwriter.ReserveMAC() called
TemporaryReservationManager.Reserve() with the ec_id and resource arguments
swapped. As a result, two reservation attempts for the same resource type
within the same LU would fail, even if the resources requested were different,
e.g.:

  $ gnt-instance add -t sharedfile -o debootstrap+default \
       --net 0:mac=00:01:02:03:04:00 \
       --net 1:mac=00:01:02:03:04:ff \
       --disk 0:size=2g  test_instance
  Failure: prerequisites not met for this operation:
  error type: resource_not_unique, error details:
  MAC address 00:01:02:03:04:ff already in use in cluster

This patch fixes the argument order in the call to Reserve().

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

The original code would get all node information and their groups
without before acquiring the necessary locks. With this patch the node
information is only retrieved once all locks have been acquired. Groups
are locked optimistically and verified after acquiring the node locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For whatever reason, my test cluster managed to acquire
shared_file_storage_dir with a None value, instead of empty
string. This is not flagged in masterd itself, but the node daemon
will fail in writing the value to disk, as it calls len() on the
received value.

Since this is a bad case, we should detect it as soon as possible (we
basically shouldn't be able to set it), but in the meantime we at
least prevent ssconf writes with such values.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch introduces core file storage support, consisting of the following:

A configure-time switch for enabling/disabling shared file storage
support and controlling the shared file storage location:
--with-shared-file-storage-dir=.  Shared file storage configuration is then
available as _autoconf.ENABLE_SHARED_FILE_STORAGE and
_autoconf.SHARED_FILE_STORAGE_DIR and there is a cluster-wide ssconf
key named "shared_file_storage_dir" for changing the file location.

A new disk template named "sharedfile" (DT_SHARED_FILE), using
ganeti.bdev.FileStorage.

Auxiliary functions in lib/config.py to handle shared file storage.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: small style fixes]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This allows calling of _UnlockedLookupNodeGroup() from within
AddNodeGroup()

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This adds checking (in the configuration) for invalid be, nd and nic
params. The code is a bit tricky as nd params are at cluster,
nodegroup and node level, nicparams are at cluster and nic level,
whereas beparams are at cluster and instance level.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This just adds a 'cluster' local variable for reducing duplication.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- Add check in ConfigWriter to prevent last node group from being
  removed
- Tidy up error message a bit

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The LVM queries should only be done for vm_capable nodes. In order to
do this, we also add a new ConfigWriter method to abstract that query.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This can be set at group creation time and via OpSetGroupParams. The default
is "preferred", and existing node groups from previous Ganeti version will
get the attribute set to this value.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the code wrongly changes the disk logical/physical id
component representing the path from "$storage_dir/$iname/disk$seq" to
"$storage_dir/$iname/disk/$seq" (note the additional slash) breaking the
rename.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In particular:

  - add AddNodeGroup and RemoveNodeGroup in config.py

  - refactor _UpgradeConfig to make use of _UnlockedAddNodeGroup

  - add support for objects.NodeGroup in ConfigWriter.Update; add missing
    "serial_no" attribute to NodeGroup objects

  - update NodeGroup.UpgradeConfig to set mtime and serial_no if missing
    (the refactored code in _UpgradeConfig was not settting mtime nor ctime)

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds a errors.ECODE_NOENT for OpPrereqError in LookupNodeGroup.

Signed-off-by: Adeodato Simo <dato@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Though code uses "nodegroup" as data member name, strings that will be
read by the user should use "node group" (with a space), since "nodegroup"
is not a word.

Signed-off-by: Adeodato Simo <dato@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

And fix an error message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Note: It appears this has been around since the initial checkin of
TemporaryReservationManager. I have no idea what this could break, so
someone else may want to test this more thoroughly.

Signed-off-by: David Knowles <dknowles@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This fixes a bug where the ssconf_instance_list file was
not updated after an instance rename.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This can be used to compute a node's instances easily, and a small
function to get all non-vm_capable nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds the flag and some initial handling. The rest of the changes,
for cmdlib, come in a separate patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In the context of a node, its group has (at least today) only one
meaning, that is the node's node group. As such, we rename
node.nodegroup to just node.group.

Note: if we want to keep node in there, it should be at least
node_group, for consistency with the other node attributes.

Similarly, we rename the OpAddNode nodegroup attribute to group.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If the configuration file doesn't denote this node as master, we prevent
startup. This would have detected our previous race condition more
easily, hence we add it as a permanent check.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should fix the case where there are two masters that both try to
distribute the configuration file to the cluster. The first one that does so,
will "win" the ownership of the config.data.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, masterd startup with old software versions is very confusing
for users: we present two tracebacks, with a message in the middle about
"version mismatch". This can lead to users believing that all that needs
to be done is to fix the config file.

This patch attempts to improve this by handling this case in masterd
itself (not in the child), and showing a more friendly message for this
case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>