GitLab

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds the flag and some initial handling. The rest of the changes,
for cmdlib, come in a separate patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In the context of a node, its group has (at least today) only one
meaning, that is the node's node group. As such, we rename
node.nodegroup to just node.group.

Note: if we want to keep node in there, it should be at least
node_group, for consistency with the other node attributes.

Similarly, we rename the OpAddNode nodegroup attribute to group.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is the first step for the support of wiping block devices prior
to creation of the instance.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Per the mailing list discussion, rename _oss to _os, both in cluster parameters
and in the rest of the code.

This is just an s/_oss/_os, with the exception of a small bit of cleanup
around the helper_os function in cmdlib.py.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The new variables are:

- a list of hidden OSes, that should not be displayed to the users in
  interactive selection (e.g. reinstall); however, if they are selected, they
  can be used
- a list of OSes that should be hidden and blocked from install-time selection

The filtering will apply at pure OS name level, not OS+variant level.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the computation of the 'pure' name or the variant is
hardcoded and spread around the functions that need it. This is not
nice, and in the future we'd spread it even more with more usage of
variants/pure os names.

This patch abstracts these functions into the OS class, and then
replaces the hardcoded uses with the new functions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If a node doesn't have a node group we'll upgrade the config making it
the cluster default. Also the node add and removal operations are
changed to set/clear the node group correctly. Finally we populate the
"members" list of nodegroups on config load with  the value from the
nodes.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also:
  - reformat the __slots__ declaration of ConfigData
  - call UpgradeConfig on each node groups to handle future upgrades
  - add nodegroups special case in {To,From}Dict
  - add nodegroups to _AllUUIDObjects
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When _ContainerFromDicts is called on some element which doesn't exist
in the config, because it is yet to be upgraded, it will receive its
value as None. We take care of this case by using an empty element of
the required target type.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The "members" slot of this object is not serialized, and is discarded
when deserializing, initializing it explicitly to [].
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We expose the ip_version (4, 6) to the external interface and internally
we convert it to ip_family (AF_INET=2, AF_INET6=10). This makes the code
more concise as all functions deal with family rather than version.

This patch does not yet expose this parameter via gnt-cluster, but only uses
the constant IP4_VERSION. This will be enabled in a future patch.
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This parameter, which is a list of regular expression patterns, will
make cluster verify ignore any such LVs. It will not prevent creation or
removal of such volumes by the backend code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Add a cluster parameter to hold the iallocator that will be used by default
when required and no alternative (manually-specified iallocator or
manually-specified node(s)) is given.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The comments seem to imply we can remove deprecated slots after 2.1 is
released. This is not quite true, if we still want to support upgrading
straight from 2.0. So let's specify better that they can be removed only
after upgrading from 2.0 can be deprecated.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The patch also modifies the instance RPC calls to fill the osparameters
correctly with the cluster defaults, and exports the OS parameters in
the instance/OS environment.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since OS objects are not stored in the configuration, we cannot put
os_hvp there, therefore the TODO is obsolete…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the existing cluster.Fill* functions take as argument an
instance. This means that in any case where we don't have an actual
instance object, we have to resort to calling the low-level
objects.FillDict function.

This is bad for two reasons:
- we have to know of, and we hardcode, the cluster object internals
  (e.g. that the nicparams are stored in a dict indexed by group)
- which can result in subtle bugs, if the underlying storage mechanisms
  change

This patch adds a lower-level implementation SimpleFillHV for FillHV and
SimpleFillBE for FillBE, and adds a completely new SimpleFillNIC (all
use cases until now hardcoded cluster.nicparams[constant.PP_DEFAULT]
directly); it then uses these new functions in cmdlib.py.

A side effect is that _CheckNicsBridgesExist loses the 'profile'
parameter, which was unused. If it's needed, we should add it later via
a proper profile parameter to SimpleFillNIC.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This reports the amount of data transferred and the throughput (averaged
over 60 seconds) to the master daemon. While not yet fully implemented,
once the export scripts report the expected data size, we can even provide
an ETA and percentage.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

For exports to/imports from the same machine, compression will
not be used anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The X509 key name and CA are passed from cmdlib all the way to
the backend import/export daemon. With the addition of an option
to choose the compression method, another parameter would have
to be passed all the way. By moving these options to a separate
object, adding new ones will become much easier.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This backend daemon for instance import and export will be used to
transfer instance data to other machines. It is implemented in a generic
way to support different ways of data input and output. The third-party
program “socat”, which is already used by the KVM hypervisor abstraction,
is used to connect to remote machines using SSL/TLS. After starting the
child processes in a separate process group, the import/export daemon
monitors their output and updates a status file regularily. This status
file can then be read by ganeti-noded (not in this patch).

Three I/O methods are supported: Raw disk, file and script. Each of these
can be used for import and export.

Similar to daemon-util, an incomplete set of tests written in Bash is
included.

Two future enhancements are planned:
- Run parts of the command chain as a dedicated user (privilege
  separation).
- Currently users of this daemon have to poll the status file while data
  is transferred. This is inefficient and creates unnecessary delays. By
  adding “dd” into the chain and sending it SIGUSR1 regularily, we can get
  some statistics, optimize the polling frequenc and even provide the user
  with an ETA (which isn't available with all current methods to
  import/export instance data).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the FillHV method is the one that does the cluster hvparams +
os hvparams merger. However, in some cases we need to do just this,
without adding the instance hvparams on top.

This patch adds a function to compute just this (hv + os hvp
combination) default dict, and modifies FillHV to use it to build the
final dict.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will be used to conditionally enable the watcher node maintenance
feature.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently SerializableConfigParser.Loads is a static method that returns
a SerializableConfigParser. With this patch we change it to a class
method that returns a member of the class. This way a subclass calling
Loads on itself will get its own member, rather than a bare
SerializableConfigParser.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch implements all modifications to support per-os-hypervisor
parameters in the framework.
Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since in objects we don't have access to utils.py, we add a warning that
the result value from objects.Disk.StaticDevPath might not be a valid
path, and change its only caller to validate the path.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 154b9580

 changed (correctly) the __slots__ usage, but this broke
dumpers/loaders since we relied directly on the own class __slots__
field.

To compensate, we introduce a simple function for computing the slots
across all parent classes (if any), and use this instead of __slots__
directly.

Note: the _all_slots() function is duplicated between objects.py and
opcodes.py, but the only other options is to introduce a lang.py for
such very basic language items.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>