GitLab

Add Gluster to Ganeti by essentially cloning the shared file behaviour
everywhere in the code base.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

A large part of the complexity in this function is due to the need
to translate from "template-specific" parameter names to
"template-agnostic" parameter names. This logic is complex and having
complex code for complex logic is okay.

However, for all other cases, the additional complexity is not needed.
Additionally this function would do the wrong thing silently for new,
unhandled disk templates by just not changing the resulting paramters.

This commit replaces the ad-hoc logic with a more generally useful
logic while leaving the DRBD code carefully alone.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add a cluster parameter to hold the iallocator parameters used
by the default instance allocator. Implement the option to
modify config.data, query config.data and upgrade man pages,
tests and cfgupgrade tool. The new default_iallocator_params is
an empty dict by default.
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Also add a method for the retrieval of "ndparams" (node parameters)
of groups, not just nodes.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

While it's generally better to beg for forgiveness than ask for
permission in Python, it's not a good idea to rely on exceptions to
model control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add pci slot in NIC/Disk objects. This slot will be used only
by hypervisor code. Currently only KVM will use it and store it
temporarily in runtime files.
Add HOTPLUG_* constants to define device types an hotplug actions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Removing a comment which falsely states that Ganeti will
stop supporting certain upgrades at some point.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch fixes issue 599. The problem was that with 2.9,
a sanity check was introduced that made sure that the
ipolicies' set of disk templates is a subset of the
cluster-wide enabled disk templates. In order to make
upgrading from 2.8 smoother we add functionaliy to adapt
the ipolicies to the enabled disk templates.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch renames the constant 'LDS_DRBD' to 'DTS_DRBD'
to make it consistent with the renaming of LD_* constants
to DT_* constants.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add the "access" parameter to the RBD configuration. Fix test broken
by this change.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

To make sure that the dynamic_params field is not serialized by accident
to the configuration file only include it in the serialization when
explicitly asked for.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The 'physical_id' field of disk objects is no longer used, so remove it.
Also, all references are removed together with the code which made sure
that the physical_id is up to date when transmitted over RPC.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The disk field 'physical_id' has to be kept up to date whenever a disk
object is sent to a node via RPC. This is done with the SetDiskID method
manually, which is a source of bugs.

This patch replaces the use of 'physical_id' with a new field names
'dynamic_params'. The RPC code is adapted to update this field whenever
a disk object is sent to a node. Furthermore, this field is only ever
set on copies of disk objects which don't get written to the
configuration file. On the node side, the use of 'physical_id' is
removed and the new dynamic parameters are used now for the same
purpose.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This patch adds auto-upgrade functionality to the disk
objects with respected to the switch from LD constants
to DT constants. Unit tests are provided.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes the somewhat over-eager renaming of
DISK_LD_DEFAULTS to DISK_DT_DEFAULTS, which clashes
with another occurence of DISK_DT_DEFAULTS. This
patch just undoes the renaming and additionally fixes
a test regarding shared file storage.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

LD_* constants are basically like DT_* constants, except
for that both file and shared file were mapped to file.
In order to not having to maintain three slightly different
sets of disk-related constants (DT, LD and ST), we merge
DT and LD here. This patch does the renaming of the
various constants and then fixes the handling of shared
file disks.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Upgrading the config with respect to the DRBD usermode
helper was so far based on a guess whether or not DRBD
is enabled by checking if there are any DRBD instances
running. This check can now be replaced to look at the
list of enabled disk templates in the cluster's
configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Fix some regular expressions so that they pass lint checks with newer versions
of pylint.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

In order to properly assert the called RPC method, __eq__ is implemented
on ConfigObject as well.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As outlined in issue 338, Ganeti failed to initialize a cluster if no
RSA SSH key is present on the master node. This patch extends Ganetis
support to DSA keys, so clusters with only DSA keys are possible now.

This fixes issue 338.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The online-update function of the cluster config so far
inspected the autoconf data to determine whether shared
file storage is enabled. This should now totally rely
on the options given at cluster initialization and thus
this legacy fallback is removed.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch removes the ENABLE_FILE_STORAGE completely
from the remaining files, such as Makefile, configure,
constants, pathutils and objects.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds some helper functions to the config and
storage utils which check whether a disk template is
enabled or not. The functions themselves are quite small
but they will be used quite often and therefore should
have a designated place.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add a new disk parameter which allows to choose the DRBD replication
protocol. The protocol is fixed to "C" during dual primary mode though.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

LUGroupVerifyDisks had some not very descriptive names which made it
hard to read and understand the code. This patch renames some methods
and variables to make it more readable.

Also, the return type of MapInstanceLvsToNodes was changed to contain
Instance objects instead of instance names. This enables easier
processing of the result.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

No longer index nodes by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary:
 * Change the index key to UUID in the configuration and the
   ConfigWriter, including all methods.
 * Change all cross-references to nodes to use UUID's.
 * External interfaces (command line interface, IAllocator interface,
   hook scripts, etc.) are kept stable.
 * RPC-calls can resolve UUID's as target node arguments, if the RPC
   runner is based on a ConfigWriter instance. The result dictionary is
   presented in the form the nodes are addressed: by UUID if UUID's were
   given, or by name if names were given.
 * Node UUID's are resolved in ExpandNames and then stored in the
   OpCode. This allows to check for node renames if the OpCode is
   reloaded after a cluster restart. This check is currently only done
   for single node parameters.
 * Variable names are renamed to follow the following pattern:
   - Suffix is 'node' or 'nodes': Variable holds Node objects
   - Suffix is 'name' or 'names': Variable holds node names
   - Suffix is 'uuid' or 'uuids': Variable holds node UUID's
 * Tests are adapted.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The disks_active flag is updated when masterd reads the configuration.
Also, cfgupgrade now removes the disks_active flag during downgrades.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This flag tracks if the disks of an instace are supposed to be active.
That's the case when an instance is running or when its disks got
activated explicitly (and in a couple of other cases).
It will be used by watcher to re-activate disks after a node reboot.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The field is filled with the value provided on the command line.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Now instance policies can contain more than one min/max specs.  This is the
main element of the "Constrained instance sizes" section in the
"Partitioned Ganeti" design doc.

This is a big patch, but changing the type of a configuration item requires
to change all the code that handles it.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is needed to be able to validate the std spec against multiple min/max
spec pairs (appearing in next patches).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Specs used to specify limits in instance policies are always complete, even
at group level (only the whole limit set can be overridden). This is in
preparation for introducing multiple limits.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This makes the code more modular, in preparation for the following patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This patch adds "uuid" and "name" slots to Disk and NIC ConfigObjects.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since managing of different storage units is now done using
disk templates and not storage types, we remove the obsolete
enabled storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtarara@google.com>

This patch adds constants for enabling disk templates to the constants,
and the cluster configuration in haskell and python. It removes the
obsolete preference list for enabled storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtarara@google.com>

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Minimum and maximum instance specs are put together into a single element
of the instance policy. This is in preparation for introducing multiple
min/max specs.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This implements an online update of the configuration for when a configuration
is loaded that does not yet contain the 'enabled_storage_types' attribute.
Note that this will require more changes as dis/enabling of (shared) file
storage at configure time will be removed.
This patch includes adding a mapping from disk templates to storage types. While
I was on it, I also reordered the disk template and storage type parameters
alphabetically, to make dealing with those easier.
Also, this mentions the configuration change in the NEWS file.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds the cluster's new field 'enabled_storage_types'
to the configuration objects in python and haskell.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>