GitLab

No longer index nodes by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary:
 * Change the index key to UUID in the configuration and the
   ConfigWriter, including all methods.
 * Change all cross-references to nodes to use UUID's.
 * External interfaces (command line interface, IAllocator interface,
   hook scripts, etc.) are kept stable.
 * RPC-calls can resolve UUID's as target node arguments, if the RPC
   runner is based on a ConfigWriter instance. The result dictionary is
   presented in the form the nodes are addressed: by UUID if UUID's were
   given, or by name if names were given.
 * Node UUID's are resolved in ExpandNames and then stored in the
   OpCode. This allows to check for node renames if the OpCode is
   reloaded after a cluster restart. This check is currently only done
   for single node parameters.
 * Variable names are renamed to follow the following pattern:
   - Suffix is 'node' or 'nodes': Variable holds Node objects
   - Suffix is 'name' or 'names': Variable holds node names
   - Suffix is 'uuid' or 'uuids': Variable holds node UUID's
 * Tests are adapted.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

hspace is run on clusters with exclusive storage enabled.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add a new data collector responsible for gathering disk performance
statistics.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add one test parsing an actual /proc/diskstats file and one QuickCheck
test.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add a parser for interpreting the content of the /proc/diskstats file,
providing information about the state of the disks of the system.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Everything should work the same when exclusive storage is active, as
spindles are not yet handled as a resource. A new test has been added for
the interactions between spindles and policies.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The example cluster contains two node groups. The first contains three nodes,
with two non-redundant instances each. As each node has only capacity for three
instances, the nodes have to be rebooted individually. Additionally, there is
a separate node-group with additional of capacity, which, however, cannot be used,
as instances are supposed to be moved within a node-group only.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

hail-alloc-invalid-network defines a cluster with two nodegroups and an
allocation request which does not fit on any of the groups. Group 1 has
invalid disk-templates while Group 2 is not connected to the right
networks.

hail-alloc-restricted-network defines a cluster wih two nodegroups.
Nodegroup Group 1 is only used as last_resort group, but is chosen by
hail as only this group is connected to the right networks.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add a NIC type and extend the Instance type by a list of NIC's. Parse
the NIC's in allocation requests and store them for now. Later patches
will make use of this field in order to ensure that the requested
instance is only placed in node groups wich are connected to those
networks.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also, cleanup the list of draft designs.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The configuration is still the same as in 2.8 (the reference stable version
for this branch), so downgrade shouldn't do anything.

Unit tests are also updated, with a new 2.8 configuration file. The
configuration file used for the upgrade+downgrade test was tailored to the
2.7 downgrade, and it's not needed any more.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Ganeti is currently not able to detect a legit shutdown request performed by a
user from inside a Xen domain.

This patch provides a design document to implement a mechanism able to cope with
such events.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The example cluster consists of 6 nodes, each hosting 2 instances and
having capacity for 3. So, while the drbd-induced graph consists of
only insulated nodes, no more than two nodes can be rebooted at the
same time.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add the man page for the Ganeti auto-repair tool.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Files in the cmdlib directory are sorted alphabetically in
Makefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting instance querying related
logical units and functions to instance_query.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting instance operations
(start/stop/reboot/etc.) related logical units and functions to
instance_operation.py.

The extracted operations have in common that they affect the operating
system in a running instance directly.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting migration related logical units
and functions to instance_migration.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting storage related logical units
and functions to instance_storage.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All remaining classes in __init__.py are extracted to misc.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUOs* classes are extracted to operating_system.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUQuery* classes are extracted to query.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUBackup* classes are extracted to backup.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUInstance* classes are extracted to instance.py. Common functions
are moved to common.py if used by non-instance logical units as well.
Additionally, helper functions which are only used by LUBackup* and
LUInstance* are moved to instance_utils.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUNode* classes are extracted to node.py. Common functions are moved
to common.py if used by non-node logical units as well.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUGroup* classes are moved to group.py. Common functions are
extracted to common.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUCluster* classes are extracted to cluster.py. Shared functions are
extracted to common.py, helper functions only used by LUCluster* are
extracted to cluster.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUTest* are moved to test.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUNetwork* and associated helper functions are extracted to network.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUTags* and their base class, TagsLU, are extracted to tags.py. An
additional shared function, _ShareAll, is extracted to common.py for
shared usage.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Base classes holding common functionality is extracted into base.py.
Utility functions used by both base classes and subclasses is moved to
common.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

In a previous commit, I accidentially changed the order of
unit tests in Makefile.am to not be alphabetically anymore.
This fixes it.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This adds functionality to retrieve disk space information
for file storage. It calls the 'df' tool and parses its
output to extract the total and free amount of disk space
on the disk where the given path is located.
The code is not integrated yet, but thoroughly unit-tested.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo dal Seno <bdalseno@google.com>

Moving 'container.py' to the storage directory.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Renaming the 'block' directory to 'storage', because I plan to
place code there that is related to file storage and leaving
it named 'block' would be misleading.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Renaming 'storage.py' to 'container.py'. It will be moved into the new
'storage' directory, which will come in later patches to avoid clashes of
notation.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This commit introduces the QA for the instance status collector.

Begin the first QA for a monitoring-related component, the files and some
functions are named after monitoring because they are meant to contain
future monitoring QAs as well.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Some functions for managing instances will have to be used by new upcoming
unit tests, so they are taken out of the instances QA file and put in a new
utilities file accessible by other QA files as well.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add the Xen instance status data collector with only its core features.
The next commits will add more reporting functionalities.

The access to the collector is made possible through the mon-collector
tool.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>