All functions/classes which are used outside of their defining module
(with tests as an exception) no longer have a leading underscore.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting storage related logical units
and functions to instance_storage.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUQuery* classes are extracted to query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUInstance* classes are extracted to instance.py. Common functions
are moved to common.py if used by non-instance logical units as well.
Additionally, helper functions which are only used by LUBackup* and
LUInstance* are moved to instance_utils.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUNode* classes are extracted to node.py. Common functions are moved
to common.py if used by non-node logical units as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUGroup* classes are moved to group.py. Common functions are
extracted to common.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUCluster* classes are extracted to cluster.py. Shared functions are
extracted to common.py, helper functions only used by LUCluster* are
extracted to cluster.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This patch introduces a test to check the compatibility
of the Haskell and the Python representation of instances.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch introduces and enhances generators for
instances:
- 'genInstWithNets' is split into the generation of an
  arbitrary instance and enhancing an instance with nets
- 'genInst' calls 'genInstWithNets' with an empty set
  of initial networks to provide a reasonable default
- the Arbitrary instance of 'Instance' uses now 'genDisks'
  to create instances with a reasonable set of disks

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The Arbitrary instance of the 'Instance' object is written
using the <*> syntax. Since it often uses the 'arbitrary'
generator for the instance's fields it is hard to figure
out which 'arbitrary' fills which instance field. This
patch annotates all fields with their name to make
maintenance of this code easier.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds generators for Disk instances to the Haskell
test code. It uses somewhat more reasonable generators to
fill the fields instead of just arbitrary values.
'genDiskWithChildren' is a generator that generates a disk
with a specified number of disk children. To avoid shooting
ourselves in the foot we do not generate further (grand)
child disks for the child disks. 'genDisk' calls
'genDiskWithChildren' by requesting three children as a
resonable default.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This simplifies my previous commit (820bade9) by using os.statvfs
instead of parsing the output of 'df'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This adds functionality to retrieve disk space information
for file storage. It calls the 'df' tool and parses its
output to extract the total and free amount of disk space
on the disk where the given path is located.
The code is not integrated yet, but thoroughly unit-tested.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo dal Seno <bdalseno@google.com>

Moving 'container.py' to the storage directory.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Renaming the 'block' directory to 'storage', because I plan to
place code there that is related to file storage and leaving
it named 'block' would be misleading.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Renaming 'storage.py' to 'container.py'. It will be moved into the new
'storage' directory, which will come in later patches to avoid clashes of
notation.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

While the multiple-tags test was added to verify that coloring is done
only after node selection (otherwise it wouldn't be possible to get in
both cases a single reboot group), it can easily be extended to also
verify that the correct nodes are selected by --node-tags.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In the example configuration, the graph constructed by just connecting
primary and secondary instances is two-colorable. However, when taking
conflicting locations of secondary nodes into account, three reboot
groups are needed. Moreover, these reboot groups are not subordinated
to any two-coloring of the first-mentioned graph.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Regular expressions are not shell globs. So "any symbol" is expressed
by a dot, not a question mark. In this case, the confusion lead to a
too liberal expectation, hence the test passed. Fix it nevertheless.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The example is a cluster of 6 nodes, paired into 3 group by three
instances. So the whole cluster would need two reboot groups. The two
tags select, in two different ways, one node of each group. So, when
restricting to one tag, a single reboot group suffices, but no
coloring of the whole cluster would achieve this.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In hroller, the option -O can be used to mark certain nodes as offline.
These nodes should then not be part of any reboot group. Add tests
to verify this behavior.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

All functionality specific to a single DRBD8 devide is now in DRBD8Dev,
whereas functionality which is valid for the whole DRBD "installation"
on a device is collected in DRBD8.

This makes it possible to remove a couple of FIXME's and clarifies the
code in some areas.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Right now the DRBD8 class has multiple responsibilities: a) it
reprensents a device which can be set up, grown, etc. and b) it
represents the whole DRBD system on a node which has a usermode helper,
which knows how to shut down _all_ devices, etc. Therefore, the DRBD8Dev
class will only be responsible for a device, whereas the soon to be
extraced DRBD8 class is responsible for the DRBD system.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

gnt-cluster verify should issue a warning if there are multiple DRBD
versions present in a node group. In order to do so, the DRBD version
has to be queryable from noded.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The way disk templates were displayed in instance policies became confusing
since the introduction of the enabled_disk_templates cluster parameter.

Fix issue 440.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This makes sure that issue 401 doesn't come back.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The haskell type definition of opcodes should remain aligned with the python
one.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since we use the primitive string type for group UUIDs, the group
fields have a bug where we pass the group name as filter for node
tests, whereas the nodes themselves use the group UUID. This results
in zero node count, empty node list, and no instances being reported
as assigned to groups.

The patch fixes this and adds a test for the node count. It does some
test generation improvement, and also cleans up whitespace issues in
Test/G/Q/Query.hs (the functions case_queryNode_allfields,
prop_queryGroup_noUnknown and case_queryGroup_allfields are unchanged
but simply have indentation fixed).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Cherry-pick of e7124835, fixes issue 436
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Conflicts:
	test/hs/Test/Ganeti/Objects.hs
	test/hs/Test/Ganeti/Query/Query.hs

The DRBD84CmdGenerator class, which generates commands suited for DRBD
8.4, is added. A common baseclass for DRBD83CmdGenerator and
DRBD84CmdGenerator is introduced as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Based on the version of DRBD (as reported through /proc/drbd), the DRBD8
class chooses helper objects which adapt for the differences between
DRBD <= 8.3 and DRBD 8.4. This new test verifies that the proper objects
are created.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As the DRBD code was extracted from bdev.py, the corresponding tests
should as well be extracted into their own file. That's what this patch
performs.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In order to easily support the DRBD 8.4 command syntax, this patch
extracts all command generation logic (i.e. where calls to `drbdsetup`
are assembled) into a new class DRBD83CmdGenerator.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Common functionality between the DRBD 8.3 and DRBD 8.4 parser has been
extracted into BaseShowInfo. A test which verifies the behaviour is
included, but the DRBD84ShowInfo class is not yet used within the DRBD8
class.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Parsing the status (/proc/drbd) and `drbdsetup show` output takes quite
a bit of code, which has to be augmented by even more with the DRBD 8.4
support. So extract all the related classes into their own file.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This class parses the `drbdsetup show` output and represents it in an
easily accessible format. It got extracted so that 1) the DRBD8 class
can focus more on DRBD logic rather than parsing and 2) it's easier to
adapt the parser to new formats.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The current parser already supports the output of DRBD 8.4 /proc/drbd,
so add tests which actually verify this behaviour.
Note that the empty lines in proc_drbd84.txt are actually present in the
output of DRBD 8.4, they always appear when minors are not contiguous.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The previous patch removed a (rather implicit) test for /proc/drbd not
containing a version line. This adds an explicit test for this case.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As the DRBD8 class got bigger due to the previous merge of BaseDRBD, now
parts of it are ripped out into DRBD8Info. This new class parses
/proc/drbd and exposes the information in an easily accessible way. This
allowed to simplify some methods in DRBD8 and do make the tests more
concise.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

BaseDRBD was probably useful when DRBD 0.7 and DRBD 8 were supported.
However, there is only one subclass of BaseDRBD remaining (DRBD8), and
the separation of responsibilities between those two classes was rather
randomly chosen.

The unification into one class also eases the introduction of
DRBD 8.4 support, as the responsibilities of the class(es) will be
distributed anew.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

That's what the method actually does, so there is no reason why we
should use a funny but unreadable name.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>