This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).

Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we
actually have two different types at play: a full type which needs to
have all keys, and the 'partial' type which has slightly different
behaviour. I've implemented these in Haskell as two different types,
Full* and Partial*, which are derived automatically from a single
Parameter type, together with the associated Fill* functions.

Furthermore, HVParams is even more special, as its contents is not
fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
which should not be customisable at instance type (yay for
exceptions). As such, this should be written in Haskell as a
multi-constructor type, but it's the only one so far and thus we don't
have support for it yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is only useful in the stand-alone command line tools:

- hbal
- hspace
- hinfo

This change also documents the flag in all man-pages.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As for the evacuate tests, we require a mirrored disk template, but
otherwise the test should work for both mirror types.

Additionally, we perform a simplification that was left as TODO.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this function expects (but doesn't really check! only via
secondary node being defined) a DRBD8 template type.

This patch (not very readable, sorry) changes it so that we check
whether we're internally mirrored, externally mirrored, or not
mirrored at all. It then proceeds to do the right thing™ per the
mirror type, so that it can work well with both.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, these presume that the instance being generated always is
DRBD/relocatable (but won't be in the future), and also presume that
all relocation modes can be used (hint: not ☺️

).

So we add a couple of helper functions to restrict the used disk
templates/relocation modes, and these two tests are ready for shared
storage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In case you just run "make htools/test" to recompile the unittests,
the old .tix file should be removed, otherwise you get the message
about inconsistent tick boxes. Let's make the compilation process
automatically remove this, for easier testing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of hard-coding the instance move operation, let's accept is as
a parameter from the caller. This is the single thing that makes this
DRBD-specific, so removing it will help for future
patches. Furthermore, this abstract the imove op type into a single
place instead of two (the inner function and the nodeEvacInstance).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of randomly generating the number of required nodes or
hard-coding it, compute it from the disk template of the
instance. Otherwise tests will fail when the disk template is not
DRBD8 and we ask for allocation on 2 nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Another test that only works for DRBD8 instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… since memory reservation only is used for this template.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this (single) test checks whether we can add an instance as
primary and secondary to a given node. This won't work for instance
templates other than DRBD8, so we split it into:

- an add to primary test, which should work for all disk templates
- an add to secondary test, for which we set the disk template to DRBD
  explicitly

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the function possibleMoves always presumes DtDrbd, which is
wrong for shared storage. To make it work with all disk templates, we
additionally pass the mirror type to it (we assume that the move
behaviour only depends on the mirror type, and not the individual
disk template; currently this is true, and we should aim to keep it
that way).

For none and external mirrors, we don't give any solutions; for
internal mirrors, we keep the previous algorithm.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… otherwise we can't run tests or simulations using non-default
templates. Tests still pass.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we don't set a boundary on the length of the score field,
which breaks all our nicely-aligned output efforts.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently Cluster.printSolutionLine function always wants to print the
old and new secondary, and as such it fails (ugly) for moves of
non-DRBD instances. Change it so that it looks explicitly for
noSecondary nodes, and format the output different in that case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we track the mirroring type as simply yes/no, with "yes"
meaning DRBD. This is not enough for handling shared storage, so we
need to add a type that gives the exact mirroring type (none,
internal, external), and a function that ties the disk template type
to the mirroring type.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of computing the bad/good node lists separately (in effect
iterating twice over the all_nodes list and also creating that list
twice from the container), let's use 'partition' and do a small
improvement.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the code in Loader.hs marks as un-movable any instances
which don't have a secondary. This is not valid for any shared storage
templates, so let's remove that check completely. The initial state of
the instance will reflect correctly the movable status, based on the
disk template of the instance, so we don't need to re-do this check.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Our current cluster model is based on per-node storage, and as such we
don't handle shared disk templates. Until we can improve the model,
let's handle nodes without local storage better: in order to still
compute a valid cluster score, we need to handle nodes with tDsk == 0.

For this, we abstract the in-line computation of pDsk (which is used
in the cluster score) to a separate function which, for tDsk == 0,
returns instead 1 (i.e. disk fully used).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)
have fixed http://bugs.python.org/issue12856 which we worked around
ourselves.

This means two things:

- we don't need to manually reset the module
- we can't test for the no-reset case

Unfortunately current Debian Sid has the 2.7.3 RC but still labeled as
2.7.2+ ☹️

, so we have to manually do an extra check (in the unit test
only); I expect Debian will update to official 2.7.3 as soon as it's
released, and then we can remove this override.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Conflicts:
        Makefile.am    (trivial, test data files added on both branches)
        lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5
                        fixed the type of the parameter)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This moves the dashes check to a new check-man-dashes script.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently the check-man test is dependent on the MAN_HAS_WARNINGS
test, which made sense as long as the script only did the warnings
check.

However, we should run the [em] tests always, and in general we could
run other tests too, even if man doesn't support --warnings; so let's
split the check man. As a first prerequisite, we rename check-man to
check-man-warnings (and the respective Makefile variables).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

I've investigated more and it seems that .DELETE_ON_ERROR not working
is a bug/regression in some versions of make; testing with make 3.82
from Debian experimental makes this special target work as
expected. Also, make 3.81 works on trivial Makefiles, but not on our
complex one.

In the meantime, let's improve the fix for the building of man pages:
make the removal implicit, instead of explicit, via the use of 'trap'
on EXIT. I just hope this behaves consistently across all bash
versions :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This make it possible to load from an Ialloc file

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This makes it possible to reuse that function when we want to use IAlloc
as a data source backend

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Because .DELETE_ON_ERROR doesn't work consistently (works when the
target we're talking about is a final target, but not for
intermediates), I missed a lot of man checks.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This helper abstract away all the unlines and unwords call you've to do
after you got a formatTable back.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This fixes the prop_Node_rMem test, which cares about memory only;
otherwise we could fail due to other problems (e.g. not enough disk).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is not perfect, as the override applies identical target test
values to both the 'fast' and 'slow' tests (making the slow tests take
a long time), but it allows a quick override for manual runs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

These just test that add/remove are idempotent. This is not perfect,
as we use unsorted lists for some values (instead of sets), so when
using non-empty nodes this would break (but for empty nodes, a
1-element list is sorted, so it's fine).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This makes balancing use the new metric.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>