Due to how conversions were implemented previously, 1TB failed to
parse on 32-bit, as we were overflowing during computation, even
though the final result would fit easily.

This patch moves the parsing of the scaling factor to a separate
function, and all the conversions are done via the Rational type
(which has unlimited arbitrary precision), and conversion to the
desired type only happens at the last step.

The unit-tests are adjusted too, unfortunately they use the same
algorithm as the code… suggestions on how to improve things are
welcome.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Extend the kernel version check to also accept Linux 3.x as valid.

Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With the new shell tests, we have for the first time the situation
where tests require a binary built (via make), but the binary itself
is not the test. Hence, we need to declare and use check_SCRIPTS for
it.

Also corrects a wrong file name (introduced during rebase/editing the
patch series).

The end result is that `make distcheck` now passes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These test that we can actually rebalance (and that it results in
replace-disks and failovers/migrations), and that we don't
double-rebalance, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This adds new offline tests:

- checks that files generated by hspace can be read by hbal/hinfo
- checks that hbal handles node groups as expected
- checks that enabling multiple backends results in failure
- checks hspace machine readable code and higher verbosity
- enables a "FAIL" message on error exit

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This allows the offline tests to also generate coverage data, and the
hs-coverage target is changed to show both unit-test and offline-tests
coverage.

The downside is that now we build yet-another-binary, which makes the
unit-test time slower…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This enables the offline-test.sh run both in the special "hs-check"
rule and during normal "make check".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This only tests the command line basic options. In the future, it can
be expanded to check other things.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

… and update live-test.sh to use it, instead of hard-coding the
program paths.

A few additional style fixes are also done (indentation).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is currently not installed anywhere, just built.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This contains a more-or-less complete implementation for the
server-side confd.

Note that most of the code is behaving identical to the Python code,
with a notable exception: the asyncore/inotify code was changed (since
Haskell doesn't have asyncore) to 3-thread system. While otherwise I
tried to keep the code flow identical, I found the original Python
code not easily to reason about, and as such I changed the
architecture in this regard.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is not complete for now, just the basic functionality has been
implemented:

- daemonize
- check we're running under the correct user
- call setup logging

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on hslogger 1.1+, which is not available in
Debian Squeeze, only in later versions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This duplicates the lib/runtime.py functionality, allowing us to check
for runtime users and groups consistency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on the following new libraries:

- Data.HMAC module from the 'crypto' library
- Data.Text from the 'text' library
- Data.ByteString from the 'bytestring' library

In Debian, the bytestring library comes with the ghc compiler, whereas
the other two are available separately in wheezy/unstable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

While we have some of these as plain types in Constants.hs, we add
proper ADT definitions for them in a new file. Furthermore, we add the
ConfdRequest and ConfdReply types here (in Python they are in
objects.py).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This only is able to load the configuration data for now.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).

Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we
actually have two different types at play: a full type which needs to
have all keys, and the 'partial' type which has slightly different
behaviour. I've implemented these in Haskell as two different types,
Full* and Partial*, which are derived automatically from a single
Parameter type, together with the associated Fill* functions.

Furthermore, HVParams is even more special, as its contents is not
fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
which should not be customisable at instance type (yay for
exceptions). As such, this should be written in Haskell as a
multi-constructor type, but it's the only one so far and thus we don't
have support for it yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is only useful in the stand-alone command line tools:

- hbal
- hspace
- hinfo

This change also documents the flag in all man-pages.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As for the evacuate tests, we require a mirrored disk template, but
otherwise the test should work for both mirror types.

Additionally, we perform a simplification that was left as TODO.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this function expects (but doesn't really check! only via
secondary node being defined) a DRBD8 template type.

This patch (not very readable, sorry) changes it so that we check
whether we're internally mirrored, externally mirrored, or not
mirrored at all. It then proceeds to do the right thing™ per the
mirror type, so that it can work well with both.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, these presume that the instance being generated always is
DRBD/relocatable (but won't be in the future), and also presume that
all relocation modes can be used (hint: not ☺️

).

So we add a couple of helper functions to restrict the used disk
templates/relocation modes, and these two tests are ready for shared
storage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In case you just run "make htools/test" to recompile the unittests,
the old .tix file should be removed, otherwise you get the message
about inconsistent tick boxes. Let's make the compilation process
automatically remove this, for easier testing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of hard-coding the instance move operation, let's accept is as
a parameter from the caller. This is the single thing that makes this
DRBD-specific, so removing it will help for future
patches. Furthermore, this abstract the imove op type into a single
place instead of two (the inner function and the nodeEvacInstance).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of randomly generating the number of required nodes or
hard-coding it, compute it from the disk template of the
instance. Otherwise tests will fail when the disk template is not
DRBD8 and we ask for allocation on 2 nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Another test that only works for DRBD8 instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… since memory reservation only is used for this template.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this (single) test checks whether we can add an instance as
primary and secondary to a given node. This won't work for instance
templates other than DRBD8, so we split it into:

- an add to primary test, which should work for all disk templates
- an add to secondary test, for which we set the disk template to DRBD
  explicitly

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the function possibleMoves always presumes DtDrbd, which is
wrong for shared storage. To make it work with all disk templates, we
additionally pass the mirror type to it (we assume that the move
behaviour only depends on the mirror type, and not the individual
disk template; currently this is true, and we should aim to keep it
that way).

For none and external mirrors, we don't give any solutions; for
internal mirrors, we keep the previous algorithm.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… otherwise we can't run tests or simulations using non-default
templates. Tests still pass.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we don't set a boundary on the length of the score field,
which breaks all our nicely-aligned output efforts.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently Cluster.printSolutionLine function always wants to print the
old and new secondary, and as such it fails (ugly) for moves of
non-DRBD instances. Change it so that it looks explicitly for
noSecondary nodes, and format the output different in that case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we track the mirroring type as simply yes/no, with "yes"
meaning DRBD. This is not enough for handling shared storage, so we
need to add a type that gives the exact mirroring type (none,
internal, external), and a function that ties the disk template type
to the mirroring type.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of computing the bad/good node lists separately (in effect
iterating twice over the all_nodes list and also creating that list
twice from the container), let's use 'partition' and do a small
improvement.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the code in Loader.hs marks as un-movable any instances
which don't have a secondary. This is not valid for any shared storage
templates, so let's remove that check completely. The initial state of
the instance will reflect correctly the movable status, based on the
disk template of the instance, so we don't need to re-do this check.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Our current cluster model is based on per-node storage, and as such we
don't handle shared disk templates. Until we can improve the model,
let's handle nodes without local storage better: in order to still
compute a valid cluster score, we need to handle nodes with tDsk == 0.

For this, we abstract the in-line computation of pDsk (which is used
in the cluster score) to a separate function which, for tDsk == 0,
returns instead 1 (i.e. disk fully used).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)
have fixed http://bugs.python.org/issue12856 which we worked around
ourselves.

This means two things:

- we don't need to manually reset the module
- we can't test for the no-reset case

Unfortunately current Debian Sid has the 2.7.3 RC but still labeled as
2.7.2+ ☹️

, so we have to manually do an extra check (in the unit test
only); I expect Debian will update to official 2.7.3 as soon as it's
released, and then we can remove this override.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>