We most always use `templateMirrorType . diskTemplate`, so let's add a
helper function for this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

hail now expects correctly that relocate_from is of equal length with
the number of required nodes (fixme: there's a lot of not well
documented behaviour here… not nice for any other potential
IAllocators). As such, we _need_ to pass just the instance's primary
node.

Additionally, update the iallocator doc to correctly specify what this
list (`relocate_from`) contains.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

After the many patches implementing the shared storage functionality
in iallocator, balances, etc., and after the patches preparing the
unit-tests to do the right thing for shared storage, we can finally
flip the bits that make instances have "random" disk templates as
opposed to just DRBD. Yes, the unit-tests still pass ☺️



We also need to enable all disk templates in the IPolicy (for
allocation/relocation).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch toggles the final bit and enables nodeEvacInstance to work
on shared storage instances. Diskless instances are handled the same
(as in, None is the same everywhere :).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of manually case-ing on the Either contents, let's just use
either with const functions (not really readable, but…).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The function is now generic, so let's rename it and cleanup/improve
the docstrings.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is generic enough (no longer depending on the replace type) that
it will be used for other replace modes too. So all we need to do is
to abstract it to a separate function (that can be called from
multiple branches) and to make the imove op_fn computation more
generic, instead of hard-coding it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this test assumes localStorageTemplates, so let's make sure
we enforce that on the given instance

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This brings together all previous pieces and allows balancing to work
(with the known caveats related to disk space calculation) for shared
storage disk templates.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also add the spindle usage to QC

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In the same transition also do a simple refactor of the caculation

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The old text files remain compatible with this change.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now we just pass in "1" as spindle usage and adapt the backend
later.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This adapts the Ganeti side to export the spindle_usage

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Due to how conversions were implemented previously, 1TB failed to
parse on 32-bit, as we were overflowing during computation, even
though the final result would fit easily.

This patch moves the parsing of the scaling factor to a separate
function, and all the conversions are done via the Rational type
(which has unlimited arbitrary precision), and conversion to the
desired type only happens at the last step.

The unit-tests are adjusted too, unfortunately they use the same
algorithm as the code… suggestions on how to improve things are
welcome.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Extend the kernel version check to also accept Linux 3.x as valid.

Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With the new shell tests, we have for the first time the situation
where tests require a binary built (via make), but the binary itself
is not the test. Hence, we need to declare and use check_SCRIPTS for
it.

Also corrects a wrong file name (introduced during rebase/editing the
patch series).

The end result is that `make distcheck` now passes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These test that we can actually rebalance (and that it results in
replace-disks and failovers/migrations), and that we don't
double-rebalance, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This adds new offline tests:

- checks that files generated by hspace can be read by hbal/hinfo
- checks that hbal handles node groups as expected
- checks that enabling multiple backends results in failure
- checks hspace machine readable code and higher verbosity
- enables a "FAIL" message on error exit

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This allows the offline tests to also generate coverage data, and the
hs-coverage target is changed to show both unit-test and offline-tests
coverage.

The downside is that now we build yet-another-binary, which makes the
unit-test time slower…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This enables the offline-test.sh run both in the special "hs-check"
rule and during normal "make check".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This only tests the command line basic options. In the future, it can
be expanded to check other things.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

… and update live-test.sh to use it, instead of hard-coding the
program paths.

A few additional style fixes are also done (indentation).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is currently not installed anywhere, just built.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This contains a more-or-less complete implementation for the
server-side confd.

Note that most of the code is behaving identical to the Python code,
with a notable exception: the asyncore/inotify code was changed (since
Haskell doesn't have asyncore) to 3-thread system. While otherwise I
tried to keep the code flow identical, I found the original Python
code not easily to reason about, and as such I changed the
architecture in this regard.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is not complete for now, just the basic functionality has been
implemented:

- daemonize
- check we're running under the correct user
- call setup logging

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on hslogger 1.1+, which is not available in
Debian Squeeze, only in later versions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This duplicates the lib/runtime.py functionality, allowing us to check
for runtime users and groups consistency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on the following new libraries:

- Data.HMAC module from the 'crypto' library
- Data.Text from the 'text' library
- Data.ByteString from the 'bytestring' library

In Debian, the bytestring library comes with the ghc compiler, whereas
the other two are available separately in wheezy/unstable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

While we have some of these as plain types in Constants.hs, we add
proper ADT definitions for them in a new file. Furthermore, we add the
ConfdRequest and ConfdReply types here (in Python they are in
objects.py).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This only is able to load the configuration data for now.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).

Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we
actually have two different types at play: a full type which needs to
have all keys, and the 'partial' type which has slightly different
behaviour. I've implemented these in Haskell as two different types,
Full* and Partial*, which are derived automatically from a single
Parameter type, together with the associated Fill* functions.

Furthermore, HVParams is even more special, as its contents is not
fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
which should not be customisable at instance type (yay for
exceptions). As such, this should be written in Haskell as a
multi-constructor type, but it's the only one so far and thus we don't
have support for it yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is only useful in the stand-alone command line tools:

- hbal
- hspace
- hinfo

This change also documents the flag in all man-pages.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As for the evacuate tests, we require a mirrored disk template, but
otherwise the test should work for both mirror types.

Additionally, we perform a simplification that was left as TODO.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this function expects (but doesn't really check! only via
secondary node being defined) a DRBD8 template type.

This patch (not very readable, sorry) changes it so that we check
whether we're internally mirrored, externally mirrored, or not
mirrored at all. It then proceeds to do the right thing™ per the
mirror type, so that it can work well with both.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, these presume that the instance being generated always is
DRBD/relocatable (but won't be in the future), and also presume that
all relocation modes can be used (hint: not ☺️

).

So we add a couple of helper functions to restrict the used disk
templates/relocation modes, and these two tests are ready for shared
storage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>