GitLab

Commit 30d25dd8

 moved the htools code to new-style exception handling,
but the hconfd code hasn't been, which fails when compiling on newer
GHC versions.

This patch does the rest of the move; however, the situation is not
very nice, we should have a better way to handle this (introduce
catchIO? or something like that).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, we're using Prelude.catch to handle I/O errors in
htools. This style of error handling has been deprecated for a while,
but it still used to work without warnings.

However, the GHC release 7.4 has started to emit deprecation warnings
for it, so we change to the Control.Exception module; the code is a
bit less clean since we only care about I/O errors (but
Control.Exception deals with other error types too), so we have to
filter the exceptions.

Note that the new style exception handling is not really "new"; it has
existed since at least GHC 6.12, which is our oldest supported
compiler.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is the last warning related to TemplateHaskell that was 6.12
specific; for some reason, it doesn't "see" that traw/tname were used.

The patch just replaces the quoting syntax with an explicit type
declaration; while less readable, it also has less warnings.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

GHC 7.4 has updated the TemplateHaskell library, and it turns out that
the way we built the JSON instance implementation for showJSON was not
good (probably this is why GHC 6.12 was generating some warnings).

The patch changes the build of showJSON to be the same as readJSON,
which was working fine. As a bonus, this fixes both the 7.4 issue and
the 6.12 one.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

While updating the confd code, I realised that we have _lots_ of
duplication in the exit model for the various programs.

So this patch attempts to abstract all the exits via a couple of new
functions; sorry for the somewhat big patch, but I hope the payoff is
worth the change: the actual exit conditions are much clearer.

Note that the patch (also) moves the exitIfBad function to Utils.hs,
since that is more logical.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… and enable it in hconfd.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently this is initialised to no from Daemon.hs, but will in the
future allow command-line options for controlling it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This patch makes hail treat '-' as denoting stdin, per the usual Unix
convention. This will help with testing.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Instead of hardcoded IPv4 INADDR_ANY, this patch changes hconfd to use
either the any network for the configured cluster address family
(ipv4/ipv6), or whatever the user passes in via the --bind option.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This implements the same logic as the Python code: if the option is
not used, use the default appropriate for the cluster, otherwise try
to parse and use whatever was passed in.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These both are work with/on the Result type, so we add them to
BasicTypes. The functions will be used as more generic versions of
some more specialised functions that are right now spread across the
modules.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This currently has only one export function in it, which will be used
for future bind address functionality in daemons.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The dashes do not need to be in, if they are then the resulting option
is:

  -p PORT  ----port=PORT     Network port (default: 1814)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This will allow offline testing of this backend (except for the actual
curl part), also in the case where we didn't compile against curl.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We most always use `templateMirrorType . diskTemplate`, so let's add a
helper function for this.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

After the many patches implementing the shared storage functionality
in iallocator, balances, etc., and after the patches preparing the
unit-tests to do the right thing for shared storage, we can finally
flip the bits that make instances have "random" disk templates as
opposed to just DRBD. Yes, the unit-tests still pass ☺



We also need to enable all disk templates in the IPolicy (for
allocation/relocation).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch toggles the final bit and enables nodeEvacInstance to work
on shared storage instances. Diskless instances are handled the same
(as in, None is the same everywhere :).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of manually case-ing on the Either contents, let's just use
either with const functions (not really readable, but…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The function is now generic, so let's rename it and cleanup/improve
the docstrings.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is generic enough (no longer depending on the replace type) that
it will be used for other replace modes too. So all we need to do is
to abstract it to a separate function (that can be called from
multiple branches) and to make the imove op_fn computation more
generic, instead of hard-coding it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this test assumes localStorageTemplates, so let's make sure
we enforce that on the given instance
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This brings together all previous pieces and allows balancing to work
(with the known caveats related to disk space calculation) for shared
storage disk templates.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also add the spindle usage to QC
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In the same transition also do a simple refactor of the caculation
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The old text files remain compatible with this change.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now we just pass in "1" as spindle usage and adapt the backend
later.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Due to how conversions were implemented previously, 1TB failed to
parse on 32-bit, as we were overflowing during computation, even
though the final result would fit easily.

This patch moves the parsing of the scaling factor to a separate
function, and all the conversions are done via the Rational type
(which has unlimited arbitrary precision), and conversion to the
desired type only happens at the last step.

The unit-tests are adjusted too, unfortunately they use the same
algorithm as the code… suggestions on how to improve things are
welcome.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This contains a more-or-less complete implementation for the
server-side confd.

Note that most of the code is behaving identical to the Python code,
with a notable exception: the asyncore/inotify code was changed (since
Haskell doesn't have asyncore) to 3-thread system. While otherwise I
tried to keep the code flow identical, I found the original Python
code not easily to reason about, and as such I changed the
architecture in this regard.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is not complete for now, just the basic functionality has been
implemented:

- daemonize
- check we're running under the correct user
- call setup logging
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on hslogger 1.1+, which is not available in
Debian Squeeze, only in later versions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This duplicates the lib/runtime.py functionality, allowing us to check
for runtime users and groups consistency.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds a dependency on the following new libraries:

- Data.HMAC module from the 'crypto' library
- Data.Text from the 'text' library
- Data.ByteString from the 'bytestring' library

In Debian, the bytestring library comes with the ghc compiler, whereas
the other two are available separately in wheezy/unstable.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

While we have some of these as plain types in Constants.hs, we add
proper ADT definitions for them in a new file. Furthermore, we add the
ConfdRequest and ConfdReply types here (in Python they are in
objects.py).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This only is able to load the configuration data for now.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).

Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we
actually have two different types at play: a full type which needs to
have all keys, and the 'partial' type which has slightly different
behaviour. I've implemented these in Haskell as two different types,
Full* and Partial*, which are derived automatically from a single
Parameter type, together with the associated Fill* functions.

Furthermore, HVParams is even more special, as its contents is not
fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
which should not be customisable at instance type (yay for
exceptions). As such, this should be written in Haskell as a
multi-constructor type, but it's the only one so far and thus we don't
have support for it yet.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is only useful in the stand-alone command line tools:

- hbal
- hspace
- hinfo

This change also documents the flag in all man-pages.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>