There was a typo in the docstring of the parseMessage function of the
confd server.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This reverts commit b9a616e1, which
depends on the "bitvec" library. We need to investigate a bit further that
dependency, as it in turns depends on "vector" 0.9.1 or *below*, since
0.10.* removed support for Data.Vector.Unboxed.Safe which it uses.

Signed-off-by: Helga Velroyen <helgav@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Implementation of the network and address pool class in
Haskell. Not complete yet. Includes unit tests that cover
all functionality that is so far implemented.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Ganeti.Jobs now holds functions that can be used to submit and monitor the
status of jobs. In particular, execJobsWait and waitForJobs are factored
out of Hbal.hs.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Previously, functions in Hbal.hs related to execution of jobsets were
returning only IO Bool, and printing any errors they found directly to
stderr on their own.

I'm going to be moving some of these functions to a library module in
future commits, and it makes sense that they won't print to stderr, but
rather return an error condition. To make diffs more readable, I change the
return value in Hbal.hs itself, so that the next commit deals only with the
move. It's now `main` that prints any Bad result to stderr.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In Hbal.hs, it's easy to get lost in the flow of exec* functions
because their names are similar and don't convey their different
purposes (e.g. runJobSet, execJobSet, execWrapper).

This patch renames 'runJobSet' to 'execWithCancel', and 'execWrapper'
to 'execCancelWrapper', since these two functions deal, in particular,
with early termination when the user presses ^C.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

'hangleSigInt' and 'hangleSigTerm' are renamed to 'handleSigInt' and
'handleSigTerm', respectively.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This leaves Ganeti/Jobs.hs and Test/Ganeti/Jobs.hs empty, but they're the
target of a future move of some functions, so we leave them around, and
don't delete them, to avoid unnecessary delete/create diffs.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We have to check that for each edge its vertices have different colors.

This is very easy to do with a vertex-to-color map, but not so easy with
a color-to-vertex one. Since all our coloring algorithms created a
vertex-to-color map behind the scenes and then converted it, we flip
them back to returning it directly, and do the conversion explicitly
where we need it (which for now is everywhere except when testing this
property).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Our Dsatur implementation was incorrect: while the paper defined the
degree of saturation of a vertex as the number of different colors it is
adjacent to, we were using the number of colors, without considering
uniqueness. This effectively implemented a different algorithm, which is
very similar to the previous one, and while it performs slightly worse
on average it still beats Dsatur on some cases.

So we refactor the implementation to effectively support both algorithms
without code duplication, and then we export both the old algorithms as
"Dcolor" and the new one as "Dsatur". Since these are all fast
algorithms in hroller we will still be able to pick the best result.

Note that the new Dsatur implementation uses an IntSet to calculate the
uniqueness. Results with nub + length on a list were significantly
slower.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function helps treating node node problems as graph problems.
It can transform a list of nodes plus a list of instances into a graph
which uses the nodes as vertices, and instances as edges connecting them
(as long as they have both a primary and a secondary node)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Check whether coloring on a given graph makes sense. This is the case
only if there are no loops and the graph is undirected.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement the Dsatur algorithm for Graph coloring. This also abstracts
the neighColors function into two subfunctions that this algorithm can
reuse.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This module implements some algorithms on Data.Graph data structures.
At the moment its main functionality is an LF-color implementation
(greedy coloring in descending order of degree). There are also a few
extra functions to calculate the degree order, and convert the node to
color mapping to color to nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will show smaller/better error messages: full node dumps are no
longer included.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes the luxi submit job calls to use wrapped opcodes,
and therefore it changes Hbal to submit actual meta opcodes. For
nicety, hbal also submits a comment now, showing who generated the
job.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes:

  Got job IDs JobId {fromJobId = 1052613}

And restores to:

  Got job IDs 1052624

Other improvements could be done here, of course.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds the "meta" opcode type and the common op
params. Compatibility tests with Python are changed to pass Meta
opcodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was deprecated via commit 88609f00, “Switch Luxi TH code from
simple to custom fields”.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the opcode and luxi "showJSON" functions generate directly
a JSValue; in contrast, the object (single-constructor) types generate
a 'toDict' function, and then `showJSON = makeObj . toDict`. This is
useful, as the 'dict' form can be manipulated if needed.

This patch changes the opcode and luxi types to behave the same; we
generate a dict, and then (since this differs between opcodes and
luxi) generate showJSON either as `makeObj . toDict` (for opcodes), or
(for luxi) `showJSON . map sn . toDict`.

The change is needed for MetaOpCode implementation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is reused in more than just a few places, so adding it makes the
signatures much nicer.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This will go into a separate type; this patch adds the needed
underlying types and parameters.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This mirrors the positive one, and will be needed for relative job
IDs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the job ID is a simple type alias. This is suboptimal, as
it means we can't use a custom JSON (or Arbitrary) instance for it.

The patch changes it into a newtype, and then a) simplifies some
deserialisation code and b) changes some more fields to this new type
(rather than plain 'Int').

We also move the JobId to types, since it will be needed in opcodes as
well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

For now, we don't need a pending job status type as well, so we'll
delay adding that until later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 213076fe (“Fix locking in networks”) changed Python OpNetworkAdd
without corresponding Haskell definition changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Due to manual conversion, a few fields were missing from the
conversion, but as they were optional our type equivalence checking
didn't detect this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The original htools opcodes were minimalistic and not 1:1 equivalent
with the Python ones. Let's add all missing fields and, since we
changed the order, switch to more readable record syntax for building
the opcodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes THH to export a new function which defines all
fields of a given OP_ID. Not very clean, since for an invalid OP_ID we
return empty list, but since it will only be used in tests it should
be good enough.

The generated code looks as follows:

    allOpFields :: String -> [String]
    allOpFields "OP_TEST_DELAY"
                  = ["duration", "on_master", "on_nodes", "repeat"]
    allOpFields "OP_INSTANCE_REPLACE_DISKS"
                  = ["disks", "early_release", "iallocator", "ignore_ipolicy",
                     "instance_name", "mode", "remote_node"]
    …
    allOpFields _ = []

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes a bit the handling of custom fields. Since in
general we use custom fields to aggregate multiple entries in the JSON
object into a safer data-type, we should also have a way to declare
which extra entries this field covers (so that in the future we can
say what are all the JSON keys for an object).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since hlint doesn't check style issues but rather code only issues,
let's add a simple Makefile check for too-long lines, and fix the
single bad case we have currently.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This function can be applied to any type of class "Element" but its
internal variables used to refer to a list of tuples containing these
elements and their names as "nodes". Moreover it built a list of name to
indices tuples referring to it as "na" (which is not very clear).

Thus nodes becomes name_element, and na name_idx.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The serialization itself is done by Text.JSON, so the tests deal with checking
that Text.JSON objects are created correctly from the DRBD parser data
structures.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The serialized JSON is not a 1:1 dump of the data structures
populated by the parser. This is done intentionally, with the
aim of producing a more stable and more meaningful output to
be used by the (future) monitoring agent and stand-alone data
collectors.

Also:
* shorten the names of some data types.
* change data type for the fields of the Time data structure

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The new support functions in module Ganeti.JSON will be used by the next
commits to build JSON objects with optional (Maybe) fields, excluding
them when they contain Nothing.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The new one is more consistent with the rest of the nodegroup
terminology and more generic for use in different htools.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the And and Or filters use very nice code, e.g. in case of
OrFilter:

   any id <$> mapM evaluateFilter flts

However, looking at the memory profiles shows that application of
any/id to monadic values via '<$>' does not work nicely, losing the
'early' success property. This results in too much memory being used
for thunks in monadic sequencing.

Rather than trying to add more strictness (not sure exactly how, TBH),
switching these to explicit recursion solves the problem, since we
take the 'early' exit problem in our hands and we are explicit about
it.

Memory usage in case of big (e.g. 1000 'Or' elements) is reduced
significantly, and thus also the runtime.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Currently, the query operators (binop, etc.) create thunks, instead of
forcing the evaluation of the simple boolean results. This results in
higher than needed memory use.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Since this program is not built normally, a few recent commits broke
it:

- 707cd3d7 (“Use exitErr instead of explicit error message and
  exitWith”) introduced wrong syntax (missing '$')
- 29a30533 (“Remove unsafePerformIO usage”) changed the signature of
  G.Path.clusterConfFile
- 5183e8be (“Convert query path from string errors to GanetiException”)
  changed the signature of 'getNode'

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It turns out that optimising 'read' derived instances (via -O) for
complex data types (like OpCode, or the various objects) can be slow
to very slow. Disabling such instances results in (time make
$all_our_haskell_binaries) large compile-time savings and also smaller
(unstripped) binaries (by a significant amount):

ghc 6.12:        time  htools sz  hconfd sz
  with read:    4m50s 12,244,694 14,927,928
  no read:      3m30s 10,234,305 12,536,745
ghc 7.6:
  with read:   14m45s 13,694,761 15,741,755
  no read:      3m40s 11,631,373 13,245,134

So let's remove these instances, since we never use read in production
for our custom types, and even when debugging in GHCI, we can simply
use the 'show' representation to create the types, without needing to
actually parse from strings.

Note: for the very slow ghc 7.6 compilation time, I filled a ticket
(ghc #7450), since it is surprising(ly slow).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>