This changes deps and comment fields to always be shown, to match the
Python behaviour for (at least) job listing/ops field.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is not a required field, but rather an optional one; we add a new
parameter and use it instead.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Python code sometimes sends this, so let's support it even though it's
non-standard.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Having makeJobIdS as a separate function will allow us to use it
outside of json encoding.

The patch also exports one more function from the Types module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Also adds a small helper for building the paths.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This implements in the Haskell codebase the opcode summary. As opposed
to Python, we always use custom code for formatting, since we don't
want to use dynamic attribute lookup.

To test this properly, we need to change MetaOpCode to record-syntax,
so that we get accessor functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Also fixes indentation for OP_STATUS.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The extra space results in inaccessible names; currently GHC doesn't
flag this as an error, but I've filled an upstream bug for that
(http://hackage.haskell.org/trac/ghc/ticket/7484

).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Two constants which we use as Integer are defined as Int in
Constants.hs (coming from constants.py), so we do the conversion every
time we use it. Let's move them to top-level names, so that the
conversion is only done once.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a "better than nothing" support, just for kvm and just joining
the machine to the opevswitch bridge with the right command.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Also update the docstring of a function.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, we log the entire response (at debug level) in the Luxi
replies. This is not a good idea; the logging library operates on
strings, and as such it will use huge amounts of memory: without debug
logging, a certain gnt-job list invocation uses 295MB (+RTS -s) and
2m35s time, when in debug mode, it's 1525MB and 48m!

So we make two changes:

- first, we switch from "show (pp_value a)" to "encode a", which
  generates a non-formatted string rather than a indented one
- second we log only the first 2000 characters; this should be enough
  to understand the first part of the response

We could go for higher, or for logging in batches (that would be
faster, as well).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Having the Luxi client be the last argument of the functions allows for
easier use with `Exception.bracket L.getClient L.closeClient ...`.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit adds shelltests for the mon-collector binary and for the DRBD
data collector.

Also, it fixes a small bug in the DRBD parser found thanks to the tests.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It is implemented as a single binary receiving, as its first parameter,
the name of the actual data collector to be run.
This way, its structure can be used for all the future data collectors.

Also, factored out of bdev.py into constants.py the location of the DRBD
status file.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implementation of the network objects and address pool. Functionality
as in the reverted commit b9a616e1, but
now using only the vector library and not the bit-vector library.
Tested with vector library version 0.9 and 0.10., which are
also installed on the buildbot machines now.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Somehow this went missing in commit 1f1188c3.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Adds a new parameter to “OpInstanceCreate” and “OpInstanceMultiAlloc” to
use opportunistic locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

There was a typo in the docstring of the parseMessage function of the
confd server.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This reverts commit b9a616e1, which
depends on the "bitvec" library. We need to investigate a bit further that
dependency, as it in turns depends on "vector" 0.9.1 or *below*, since
0.10.* removed support for Data.Vector.Unboxed.Safe which it uses.

Signed-off-by: Helga Velroyen <helgav@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Implementation of the network and address pool class in
Haskell. Not complete yet. Includes unit tests that cover
all functionality that is so far implemented.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Ganeti.Jobs now holds functions that can be used to submit and monitor the
status of jobs. In particular, execJobsWait and waitForJobs are factored
out of Hbal.hs.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Previously, functions in Hbal.hs related to execution of jobsets were
returning only IO Bool, and printing any errors they found directly to
stderr on their own.

I'm going to be moving some of these functions to a library module in
future commits, and it makes sense that they won't print to stderr, but
rather return an error condition. To make diffs more readable, I change the
return value in Hbal.hs itself, so that the next commit deals only with the
move. It's now `main` that prints any Bad result to stderr.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In Hbal.hs, it's easy to get lost in the flow of exec* functions
because their names are similar and don't convey their different
purposes (e.g. runJobSet, execJobSet, execWrapper).

This patch renames 'runJobSet' to 'execWithCancel', and 'execWrapper'
to 'execCancelWrapper', since these two functions deal, in particular,
with early termination when the user presses ^C.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

'hangleSigInt' and 'hangleSigTerm' are renamed to 'handleSigInt' and
'handleSigTerm', respectively.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This leaves Ganeti/Jobs.hs and Test/Ganeti/Jobs.hs empty, but they're the
target of a future move of some functions, so we leave them around, and
don't delete them, to avoid unnecessary delete/create diffs.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We have to check that for each edge its vertices have different colors.

This is very easy to do with a vertex-to-color map, but not so easy with
a color-to-vertex one. Since all our coloring algorithms created a
vertex-to-color map behind the scenes and then converted it, we flip
them back to returning it directly, and do the conversion explicitly
where we need it (which for now is everywhere except when testing this
property).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Our Dsatur implementation was incorrect: while the paper defined the
degree of saturation of a vertex as the number of different colors it is
adjacent to, we were using the number of colors, without considering
uniqueness. This effectively implemented a different algorithm, which is
very similar to the previous one, and while it performs slightly worse
on average it still beats Dsatur on some cases.

So we refactor the implementation to effectively support both algorithms
without code duplication, and then we export both the old algorithms as
"Dcolor" and the new one as "Dsatur". Since these are all fast
algorithms in hroller we will still be able to pick the best result.

Note that the new Dsatur implementation uses an IntSet to calculate the
uniqueness. Results with nub + length on a list were significantly
slower.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function helps treating node node problems as graph problems.
It can transform a list of nodes plus a list of instances into a graph
which uses the nodes as vertices, and instances as edges connecting them
(as long as they have both a primary and a secondary node)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Check whether coloring on a given graph makes sense. This is the case
only if there are no loops and the graph is undirected.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement the Dsatur algorithm for Graph coloring. This also abstracts
the neighColors function into two subfunctions that this algorithm can
reuse.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This module implements some algorithms on Data.Graph data structures.
At the moment its main functionality is an LF-color implementation
(greedy coloring in descending order of degree). There are also a few
extra functions to calculate the degree order, and convert the node to
color mapping to color to nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will show smaller/better error messages: full node dumps are no
longer included.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes the luxi submit job calls to use wrapped opcodes,
and therefore it changes Hbal to submit actual meta opcodes. For
nicety, hbal also submits a comment now, showing who generated the
job.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes:

  Got job IDs JobId {fromJobId = 1052613}

And restores to:

  Got job IDs 1052624

Other improvements could be done here, of course.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds the "meta" opcode type and the common op
params. Compatibility tests with Python are changed to pass Meta
opcodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was deprecated via commit 88609f00, “Switch Luxi TH code from
simple to custom fields”.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the opcode and luxi "showJSON" functions generate directly
a JSValue; in contrast, the object (single-constructor) types generate
a 'toDict' function, and then `showJSON = makeObj . toDict`. This is
useful, as the 'dict' form can be manipulated if needed.

This patch changes the opcode and luxi types to behave the same; we
generate a dict, and then (since this differs between opcodes and
luxi) generate showJSON either as `makeObj . toDict` (for opcodes), or
(for luxi) `showJSON . map sn . toDict`.

The change is needed for MetaOpCode implementation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>