If cluster is initialized with --no-lvm-storage then volume_group_name
does not exist in config.data. Thus we must define it as optional in
confd.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
[iustin@google.com: fixed Haskell RPC definition]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

… no comment :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds the option to hbal, and uses it to tweak the submitted
jobs. There are also two small shelltests for testing the parsing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This just defined the new priority, with the same name as the Python one.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The current serialisation format for submit priorities is
integer-based, same as the opcode json serialisation. But for CLI
level, we need to support a string-based format, so we add functions
to parse and format this representation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This simply updates the metaopcode submit priority.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, hbal code always uses annotateOpCode function, which means
we would have to pass the options data to all function in the call
chain if we wanted to make this more flexible.

By abstracting the type of the annotator and passing it as an argument
to the function, we can be more flexible.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes the check for conflicts from the Haskell
version of the OpCode NetworkDisconnect. This alignes
the Haskell code with the patch
"Force conflicts check in LUNetworkDisconnect" (which
is currently under review). I will submit these patches
together then.

This is a cherry-pick of commit 0ae4b355.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This info is not used by Ganeti and therefore is removed.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This slot was not used by Ganeti so the same info can be
provided via tags. In order not to break configuration data
we add a FromDict() method in Network config object that
removes the deprecated network_type (if found) and then invoke
the parent's method (which does the validation).

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was applied to "master" along with extra changes affecting the
master branch only. Cherry-picking just the Objects.hs change.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since 'head' is unsafe to use in most cases, this patch removes its
use from most of the code, adds a lint warning for it (and for tail as
well), and adds override annotations in the few cases where it's
actually OK to use it (mainly when using head over the result of
functions which guarantee to return a non-empty list by documentation,
not type).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Commit 3329f4de changed the Haskell log file from constants to
functions, but introduced a bug: it uses now the daemon name instead
of the correct log file, which means "ganeti-confd.log" instead of
"conf-daemon.log".

In order to fix this, we need to abstract the log file base (in
constants.py) into a separate set of constants, so that we can reuse
it in the Haskell code.

This fixes issue 343.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and
is consistent with the rest of the file.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The current names are quite confusing; this patch cleans up the
confusion by making sure we use different terms for the two threads,
etc.

No actual code changes besides the renames.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the polling thread sleeps at the end of its body; the
original reason was that when the thread initially starts, we want to
load the config immediately (in the old-style config load model).

However, now we explicitly load the config, so we can make this thread
behave like the watcher thread: sleep first, then do the work. This
model improves for example the startup of the thread, by not redoing a
config check immediately after switching to polling mode.

We also increase the max idle poll rounds interval from 2 to 3; the
new behaviour means that we revert from polling to inotify after
around one second of no changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the Haskell logging goes with the default time formatting,
which lacks sub-second resolution. This is not good, as it's too
coarse.

The patch adds picoseconds to the log; they will contain usually too
many zeroes, but the other option is to use a log format that (for
precisely zero fractional part) omits the separator, so it's not
consistent from line to line, and which also uses a dot as a
separator, different from the Python code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In some cases we need higher resolution that seconds; I've settled on
microseconds as that is what 'threadDelay' wants, for exactly, so it's
easier if we keep the same units.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch changes the default model—where confd is always started in
polling mode—to a model where if possible we enable inotify mode
before starting any of the background threads.

There are some tricky details here: if we enable inotify, we should
not modify the server state after wards, as inotify events could have
already fired and took ownership; therefore we presume that inotify
will work and set reloadModel=ReloadNotify, and only if we fail to
enable it we change it back to polling mode.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, confd main function doesn't load the config, but simply
forks the individual timer threads, which will—as a side effect-also
load the configuration as soon as they start.

However, this makes codes to the startup more complex, so let's
explicitly (try to) load the configuration as first thing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds two functions in Utils.hs: 'wrap' and 'trim' along with some
simple tests. 'wrap' is used in Common.hs for wrapping command
descriptions to a fixed line length, e.g. useful for the "--help"
option.

Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this commit, the DRBD data collector does not only print the data
it extracts, but includes them in the proper JSON structure common to
all the data collectors, as prescribed by the design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit adds the part of the JSON report generation code that will be
common to all the data collectors, according to the format specified in the
design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The getCurrentTime function is of wide interest, so its moved to a more
appropriate position.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, niceSort behaves differently than niceSortKey in cases
where the parsed groups are equal but the original not, as it uses
both members of the tuple as opposed to just the first one. Both
niceSortKey and the Python code use only the first one, so in corner
cases we can have different behaviour of these functions.

To rectify this, we change niceSort to be a simple wrapper over
"niceSortKey id", so that we get consistent behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes a bug that corrups the config when one adds a
network to a node group. I wil soon extend QA by some commands
that would have detected this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what
would be the next-in-line policy to use (and would have to re-parse all
tags again).

To avoid this, we pass the current time to 'mergeData' from ExtLoader.hs,
and we propagate it to Loader.getArPolicy. ExtLoader.loadExternalData is in
the IO monad, so it has ready access to getClockTime.

A few other places were calling 'mergeData' directly. For Hscan.hs and
IAlloc.hs, we add appropriate calls to getClockTime. For files under test/,
we use a current time of 0.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

'getArPolicy' and 'setArPolicy' follow the precedence rules introduced in
b1eb71c7: within an object, the most restrictive tag wins; across object,
the nearest tag wins.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some auto-repair tags are composed of a prefix and then an optional
argument; the new "chompPrefix" function in Utils.hs allows to strip a
prefix, allowing the last character (the separator) to be absent if there
is no argument.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows to easily convert a ClockTime object to a string
representation of its timestamp (seconds-only).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

AutoRepairPolicy, AutoRepairStatus, and other auxiliary types are added.
These are used only internally by the auto-repair tool, and parsed from the
various object tags as defined in the design doc.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In this commit, the AutoRepairType and AutoRepairResult types are defined,
with the possible values specified in doc/design-autorepair.rst.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A missing letter has been added to a docstring.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This commits enables the DRBD data collector to use the Confd client to
gather information about the pairing between DRBD minors and instances.

For testing purposes, the DRBD data collector now requires either zero
or two parameters: one is the DRBD file, one is the pairings file.
When no parameter is passed, the collector is in "production mode" and takes
the data from /proc/drbd and from the Confd client.

The shell tests of mon-collector are updated accordingly.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit modifies the DRBD parser and its data structures to include
information about the instance a DRBD minor belongs to.

Test files have been updated as well, to support the new field in the data
structures.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Completion information for multi-command binaries were not generated in
the correct way, so the build-bash-complation script failed when such a
binary was actually found.

This commit fixes this issue, and a related docstring typo as well.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>