This is the beginning of the implementation of network
queries. This includes establishing all infrastructure
to run the network queries and implement querying of
some simpler fields and the node group listing.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When displaying query results of type Maybe, one could use the
function rsMaybe. Unfortunately, this function maps 'Nothing'
values to RSNoData which gets displayed as '?' in the list of
query results. These semantics do not fit if the result is of
a Maybe type, because it is an optional field, because in this
case 'Nothing' is not an error (like for example the consequence
of an RPC error), but a legitimate result and as such should
be displayed as '-' in the result list. For this purpose,
this patch introduces another version of rsMaybe, which is
called rsMaybeUnavail to be used for optional fields.
For consistency, rsMaybe gets renamed to rsMaybeNoData.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For network queries to work, we need to extend the general
config type to include all available networks. Additionally,
we add UUIDs to the network type itself.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The current names are quite confusing; this patch cleans up the
confusion by making sure we use different terms for the two threads,
etc.

No actual code changes besides the renames.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the polling thread sleeps at the end of its body; the
original reason was that when the thread initially starts, we want to
load the config immediately (in the old-style config load model).

However, now we explicitly load the config, so we can make this thread
behave like the watcher thread: sleep first, then do the work. This
model improves for example the startup of the thread, by not redoing a
config check immediately after switching to polling mode.

We also increase the max idle poll rounds interval from 2 to 3; the
new behaviour means that we revert from polling to inotify after
around one second of no changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the Haskell logging goes with the default time formatting,
which lacks sub-second resolution. This is not good, as it's too
coarse.

The patch adds picoseconds to the log; they will contain usually too
many zeroes, but the other option is to use a log format that (for
precisely zero fractional part) omits the separator, so it's not
consistent from line to line, and which also uses a dot as a
separator, different from the Python code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In some cases we need higher resolution that seconds; I've settled on
microseconds as that is what 'threadDelay' wants, for exactly, so it's
easier if we keep the same units.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch changes the default model—where confd is always started in
polling mode—to a model where if possible we enable inotify mode
before starting any of the background threads.

There are some tricky details here: if we enable inotify, we should
not modify the server state after wards, as inotify events could have
already fired and took ownership; therefore we presume that inotify
will work and set reloadModel=ReloadNotify, and only if we fail to
enable it we change it back to polling mode.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, confd main function doesn't load the config, but simply
forks the individual timer threads, which will—as a side effect-also
load the configuration as soon as they start.

However, this makes codes to the startup more complex, so let's
explicitly (try to) load the configuration as first thing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds two functions in Utils.hs: 'wrap' and 'trim' along with some
simple tests. 'wrap' is used in Common.hs for wrapping command
descriptions to a fixed line length, e.g. useful for the "--help"
option.

Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this commit, the DRBD data collector does not only print the data
it extracts, but includes them in the proper JSON structure common to
all the data collectors, as prescribed by the design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit adds the part of the JSON report generation code that will be
common to all the data collectors, according to the format specified in the
design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The getCurrentTime function is of wide interest, so its moved to a more
appropriate position.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, niceSort behaves differently than niceSortKey in cases
where the parsed groups are equal but the original not, as it uses
both members of the tuple as opposed to just the first one. Both
niceSortKey and the Python code use only the first one, so in corner
cases we can have different behaviour of these functions.

To rectify this, we change niceSort to be a simple wrapper over
"niceSortKey id", so that we get consistent behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes a bug that corrups the config when one adds a
network to a node group. I wil soon extend QA by some commands
that would have detected this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what
would be the next-in-line policy to use (and would have to re-parse all
tags again).

To avoid this, we pass the current time to 'mergeData' from ExtLoader.hs,
and we propagate it to Loader.getArPolicy. ExtLoader.loadExternalData is in
the IO monad, so it has ready access to getClockTime.

A few other places were calling 'mergeData' directly. For Hscan.hs and
IAlloc.hs, we add appropriate calls to getClockTime. For files under test/,
we use a current time of 0.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

'getArPolicy' and 'setArPolicy' follow the precedence rules introduced in
b1eb71c7: within an object, the most restrictive tag wins; across object,
the nearest tag wins.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some auto-repair tags are composed of a prefix and then an optional
argument; the new "chompPrefix" function in Utils.hs allows to strip a
prefix, allowing the last character (the separator) to be absent if there
is no argument.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows to easily convert a ClockTime object to a string
representation of its timestamp (seconds-only).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

AutoRepairPolicy, AutoRepairStatus, and other auxiliary types are added.
These are used only internally by the auto-repair tool, and parsed from the
various object tags as defined in the design doc.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In this commit, the AutoRepairType and AutoRepairResult types are defined,
with the possible values specified in doc/design-autorepair.rst.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A missing letter has been added to a docstring.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This commits enables the DRBD data collector to use the Confd client to
gather information about the pairing between DRBD minors and instances.

For testing purposes, the DRBD data collector now requires either zero
or two parameters: one is the DRBD file, one is the pairings file.
When no parameter is passed, the collector is in "production mode" and takes
the data from /proc/drbd and from the Confd client.

The shell tests of mon-collector are updated accordingly.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit modifies the DRBD parser and its data structures to include
information about the instance a DRBD minor belongs to.

Test files have been updated as well, to support the new field in the data
structures.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Completion information for multi-command binaries were not generated in
the correct way, so the build-bash-complation script failed when such a
binary was actually found.

This commit fixes this issue, and a related docstring typo as well.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was extracted from a patch by Dimitris Aragiorgis with the subject
“Add locking to _NetworkQuery”. The rest of the patch did no longer
apply and will require more work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If a received reply is valid and there is still no valid reply,
keep it, no matter how new it is.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

While trying to understand why some code was not being tested, I
realised that we have a bad data type in Hcheck.hs.

We have "data Level = GroupLvl | ClusterLvl", but then we need to pass
the group name/index as well, so we have functions that look like the
following:

  f :: Level -> Maybe String -> …
  f GroupLvl (Just gname) = …
  f GroupLvl Nothing = … -- never used
  f ClusterLvl _ = … -- wrong, since a (Just x) could be passed

It's clear that we just need to move the group name in the "GroupLvl"
constructor, and simplify the call chain.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in
the future).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The modules are moved from the ‘top’ namespace to ‘Ganeti.HTools’, in
compliance with standard practices.

This patch moves the version string creation into a function in Utils
which shows some more information.

This patch adds a -V, --version command line option that shows the
program version and also updates the hn1 usage string (similar to hbal).

This just reorders some functions for a more logical ordering.

This patch changes the solution list to include a step counter so that
it's more clear these are successive steps (in a definite order), and
not just an unordered list of changes.

This patch changes the gnt-instance failover to migrate, and fixes a bug
in the formatting of commands.