add-node now performs a few checks on LVM before adding a node to the
cluster.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will be used in node-add.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When exclusive_storage is set, cluster-verify complains if the size of PVs
is not uniform across a node group.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

utils.LvmExclusiveCheckNodePvs() now returns the size of the smallest and
biggest PVs of a node. They will be used for cross-node consistency checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The value of the flag should be the same for nodes within the same group.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- compat.UniqueFrozenset used in one more place, for consistency.
- The order of NV_XXX constants is fixed so that all values are now sorted
by name.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

By creating symlinks instead of actually creating files we still get the
benefits of using a temporary directory, but the preparation is about
three times as fast. Test used:

$ time bash -c 'for ((i=0; i < 100; ++i)); do
    autotools/run-in-tempdir true; done'

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

To preserve disk space on copying all files to a temporary directory,
the “doc” directory is only copied when building documentation. A
comment describing the need is also added to Makefile.am, as well as a
superfluous direct invocation of Bash removed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

With this commit, the DRBD data collector does not only print the data
it extracts, but includes them in the proper JSON structure common to
all the data collectors, as prescribed by the design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit adds the part of the JSON report generation code that will be
common to all the data collectors, according to the format specified in the
design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The getCurrentTime function is of wide interest, so its moved to a more
appropriate position.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When new a new disk template is added, burnin should be updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This fixes Issue 225.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently kvm --help is called twice at instance startup, this makes
sure it's called only once.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will be used to be able to customize the path in an hv parameter.

Also fix two comments that still mentioned version-based checking.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is needed in case an instance must be booted with an exact type of
machine version (due to e.g. outdated drivers). If this hv param
explicitly defined (during instance creation/modification) then the kvm
process will be started having this version for the -M option. Otherwise
the default (derived from _GetDefaultMachineVersion() will be used.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Similar to other scripts, “tools/burnin” just becomes a wrapper script.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since now niceSort is more consistently behaved, we don't have the
special casing. This should have been removed in commit a7f0953a, but
I didn't realise it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As reported by Constantinos Venetsanopoulos, DT_RBD and DT_EXT work on
single node clusters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch depends on “Option to include man pages in documentation”. In
the documentation build including man pages, all “:manpage:`…`”
references are converted to links. For man pages not provided by Ganeti,
Sphinx' standard formatting is used.

A small dance is necessary to hook into Sphinx' processing of man page
roles and to generate automatically resolved links. The code converts
“:manpage:`…`” for known man pages to the data structure equivalent of
“:doc:`$name($section) <man-$name>`”. Additionally it checks the section
numbers and formatting of references (in all builds).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, niceSort behaves differently than niceSortKey in cases
where the parsed groups are equal but the original not, as it uses
both members of the tuple as opposed to just the first one. Both
niceSortKey and the Python code use only the first one, so in corner
cases we can have different behaviour of these functions.

To rectify this, we change niceSort to be a simple wrapper over
"niceSortKey id", so that we get consistent behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Sphinx 1.0 and above can check version by setting “needs_sphinx” in the
configuration.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Before this patch, HTML versions of man pages (man/*.rst) were already
built. However, since they are separate from the normal documentation,
their content is not indexed for Sphinx' search functionality.
Additionally it would simply be nice to have everything in one place.

To this end a new configure-time option is added to enable the inclusion
of man pages into the documentation. A dedicated option is necessary to
still be able to provide a static documentation build in the tarball
(not including man pages) as man pages contain build-specific paths and
values. The documentation with man pages is written to the directory
“doc/man-html”.

A future patch will extend Sphinx to link occurences of “:manpage:`…`”
to these man pages.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commas are not valid characters in tags, hence they can't be use to
separate the different job IDs; plus signs (+) are available, and not too
bad.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Burnin doesn't support all disk templates (specifically it doesn't
support “blockdev”). The code, however, used manually composed lists in
some places, constants.DISK_TEMPLATES in others, and an internal list of
supported templates in some.

This patch changes burnin to only use its internal list of supported
disk templates.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Keep the lists at the top instead of having them embedded in the code.
Use frozenset, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes a bug that corrups the config when one adds a
network to a node group. I wil soon extend QA by some commands
that would have detected this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what
would be the next-in-line policy to use (and would have to re-parse all
tags again).

To avoid this, we pass the current time to 'mergeData' from ExtLoader.hs,
and we propagate it to Loader.getArPolicy. ExtLoader.loadExternalData is in
the IO monad, so it has ready access to getClockTime.

A few other places were calling 'mergeData' directly. For Hscan.hs and
IAlloc.hs, we add appropriate calls to getClockTime. For files under test/,
we use a current time of 0.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

'getArPolicy' and 'setArPolicy' follow the precedence rules introduced in
b1eb71c7: within an object, the most restrictive tag wins; across object,
the nearest tag wins.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit clarifies one particular point of the auto-repair workflow:
what to do when multiple, conflicting administrator-set tags exist in an
object; and how tags at different levels (cluster, node group and instance)
interact.

For conflict within an object, we choose to always let the most restrictive
tag win (i.e. the least destructive repair, and the longest suspension
time). For tags at different levels, we follow a simple "nearest tag wins"
rule.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some auto-repair tags are composed of a prefix and then an optional
argument; the new "chompPrefix" function in Utils.hs allows to strip a
prefix, allowing the last character (the separator) to be absent if there
is no argument.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows to easily convert a ClockTime object to a string
representation of its timestamp (seconds-only).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The graph added here follows the data types introduced in 7ec2f76b, by which
"Suspended" is not a state, but a possible value of the auto-repair policy.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

AutoRepairPolicy, AutoRepairStatus, and other auxiliary types are added.
These are used only internally by the auto-repair tool, and parsed from the
various object tags as defined in the design doc.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In this commit, the AutoRepairType and AutoRepairResult types are defined,
with the possible values specified in doc/design-autorepair.rst.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit changes the "ganeti:watcher:repair:pending" and
"ganeti:watcher:repair:result" tag prefixes to use
"ganeti:watcher:autorepair:{pending,result}", so that the same prefix is
used in all cases.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>