_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch moves the functions “_ReadTestData” and “_TestDataFilename”
to module level. They do not depend on a class instance at all.

Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also
corrected.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Add a unit test for the trivial “_GetBlockDevSymlinkPath” function in
backend (small changes in the function were required).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Add a unittest for a function formatting CPU pinning information for
Xen's configuration.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds two functions in Utils.hs: 'wrap' and 'trim' along with some
simple tests. 'wrap' is used in Common.hs for wrapping command
descriptions to a fixed line length, e.g. useful for the "--help"
option.

Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

utils.LvmExclusiveCheckNodePvs() now returns the size of the smallest and
biggest PVs of a node. They will be used for cross-node consistency checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When new a new disk template is added, burnin should be updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since now niceSort is more consistently behaved, we don't have the
special casing. This should have been removed in commit a7f0953a, but
I didn't realise it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This fixes a bug that corrups the config when one adds a
network to a node group. I wil soon extend QA by some commands
that would have detected this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what
would be the next-in-line policy to use (and would have to re-parse all
tags again).

To avoid this, we pass the current time to 'mergeData' from ExtLoader.hs,
and we propagate it to Loader.getArPolicy. ExtLoader.loadExternalData is in
the IO monad, so it has ready access to getClockTime.

A few other places were calling 'mergeData' directly. For Hscan.hs and
IAlloc.hs, we add appropriate calls to getClockTime. For files under test/,
we use a current time of 0.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some auto-repair tags are composed of a prefix and then an optional
argument; the new "chompPrefix" function in Utils.hs allows to strip a
prefix, allowing the last character (the separator) to be absent if there
is no argument.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows to easily convert a ClockTime object to a string
representation of its timestamp (seconds-only).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In this commit, the AutoRepairType and AutoRepairResult types are defined,
with the possible values specified in doc/design-autorepair.rst.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

One is the --help output for kvm 1.1.2 and the second one is the same as
0.9.1 but with a fake option added between -drive and its boot=on|off
option: this tests that if boot=on|off appears in another option after
-drive it is not accepted by our regexp.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

As discussed on the list some people backport features to different kvm
versions, making the version based detection system unreliable. We
change wherever we can to detection on the help output.

This also fixes Issue 250

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will be used in QA to format network interface parameters.

This is a cherry-pick of master commit eac9b7b8

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also amended a comment of a tested method.
The tests are for methods introduced in commit 63c73073 (LVM disk creation
uses dedicated PVs).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This commits enables the DRBD data collector to use the Confd client to
gather information about the pairing between DRBD minors and instances.

For testing purposes, the DRBD data collector now requires either zero
or two parameters: one is the DRBD file, one is the pairings file.
When no parameter is passed, the collector is in "production mode" and takes
the data from /proc/drbd and from the Confd client.

The shell tests of mon-collector are updated accordingly.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This commit modifies the DRBD parser and its data structures to include
information about the instance a DRBD minor belongs to.

Test files have been updated as well, to support the new field in the data
structures.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was extracted from a patch by Dimitris Aragiorgis with the subject
“Add locking to _NetworkQuery”. The rest of the patch did no longer
apply and will require more work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in
the future).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch unifies the functions that generated the tree
reorganisation discussion in the first place :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The current base name of 'test' means that this binary will generate
profiling, coverage, etc. files with the name 'test.*', result in
tab-completion conflicts with the test directory. Let's slightly
change its base name for typing easiness.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

htest/data becomes test/data/htools (basically reverting commit
8feabc89), and htest/* becomes test/hs/*.

Most changes beside the rename are trivial s/…/…, with the exception
of autotools/run-in-tempdir, which needed some more changes now that
test/ is not just Python files:

- test/py is still being copied
- test/hs moves from individual symlinks to entire dir symlink
- test/data is symlinked in its entirety

Checked with make distcheck, pep8 and pylint, so at least VPATH builds
are OK.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is the first step of the test files reorganisation: moving test/*
(except test/data) to new directory test/py/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.

For now, only two commands are supported: 'info' and 'diagnose'.

'diagnose' calculates the node status of each provider on each node,
similarly to gnt-os diagnose. Furthermore, for every provider, it
calculates it's nodegroup validity for each nodegroup. This is done
inside the LU and not the client (marked as 'TODO' for the  global
validity of gnt-os diagnose).

In the future, gnt-storage can be used to manage storage pools,
or even be extended to diagnose other storage types supported by
Ganeti, such as lvm, drbd (INT_MIRROR) or rbd (EXT_MIRROR).

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed Haskell compatibility and style fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

For people writing query filters (documented in ganeti(7)) knowing a field's
type can be useful.

$ gnt-instance list-fields name be/memory
Name      Type         Title        Description
name      Text         Instance     Instance name
be/memory Storage size ConfigMaxMem The "maxmem" backend parameter

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will be used in QA to format network interface parameters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If opportunistic locking is requested in the opcode, but no iallocator
is used (not specified or no default), an error will be raised.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is not a trivial s/frozenset/compat.UniqueFrozenset/, but rather
only replaces “frozenset” where appropriate. Most of the places are
“static” information that doesn't change after the module has been
loaded.

Some docstrings and code formatting (e.g. empty lines) issues are
addressed as well. Some lines got too long and were wrapped.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Unit tests updated and expanded with an inheritance check.

The flag has no effect yet.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 141d1489 was a bit too enthusiastic. The three parameters added to
the list of parameters to be checked default to a value not evaluating
to false, leading to a failure on cluster initialization.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

“_SPICE_ADDITIONAL_PARAMS” is supposed to be the full list of
SPICE-related KVM hypervisor parameters with the exception of
“HV_KVM_SPICE_BIND”. The new test checks if all parameters starting with
“HV_KVM_SPICE_*” are included. Three previously missing parameters are
added to the list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

For some opcodes, the output is not "stable", and depends on the exact
input values; this makes it harder to check consistency against
Haskell code.

To compensate for this, we add a way to override the formatting of the
OP_DSC_FIELD; by default, this is always "%s", but if the
OP_DSC_FORMATTER is defined (must be a callable), it is used to format
the actual value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The watcher pause file should be set/unset on all nodes at once, not
only the master node. For that a new RPC is needed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

“TestWriteFile” was not renamed when adding the file based on
“ganeti.utils.io_unittest-runasroot.py”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This addresses issue 218. When the number of inotify watches is
exhausted, for example by being set too low from the beginning or by
other programs, waiting for a job to change would just report a lost job
(e.g. “Error checking job status: Job with id 7817 lost”).

This patch changes the job watcher to no longer catch
“errors.InotifyError” and, this is by far the larger part of this patch,
adds unittests for this situation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Add more checks, some of them are deliberately redundant
- Descriptive error messages
- Add comment describing order to “tools.ensure_dirs”
- Avoid copying a list in an assertion in “tools.ensure_dirs”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>