The global 'cfg' variable is initialized to an empty dictionary, so there
is no more need to disable some pylint checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The disk templates used by the instances are tracked in the QA
configuration.  This will be used to selectively enable tests according to
the configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The test was broken, out of sync with the rest of the code, and prone to
crashes. Until someone does a better job, it's better to remove the
test. This simplifies further refactoring.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When the instance is removed from the cluster by this test, it's also freed
in the QA configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Exclusive storage forbids sharing PVs between unrelated LVs. This is a test
that cluster-verify correctly report such cases.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The LVM volume group becomes a configurable parameter, and a prefix for
logical volume names used in tests is introduced as a constant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Cluster-verify should report instances whose disk template is not
compatible with the exlusive storage setting.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Cluster-verify should report nodes in the same nodegroup with different
values of the exclusive_storage flag.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Check that creation of instances doesn't trigger any error.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This function will be used in following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now it's possible to check that cluster-verify reports all the expected
errors, instead of simply checking that it fails.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Specify clearly that the nic parameters depend on the
network-to-nodegroup connection.

Clarify in gnt-network how instance nics are connected to a network.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Include a handful of important changes that weren't mentioned.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Move two designs that have been implemented in 2.7 to design-2.7 and out
of drafts.

Other designs were partly implemented: explain what part was, and leave
them as drafts pending their completion.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the node listing RPC is very slow due to missing
parallelisation. For the 2.7 release, we reset these back to masterd,
hoping to revert them by the time 2.8 is ready.

There are a number of queries that I've left pointing to confd, as
they are non-RPC:

- node fields query
- node static fields query (and added a note in one case where it
  could become a problem)

All the other converted queries (job listing, group queries, tags,
cluster config) remain pointing to confd, as they don't use RPC.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Include original error message when creating TAP interface failed
- Pass keyword parameter as such

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and
is consistent with the rest of the file.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

According to doc/design-network.rst, there is a new NIC
parameter 'network', which can be used to link an instance
to a network. This was so far undocumented in the man
page of 'gnt-instance'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

'explicitly' was spelled the wrong way

Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

_GetKVMOutput expects the command to succeed, but unfortunately on some
versions of kvm "-device ?" will output a correct list of devices, while
exiting with an error code.

To fix this we accept failure in that case (note that this doesn't
change the situation when this command is not supported, as in that case
the output will still not contain the correct string).

This patch also updates the docstring, adding the parameter types and
description.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

If stopping an instance failed, the configuration would already be gone
and other operations depending on it (e.g. migration) would no longer
work. With this patch the configuration file is only removed once the
instance was succesfully stopped or destroyed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Change all “Verify” methods in hypervisor abstractions to explicitely
return None if no problem was detected. Remove punctuation from error
messages. Update docstrings with “@return” and some small mistakes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* master:
  Handle the result of QueryGroups() correctly

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit af3aea48)

If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In _ReadConfigFile, the filename should be prepared outside the
try/except block. Fixes bad code formatting, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This patch moves the functions “_ReadTestData” and “_TestDataFilename”
to module level. They do not depend on a class instance at all.

Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also
corrected.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Add a unit test for the trivial “_GetBlockDevSymlinkPath” function in
backend (small changes in the function were required).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This fixes a missing 's' in the format string and
the wrong quotes. Those bugs were introduced in
commit 6e043e60.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The current names are quite confusing; this patch cleans up the
confusion by making sure we use different terms for the two threads,
etc.

No actual code changes besides the renames.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the polling thread sleeps at the end of its body; the
original reason was that when the thread initially starts, we want to
load the config immediately (in the old-style config load model).

However, now we explicitly load the config, so we can make this thread
behave like the watcher thread: sleep first, then do the work. This
model improves for example the startup of the thread, by not redoing a
config check immediately after switching to polling mode.

We also increase the max idle poll rounds interval from 2 to 3; the
new behaviour means that we revert from polling to inotify after
around one second of no changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the Haskell logging goes with the default time formatting,
which lacks sub-second resolution. This is not good, as it's too
coarse.

The patch adds picoseconds to the log; they will contain usually too
many zeroes, but the other option is to use a log format that (for
precisely zero fractional part) omits the separator, so it's not
consistent from line to line, and which also uses a dot as a
separator, different from the Python code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In some cases we need higher resolution that seconds; I've settled on
microseconds as that is what 'threadDelay' wants, for exactly, so it's
easier if we keep the same units.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch changes the default model—where confd is always started in
polling mode—to a model where if possible we enable inotify mode
before starting any of the background threads.

There are some tricky details here: if we enable inotify, we should
not modify the server state after wards, as inotify events could have
already fired and took ownership; therefore we presume that inotify
will work and set reloadModel=ReloadNotify, and only if we fail to
enable it we change it back to polling mode.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, confd main function doesn't load the config, but simply
forks the individual timer threads, which will—as a side effect-also
load the configuration as soon as they start.

However, this makes codes to the startup more complex, so let's
explicitly (try to) load the configuration as first thing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If opportunistic_locking is used, then 'node_whitelist' parameter passed
to the allocator is set to the LU's owned node locks. However, LU owned_locks
has type of 'set' while IReqInstanceAlloc expects type of
'ht.TMaybeListOf(ht.TNonEmptyString)'.

Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>