Add support for the memory resource controller, useful to actually place
memory limits on instances.

Support is still optional, in case the kernel doesn't have it compiled
in, or -in the case of Debian- has it dependent on a kernel command-line
switch (cgroup_enable=memory specifically).

Signed-off-by: Faidon Liambotis <paravoid@debian.org>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently hv_lxc depends on the behavior and output of older LXC tools,
which have since changed, making it unable to function in current
distributions (e.g. Debian wheezy).

Adapt the tools and expectation for the output and make it into a
working state again, at least for LXC 0.8.0.

LXC tools seem to be split into two categories: the actual, binary lxc
tools, plus a set of shell scripts to provide a higher-level interface.
Ganeti doesn't need the latter and the shell scripts make some
assumptions (like /var/lib/lxc) so remove the only reference to such a
shell script (lxc-ls).

While at it, remove a few comments about "current (Spring 2010)
distributions" that do not apply anymore.

This is a backwards-incompatible change, meaning it won't work on
systems with older LXC userspace. However, hv_lxc is experimental anyway
and there's virtually no documentation about it in the tree.

Signed-off-by: Faidon Liambotis <paravoid@debian.org>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add a missing "in".

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instance tests are run with or without the exclusive-storage flag set. More
configurations can be added easily.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We want to make sure that instance operations don't leave the cluster in a
bad state. The "instance-remove-drbd-offline" test leaves some debris
behind, so it's been moved to the last position.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The code launching instance tests for "plain" and "drbd" templates has been
unified. In this way it's easier to add more templates, and to re-use the same
tests with different configurations.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Some instance test functions took two node arguments, some took one, and
some took two but the second argument could be None. This patch makes such
functions uniform by using a list of nodes as an argument.  This simplifies
the following refactoring.

No test logic has been changed.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instance test functions check the current configuration, and they will run
the actual tests only if the configuration supports them.  This will be
used for refactoring in following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These make possible to simplify the logic of tests requiring more
nodes. Used in the following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

QA configuration now tracks the state of the exclusive_storage flag at
cluster level. This will be used to selectively enable tests according to
the configuration.

Also, it's now possible to specify the initial value of the
exclusive_storage flag in the configuration file.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The global 'cfg' variable is initialized to an empty dictionary, so there
is no more need to disable some pylint checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The disk templates used by the instances are tracked in the QA
configuration.  This will be used to selectively enable tests according to
the configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The test was broken, out of sync with the rest of the code, and prone to
crashes. Until someone does a better job, it's better to remove the
test. This simplifies further refactoring.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When the instance is removed from the cluster by this test, it's also freed
in the QA configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Exclusive storage forbids sharing PVs between unrelated LVs. This is a test
that cluster-verify correctly report such cases.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The LVM volume group becomes a configurable parameter, and a prefix for
logical volume names used in tests is introduced as a constant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Cluster-verify should report instances whose disk template is not
compatible with the exlusive storage setting.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Cluster-verify should report nodes in the same nodegroup with different
values of the exclusive_storage flag.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Check that creation of instances doesn't trigger any error.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This function will be used in following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now it's possible to check that cluster-verify reports all the expected
errors, instead of simply checking that it fails.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Specify clearly that the nic parameters depend on the
network-to-nodegroup connection.

Clarify in gnt-network how instance nics are connected to a network.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Include a handful of important changes that weren't mentioned.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Move two designs that have been implemented in 2.7 to design-2.7 and out
of drafts.

Other designs were partly implemented: explain what part was, and leave
them as drafts pending their completion.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the node listing RPC is very slow due to missing
parallelisation. For the 2.7 release, we reset these back to masterd,
hoping to revert them by the time 2.8 is ready.

There are a number of queries that I've left pointing to confd, as
they are non-RPC:

- node fields query
- node static fields query (and added a note in one case where it
  could become a problem)

All the other converted queries (job listing, group queries, tags,
cluster config) remain pointing to confd, as they don't use RPC.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Include original error message when creating TAP interface failed
- Pass keyword parameter as such

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and
is consistent with the rest of the file.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

According to doc/design-network.rst, there is a new NIC
parameter 'network', which can be used to link an instance
to a network. This was so far undocumented in the man
page of 'gnt-instance'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

'explicitly' was spelled the wrong way

Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

_GetKVMOutput expects the command to succeed, but unfortunately on some
versions of kvm "-device ?" will output a correct list of devices, while
exiting with an error code.

To fix this we accept failure in that case (note that this doesn't
change the situation when this command is not supported, as in that case
the output will still not contain the correct string).

This patch also updates the docstring, adding the parameter types and
description.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

If stopping an instance failed, the configuration would already be gone
and other operations depending on it (e.g. migration) would no longer
work. With this patch the configuration file is only removed once the
instance was succesfully stopped or destroyed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Change all “Verify” methods in hypervisor abstractions to explicitely
return None if no problem was detected. Remove punctuation from error
messages. Update docstrings with “@return” and some small mistakes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* master:
  Handle the result of QueryGroups() correctly

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit af3aea48)

If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In _ReadConfigFile, the filename should be prepared outside the
try/except block. Fixes bad code formatting, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This patch moves the functions “_ReadTestData” and “_TestDataFilename”
to module level. They do not depend on a class instance at all.

Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also
corrected.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Add a unit test for the trivial “_GetBlockDevSymlinkPath” function in
backend (small changes in the function were required).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>