The new configuration object for nodes will not support it anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is a temporary solution until nodes and instances are converted to
objects with attributes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The same code can be written in a single expression. The earlier patch
named “Refactor storage of runtime exclusive storage flag in QA”
provided the unit test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is a follow-up for “qa_config: Remove exclusive storage flag from
config”. Instead of storing the flag in a module-level variable it is
now stored within the new QA configuration class and unit tests are
provided. Wrappers in “qa_config” maintain the existing interface.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Ever since its introduction (sometime before commit cec9845c in September
2007), the QA configuration was stored in a dictionary at module-level
in “qa/qa_config.py”. The configuration was loaded, verified and
evaluated using module-level functions. Since then the configuration has
become more complicated and more functionality has been added. This
patch refactors handling the configuration to use a class and provides
unittests.

- The configuration is loaded through a class method which also verifies
  it for consistency
- Wrapper methods are provided in “qa_config” to not change the
  interface
- Unit tests are provided for the new configuration class
- The configuration object is still stored in a module-level variable
  and can be retrieved using “GetConfig” (direct access should be
  avoided so an uninitialized configuration can be detected)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Commit 6a0f22e1 added the ability to track the status of the cluster-wide
flag for exclusive storage. It did so in a way which modifies the
in-memory configuration dictionary. This patch changes the code to use a
plain global variable as a temporary solution until a configuration
object is added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This enabled the redirection; QA output after this patch should become
much much smaller (on a 'quick' QA, we are left with ~0.5MB output on
stdout and 64MB redirected output).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Some QA commands generate non-trivial output: for a full QA, the gnt-*
list commands generate more than 300MB of data. To make the QA logs
more parseable by humans, we need support for redirecting output of
commands to a different place.

This patch adds a 'AssertRedirectedCommand' that does this,
redirecting the output to a (fixed) log file in the Ganeti log
directory.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds an optional kwarg that allows skipping the "Command:
…" line in QA output. This is helpful if we only need to run some
maintenance actions, that are QA-related and not Ganeti-related.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This test used to leave a node with orphan logical volumes and unused DRBD
minors. Now, the expected leftovers are cleaned up after the test, and
cluster-verify is run.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It is only used in one place and not necessary.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instance tests are run with or without the exclusive-storage flag set. More
configurations can be added easily.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We want to make sure that instance operations don't leave the cluster in a
bad state. The "instance-remove-drbd-offline" test leaves some debris
behind, so it's been moved to the last position.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The code launching instance tests for "plain" and "drbd" templates has been
unified. In this way it's easier to add more templates, and to re-use the same
tests with different configurations.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Some instance test functions took two node arguments, some took one, and
some took two but the second argument could be None. This patch makes such
functions uniform by using a list of nodes as an argument.  This simplifies
the following refactoring.

No test logic has been changed.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instance test functions check the current configuration, and they will run
the actual tests only if the configuration supports them.  This will be
used for refactoring in following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These make possible to simplify the logic of tests requiring more
nodes. Used in the following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

QA configuration now tracks the state of the exclusive_storage flag at
cluster level. This will be used to selectively enable tests according to
the configuration.

Also, it's now possible to specify the initial value of the
exclusive_storage flag in the configuration file.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The global 'cfg' variable is initialized to an empty dictionary, so there
is no more need to disable some pylint checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The disk templates used by the instances are tracked in the QA
configuration.  This will be used to selectively enable tests according to
the configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The test was broken, out of sync with the rest of the code, and prone to
crashes. Until someone does a better job, it's better to remove the
test. This simplifies further refactoring.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When the instance is removed from the cluster by this test, it's also freed
in the QA configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Exclusive storage forbids sharing PVs between unrelated LVs. This is a test
that cluster-verify correctly report such cases.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The LVM volume group becomes a configurable parameter, and a prefix for
logical volume names used in tests is introduced as a constant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Cluster-verify should report instances whose disk template is not
compatible with the exlusive storage setting.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Cluster-verify should report nodes in the same nodegroup with different
values of the exclusive_storage flag.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Check that creation of instances doesn't trigger any error.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This function will be used in following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now it's possible to check that cluster-verify reports all the expected
errors, instead of simply checking that it fails.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds a script for the QA of 'gnt-network'. So far it
covers adding/removing and connecting/disconnecting networks.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The additional comma makes this file invalid JSON.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Reported in issue 341. In some setups the instance live in a different
netblock from the cluster. Therefore a the configuration-global “rename”
name shouldn't be used for them. Instead another instance name is used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also add a check to make sure there are enough groups defined.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Requested in issue 337.

The parameter “bridge” was not documented and is therefore silently
replaced with “master-netdev”. A note is added to “qa-sample.json”
describing how comments work.

This is a cherry-pick of master commit
3601d488

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit ec0652ad (June 2009) removed the option.

This is a cherry-pick of master commit
78453739

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Requested in issue 337.

The parameter “bridge” was not documented and is therefore silently
replaced with “master-netdev”. A note is added to “qa-sample.json”
describing how comments work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit ec0652ad (June 2009) removed the option.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The previous patch series made it simpler to deal with ERROR_up offline
instances, but didn't fix qa, that had the old expectations. This fixes
that, and adds more testing about the expected behavior.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is not a trivial s/frozenset/compat.UniqueFrozenset/, but rather
only replaces “frozenset” where appropriate. Most of the places are
“static” information that doesn't change after the module has been
loaded.

Some docstrings and code formatting (e.g. empty lines) issues are
addressed as well. Some lines got too long and were wrapped.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch removes code which is no longer used due to refactoring:

- http.InitSsl, last usage removed in commit 33231500 (“Convert RPC
  client to PycURL”)
- rapi.baserlib.MakeParamsDict, last usage remove in commit 4e5a68f8
  (“RAPI: rlib1 removal”)
- rapi.baserlib.ExtractField, last usage removed in commit 028c6b76
  (“RAPI:Fix root list and unittest for it.”)
- qa.qa_utils.AssertNotEqual, last real usage removed in commit
  2f4b4f78 (“Simplify QA commands”) (but it was still imported for a
  while, until we extended pylint to run over the QA code as well)
- qlang._IsHostname, added in commit f8638e28 (“Detect globbing
  patterns as query arguments”) but never used
- cmdlib._BuildNetworkHookEnvByObject, last usage removed in commit
  f22433c0 (“Locking related fixes for networks”)
- NLD constants, which are obsolete—the nbma project is no longer
  maintained and it's not working with current ganeti

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>