GitLab

To compensate for the cases where a QA test is supposed to block when
a lock is present, add an additional option showing whether blocking is
supposed to happen or not.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

(cherry picked from commit 57efdaf5)

Conflicts:
	qa/qa_rapi.py (didn't backport changes to this file)

This patch adds threading to the RunWithTests function, allowing one
thread to execute the QA test, and the other to monitor if it is being
blocked by locks set up during the test. If it is, terminate the
blocking job, and let the QA continue, reporting the test failure at
the very end.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

(cherry picked from commit 34c5ec6c)

This patch adds a QA utility function that acquires a set of locks, and
attempts to run a given function with the locks in place. Should the
given function block, this function does not detect this - later
patches will address the issue.

An example of its use is provided by having the move-instance test
modified to use it.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

(cherry picked from commit 343c9ed2)

Conflicts:
	qa/qa_rapi.py (didn't backport changes to this file)

The list-drbd QA test assumed falsely that no instance has more than
one disk. This can happen, and this patch accounts for it.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When disk templates were ignored in the QA, no messages were emitted,
in contrast to pretty much all tests. To make it easier to see why
a test was not executed, this patch performs a minor refactoring and
adds some output for the templates.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds a tiny helper function that allows error messages
output by the QA asserts to be prefixed with whatever the user
supplies.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The UUID/name switch avoided this particular bit of code, and as a
result the list-drbd command failed as it tried to compare UUIDs and
names. This patch fixes the retrieval, converts the newly returned
UUIDs to names, and modifies the QA to the results and not only the
invocation are checked.
Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This patch invokes the RAPI symmetry tests for groups.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch allows users to specify specific parameter aliases enabled
by RAPI, and test whether their value can be retrieved and set.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch adds the RAPI symmetry test for nodes.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The original design of the RAPI symmetry test assumed that all RAPI
calls that modify parameters use the PUT method. This is not true in
the case of the node parameter modification method, and while that
itself should be addressed or at least examined, the test method has
to change to accomodate both PUT and POST requests.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch adds the test for RAPI symmetry at the cluster level. There
are some fields that have not been exposed yet, and their absence was
noted.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The RAPI should allow all the parameters of objects to be gotten and
set under the same names. This patch adds a test that checks if this is
the case by using the underlying opcode arguments.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch does some housecleaning and sorts imports.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The previous patch loading utilities omitted empty patches, as they
were thought to be of no significance, and when no patches were used,
the import and therefore dependency should not be used. If a user has
added an empty patch file, and made an entry in the order file, the QA
would treat this as an error as it had no means of differentiating
between a patch not present and an empty patch.

This patch fixes the solution by better handling empty patches, and
logging warnings when they are encountered.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The formatting functions in qa_utils.py cannot be used by modules
imported there, such as qa_config. This patch factors the function
calls into a separate file. Also reorders imports in touched files.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To explicitly specify the order of patches executed, the QA provides an
"order" file. It can contain names of patches that will be executed
first, and in the order listed, before all the other patches that still
follow an alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch allows support for multiple patches placed in the "patch"
directory, which are executed in alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* Refactors the current patch code to allow for multiple patches that
  can be applied, yet leaves only one for now.

* Rewords the message shown to the user in case the modules needed for
  patching are missing.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In order to work, it is necessary:
* Write a JSON Patch to file "qa/qa-patch.json" in a source repository.
  (And be careful not to commit it!) The file must conform to RFC6902
  and describes what changes should be done to the QA JSON
  configuration. This is simple as
   [
     { "op": "add",
       "path": "default",
       "value": false },
     { "op": "add",
       "path": "/tests/instance-add-file",
       "value": true }
   ]

* Install "jsonpatch" and "jsonpointer" modules on the machines QA is
  running on. Either using "easy_install" or as Debian packages using
  "pypi-install".

QA only loads the modules if the patch file is present and non-empty.
Therefore no changes to buildbots are needed, if the feature is not
used.

It's advisable to add a git pre-commit hook for the main repository to
prevent changes to "qa/patch.json", keeping it to "[]". Suggestion for
such a hook:

  # Exit code 1 if the patch file isn't just "[]":
  python -c 'import json, sys; \
    exit(0 if json.load(open("qa/qa-patch.json", "r")) == [] else 1)'

Locally it's possible to tell git not to track changes to the file at
all (see https://help.github.com/articles/ignoring-files):

  git update-index --assume-unchanged qa/qa-patch.json

Cherry-pick of e5398c3a

 from stable-2.11.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The instance multi-allocation had no tests to detect its breakage, and
this patch fixes that.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The newly added job QA has some flakiness with respect to its use of
gnt-job watch. Fix this by waiting until the canceling status is
replaced with the canceled status, or a timeout is reached.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch introduces a QA test in which a job is cancelled while
waiting.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The QA did not have a test for network tags until now, and this patch
remedies the situation.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* Mention restrictions in NEWS
  - RBD userspace access mode
  - In case of a downgrade instances should suffer a reboot to be migrateable
* Bypass interactive verification in NIC modifications via --force option
  - Mention it in man page
* Print "modifications take place after restart" message no matter if
  --hotplug option is passed. Change cmdlib to append hotplug status info
  (if any) to the final result printed out to the user
* Change hotplug option description
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Only if default hypervisor is KVM try all hotplug related actions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch adds additional 'gnt-node list' commands to
the testing of 'gnt-cluster master-failover' in order to
test if ConfD (or LuxiD) is still running after a
master-failover.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch removes a QA test that was supposed to test
that unused disk templates can be disabled safely. The
test is now subsumed by unit tests and thus we remove it
here. Besides that, it was actually testing the wrong
thing, as it tested that a _used_ disk template could
be disabled.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch replaces all usages of the utility function
'GetLvmDiskTemplate' by the new 'DTS_LVM' constant
to make it consistant with the usage of other DTS_*
constants.

Additionally, it provides a unit tests to ensure
consistancy between DTS_LVM and the mapping of disk
templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Newer versions of pylint are more strict regarding indentation rules. This
commit fixes a couple of indentations that they consider wrong.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Update references to constants that were previously in the generated
'lib/_autoconf.py' and since the previous patch have been included in
the Hs2Py constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Fix all instances of pep8's error: "E128 continuation line under-indented for
visual indent".
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Mark a string representing a "sed" command including a regular expression as a
raw string, so that the entire escaping is performed by "sed" itself.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Newer pylint versions complain that QaConfig is not a proper container because
it does not implement the __len__(), __delitem__() and __setitem__() methods.
This commit implements them.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

One feature of the new private-module layout is that a python
script cannot simply 'import ganeti'. The reason is that the
interfaces to these modules are not stable and hence should not
be used outside Ganeti. However, this is what the burnin test
in the qa does. The good news is that we know that the burnin
script will always be from the same version of ganeti as the
one we're testing against; so we can simply provide the appropriate
PYTHONPATH.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

LD_* constants are basically like DT_* constants, except
for that both file and shared file were mapped to file.
In order to not having to maintain three slightly different
sets of disk-related constants (DT, LD and ST), we merge
DT and LD here. This patch does the renaming of the
various constants and then fixes the handling of shared
file disks.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Ipolicy-allowed disk templates are now dependend on the
cluster-wide enabled disk-templates. The QA for file
storage did not consider this updated logic so far.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes the '--no-drbd-storage' option completely
from the command line interface. The hugepages design
doc still had a left-over reference to --no-lvm-storage,
which is removed in this patch as well.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The logic around the DRBD usermode helper so far was that
setting it was only possible when DRDB was not explicitely
disabled. This patches changes it in a way that it is
consistent to how Ganeti handles the volume group name.

Now, the user can specify a DRBD usermode helper independent
of whether or not DRBD is enabled or not. She will however
get a warning when she sets a helper without having DRBD
enabled. The reasoning behind this is that one might want
to configure a helper while not yet having set up DRBD
completely or while having DRBD disabled temporarily without
loosing this piece of configuration.

This change was done earlier in the patch series, because
I wanted to do the refactoring in two steps, first
just transforming the original logic from --no-drbd-storage
to --enabled-disk-templates and if that goes well, adjust
to the more user-friendly behavior.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>