GitLab

The list-drbd QA test assumed falsely that no instance has more than
one disk. This can happen, and this patch accounts for it.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When disk templates were ignored in the QA, no messages were emitted,
in contrast to pretty much all tests. To make it easier to see why
a test was not executed, this patch performs a minor refactoring and
adds some output for the templates.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds a tiny helper function that allows error messages
output by the QA asserts to be prefixed with whatever the user
supplies.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.8
  Add reason parameter to RAPI client functions
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The UUID/name switch avoided this particular bit of code, and as a
result the list-drbd command failed as it tried to compare UUIDs and
names. This patch fixes the retrieval, converts the newly returned
UUIDs to names, and modifies the QA to the results and not only the
invocation are checked.
Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Only the functions for starting, stopping and rebooting a VM had a reason
parameter. Now, all the RAPI client functions generating opcodes do.

Also, one test is expanded to verify that a RAPI request with both body and
query supports the reason parameter properly.

Fixes Issue 776.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Make the watcher collect its data using low-priority jobs,
to avoid blocking user/admin jobs. Note that repair jobs are
still submitted normal priority. Fixes issue 772.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.8
  Include qa/patch in Makefile
  Handle empty patches better
  Move message formatting functions to separate file
  Add optional ordering of QA patch files
  Allow multiple QA patches
  Refactor current patching code

Conflicts:
	Makefile.am (include all added files)
	qa/qa_cluster.py (trivial)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fix the build by including the qa/patch directory in Makefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The previous patch loading utilities omitted empty patches, as they
were thought to be of no significance, and when no patches were used,
the import and therefore dependency should not be used. If a user has
added an empty patch file, and made an entry in the order file, the QA
would treat this as an error as it had no means of differentiating
between a patch not present and an empty patch.

This patch fixes the solution by better handling empty patches, and
logging warnings when they are encountered.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The formatting functions in qa_utils.py cannot be used by modules
imported there, such as qa_config. This patch factors the function
calls into a separate file. Also reorders imports in touched files.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To explicitly specify the order of patches executed, the QA provides an
"order" file. It can contain names of patches that will be executed
first, and in the order listed, before all the other patches that still
follow an alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch allows support for multiple patches placed in the "patch"
directory, which are executed in alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* Refactors the current patch code to allow for multiple patches that
  can be applied, yet leaves only one for now.

* Rewords the message shown to the user in case the modules needed for
  patching are missing.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

A parameter was lost while resolving a conflict in the signature of a function.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If, on the command line, the argument to gnt-cluster copyfile is
a relative path, consider this a shorthand for the corresponding
absolute path.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.8
  Improve RAPI detection of the watcher
  Add patching QA configuration files on buildbots
  Enable a timeout for instance shutdown
  Allow KVM commands to have a timeout
  Allow xen commands to have a timeout
  Fix wrong docstring

Conflicts:
	lib/hypervisor/hv_xen.py
	test/py/ganeti.hypervisor.hv_xen_unittest.py

Resolution:
  The conflicts were due to a new mandatory parameter hvparams being added
  (in 2.9) to the same functions that received the optional timeout parameter
  (in 2.8). They were resolved by keeping the hvparams in its place, and
  enqueuing the timeout parameter after it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If the watcher is not allowed to access RAPI, it doesn't mean that it is dead
and needs to be restarted.

Fixes Issue 752.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In order to work, it is necessary:
* Write a JSON Patch to file "qa/qa-patch.json" in a source repository.
  (And be careful not to commit it!) The file must conform to RFC6902
  and describes what changes should be done to the QA JSON
  configuration. This is simple as
   [
     { "op": "add",
       "path": "default",
       "value": false },
     { "op": "add",
       "path": "/tests/instance-add-file",
       "value": true }
   ]

* Install "jsonpatch" and "jsonpointer" modules on the machines QA is
  running on. Either using "easy_install" or as Debian packages using
  "pypi-install".

QA only loads the modules if the patch file is present and non-empty.
Therefore no changes to buildbots are needed, if the feature is not
used.

It's advisable to add a git pre-commit hook for the main repository to
prevent changes to "qa/patch.json", keeping it to "[]". Suggestion for
such a hook:

  # Exit code 1 if the patch file isn't just "[]":
  python -c 'import json, sys; \
    exit(0 if json.load(open("qa/qa-patch.json", "r")) == [] else 1)'

Locally it's possible to tell git not to track changes to the file at
all (see https://help.github.com/articles/ignoring-files):

  git update-index --assume-unchanged qa/qa-patch.json

Cherry-pick of e5398c3a

 from stable-2.11.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Add the timeout parameter to the StopInstance function of the hypervisor base
class and to all its implementations.

Also, change the tests as required by this change.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Modify the function that sends commands to the KVM monitor so that it is
possible to specify an optional timeout after which the command is killed.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Modify the function that runs Xen commands so that it is possible to specify an
optional timeout after which the command is killed.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fields must be the final elements in an epytext string.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When a job is dependent on other jobs, a fake lock is created whose
pending entry contains a list of job ids waiting on the job. gnt-debug
locks did not expect the job ids to be ints, crashing when encountering
them. This patch fixes the situation by forcing a string cast.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

NiceSort is invoked on arrays that may contain strings, but in other
situations can contain ints as well. As this surprisingly makes sense,
add a tiny modification to make NiceSort work in these conditions.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Describe the changes since 2.9.4 for the soon-to-be-released
version 2.9.5.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.8
  Fix expression describing optional parameters
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The NIC's network and vlan are also newly added, hence need to be
considered optional to remain backwards compatible.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

setup_route should run in case or routed mode and not
openvswitch.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Cherry-picked from 49c60905

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

With the introduction of spindles as resources, the machine-readable
output of hspace(1) has been extended. Document the additional parameters.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

As spindles are now considered a resource, they are also output in
the TSPEC parameter of hspace --machine-readable. Document this
interface change.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

In particular explicitly note that a node failover causes all instances
to be stopped and later restarted (issue #724).
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.8
  Improve job status assert affected by race condition
  Export and import Disk/NIC name
  Fix backup import in case NIC is inside a network
  Override get() method of ConfigParser

Conflicts:
	lib/cmdlib/instance.py: manually apply 0973f9ed

 on
	  the stable-2.9 version
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In the sliver of time between choosing a waiting job to be executed and
trying to acquire locks for its execution, the status of the job can be
changed to canceling. An assert checking the job status neglected to
take this into account, and raised an error that managed to perpetually
lock the job in the canceling state. This patch resolves the issue by
making the assert accept the canceling state as well, and exiting if
the job was cancelled.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Name of Disk/NIC were not exported during backup until now.
Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Network UUID is written in .ini file during backup export
but is not used by _ReadExportParams(). This patch fixes it.

Please note that in case a network is given, link and mode should
not be included in NIC options.

This fixes issue 716.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

During backup import/export SafeConfigParser() is used to
save/restore instance's configuration. There is a possibility if an
export is done with a different Ganeti version, a specific value not
to be saved during export (e.g. the NIC/Disk name) but still
requested during import.

With this patch we override the get() method of SafeConfigParser()
and catch NoOptionError if raised and return None. Additionally we
translate "None" values read from .ini file into None.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Since 2.6, hbal crashes when used on a cluster where no
LVM storage is enabled at all. The problem is that it
always queries for fields that only sometimes make
sense for certain types of storage. This patch will
make it fall-back to a default value if those fields
are not returned by a query. This way it will simply
disregard the space information in its calculation
which is the best we can do in this situation, because
not all storage types support space reporting.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>