GitLab

* stable-2.8
  Include qa/patch in Makefile
  Handle empty patches better
  Move message formatting functions to separate file
  Add optional ordering of QA patch files
  Allow multiple QA patches
  Refactor current patching code
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.8
  Include qa/patch in Makefile
  Handle empty patches better
  Move message formatting functions to separate file
  Add optional ordering of QA patch files
  Allow multiple QA patches
  Refactor current patching code

Conflicts:
	Makefile.am (include all added files)
	qa/qa_cluster.py (trivial)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fix the build by including the qa/patch directory in Makefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The previous patch loading utilities omitted empty patches, as they
were thought to be of no significance, and when no patches were used,
the import and therefore dependency should not be used. If a user has
added an empty patch file, and made an entry in the order file, the QA
would treat this as an error as it had no means of differentiating
between a patch not present and an empty patch.

This patch fixes the solution by better handling empty patches, and
logging warnings when they are encountered.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The formatting functions in qa_utils.py cannot be used by modules
imported there, such as qa_config. This patch factors the function
calls into a separate file. Also reorders imports in touched files.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To explicitly specify the order of patches executed, the QA provides an
"order" file. It can contain names of patches that will be executed
first, and in the order listed, before all the other patches that still
follow an alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch allows support for multiple patches placed in the "patch"
directory, which are executed in alphabetical order.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* Refactors the current patch code to allow for multiple patches that
  can be applied, yet leaves only one for now.

* Rewords the message shown to the user in case the modules needed for
  patching are missing.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

With regard to PCI slot occupied by a KVM instance we have
observed the following:

1) Slot 0 will always be Host bridge.
2) Slot 1 will always be ISA bridge.
3) Slot 2 will always be VGA controller (even with -display none).
4) If soundhw=hda|ac97|es1370 an extra PCI slot is occupied.
   This slot *MUST* be the No. 3.

5) Option '-balloon virtio' gets an extra PCI slot.
   Still it can take id, bus, and addr args and be placed anywhere

6) If spice is used instead of vnc we have:
   * No extra PCI slot gets occupied without vdagent
   * Otherwise we have the following extra optionsa [1]
     a) -device virtio-serial-pci
        (this can take id, bus, and addr args too)
     b) -device virtserialport,chardev=spicechannel0,...
     c) -chardev spicevmc,id=spicechannel0

This patch does the following:

1) Change _DEFAULT_PCI_RESERVATIONS to "1110...."
2) Move soundhw option early in the command line and if hda etc.
   reserve slot 3.
3) Add id, bus, and addr in balloon option and reserve next slot.
4) Add id, bus, and addr in -device virtio-serial-pci option and if
   passed reserve next slot.

[1] http://www.linux-kvm.org/page/SPICE



Fixes issue 757.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Introduce new method `_GetFreeSlot()` responsible only for bitarray
operations. It fixes search in case of bitarray is either '0000..'
or '1111..'.

Use it instead of `_UpdatePCISlots()` and in `_GetFreePCISlot()`.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

* stable-2.9
  Fix error introduced during merge
  gnt-cluster copyfile: accept relative paths

* stable-2.8
  Improve RAPI detection of the watcher
  Add patching QA configuration files on buildbots
  Enable a timeout for instance shutdown
  Allow KVM commands to have a timeout
  Allow xen commands to have a timeout
  Fix wrong docstring
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

A parameter was lost while resolving a conflict in the signature of a function.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If, on the command line, the argument to gnt-cluster copyfile is
a relative path, consider this a shorthand for the corresponding
absolute path.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.8
  Improve RAPI detection of the watcher
  Add patching QA configuration files on buildbots
  Enable a timeout for instance shutdown
  Allow KVM commands to have a timeout
  Allow xen commands to have a timeout
  Fix wrong docstring

Conflicts:
	lib/hypervisor/hv_xen.py
	test/py/ganeti.hypervisor.hv_xen_unittest.py

Resolution:
  The conflicts were due to a new mandatory parameter hvparams being added
  (in 2.9) to the same functions that received the optional timeout parameter
  (in 2.8). They were resolved by keeping the hvparams in its place, and
  enqueuing the timeout parameter after it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If the watcher is not allowed to access RAPI, it doesn't mean that it is dead
and needs to be restarted.

Fixes Issue 752.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In order to work, it is necessary:
* Write a JSON Patch to file "qa/qa-patch.json" in a source repository.
  (And be careful not to commit it!) The file must conform to RFC6902
  and describes what changes should be done to the QA JSON
  configuration. This is simple as
   [
     { "op": "add",
       "path": "default",
       "value": false },
     { "op": "add",
       "path": "/tests/instance-add-file",
       "value": true }
   ]

* Install "jsonpatch" and "jsonpointer" modules on the machines QA is
  running on. Either using "easy_install" or as Debian packages using
  "pypi-install".

QA only loads the modules if the patch file is present and non-empty.
Therefore no changes to buildbots are needed, if the feature is not
used.

It's advisable to add a git pre-commit hook for the main repository to
prevent changes to "qa/patch.json", keeping it to "[]". Suggestion for
such a hook:

  # Exit code 1 if the patch file isn't just "[]":
  python -c 'import json, sys; \
    exit(0 if json.load(open("qa/qa-patch.json", "r")) == [] else 1)'

Locally it's possible to tell git not to track changes to the file at
all (see https://help.github.com/articles/ignoring-files):

  git update-index --assume-unchanged qa/qa-patch.json

Cherry-pick of e5398c3a

 from stable-2.11.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Add the timeout parameter to the StopInstance function of the hypervisor base
class and to all its implementations.

Also, change the tests as required by this change.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Modify the function that sends commands to the KVM monitor so that it is
possible to specify an optional timeout after which the command is killed.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Modify the function that runs Xen commands so that it is possible to specify an
optional timeout after which the command is killed.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fields must be the final elements in an epytext string.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Set the version to 2.10.1.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Set the release date to tomorrow and mentioned fixed bugs in the NEWS
file.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

LUNodeAdd, the only LU using a node name still, is changed to overwrite
PreparePostHookNodes() and use node UUIDs only as well.
This allows to remove the support for 3-tuples as results of
BuildHooksNodes() and removes the translation to node names.

Fixes issue 742.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This method can be used to alter the list of node UUIDs on which post
hooks are executed. PreparePostHookNodes is called after Exec, so LUs
can use data only known after the execution of the LU.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

An error in the post-commit hooks could not be propagated correctly and could
result in e.g. the return code of gnt-cluster verify to be 0 even in presence of
an error in its output.

Fixes Issue 744.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

* stable-2.9
  Make gnt-debug locks display fake job locks properly
  Make NiceSort treat integers well
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

When a job is dependent on other jobs, a fake lock is created whose
pending entry contains a list of job ids waiting on the job. gnt-debug
locks did not expect the job ids to be ints, crashing when encountering
them. This patch fixes the situation by forcing a string cast.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

NiceSort is invoked on arrays that may contain strings, but in other
situations can contain ints as well. As this surprisingly makes sense,
add a tiny modification to make NiceSort work in these conditions.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

* stable-2.9
  Revision bump for 2.9.5
  Set release date for 2.9.5
  Add NEWS entry for 2.9.5

Conflicts:
	NEWS: take both additions
	configure.ac: ignore revision bump
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Describe the changes since 2.9.4 for the soon-to-be-released
version 2.9.5.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.8
  Fix expression describing optional parameters
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

* stable-2.8
  Fix expression describing optional parameters
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The NIC's network and vlan are also newly added, hence need to be
considered optional to remain backwards compatible.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.9
  Fix net-common script in case of routed mode
  Document hspace's SPN parameters
  Document spindles in TSPEC
  man: Update the manual page of gnt-node FAILOVER
  Make hbal deal with no-LVM storage space properly

* stable-2.8
  Improve job status assert affected by race condition
  Export and import Disk/NIC name
  Fix backup import in case NIC is inside a network
  Override get() method of ConfigParser
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

setup_route should run in case or routed mode and not
openvswitch.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Cherry-picked from 49c60905

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

With the introduction of spindles as resources, the machine-readable
output of hspace(1) has been extended. Document the additional parameters.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

As spindles are now considered a resource, they are also output in
the TSPEC parameter of hspace --machine-readable. Document this
interface change.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Before the tuple of nodes of an instance was created from a set, listing
the nodes in alphabetical order. This patch ensures that the primary
node is always the first one in the list.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>