Add “cluster-oob” to sample configuration file. Don't run RAPI group
tests if disabled.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.3:
  Wait for master to become available on initialization
  Start all daemons on cluster initialization
  Clarify job processing order in admin guide
  Improve option descriptions
  Remove two unused variables
  Fix LUOSDiagnose and non-vm_capable nodes
  Rephrasing two error messages for auto promotion
  storage: Check that mapper is either used or None
  Fix bug in “gnt-node list-storage”
  Improve import/export timeout settings
  Increase remote import/export timeout

Conflicts:
	lib/constants.py: Trivial
	lib/objects.py: Trivial
	qa/qa_node.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the validity of the hypervisor parameters is only checked
at init/modification time, and not in the cluster verify. This is bad,
as it can lead to inconsistent state that is only detected when the
next modification (which can be unrelated) is made, leading to
unexpected error messages.

This patch adds both syntax verification (in masterd) and validity
verification on remote nodes. The downside of the patch is that on
clusters with many instances which have custom parameters, it will be
slow. A possible improvement would be to detect duplicate, identical
set of parameters, and collapse these into a single verification, but
that is left as a TODO (in case it becomes problematic).

An additional change is in utils.ForceDict, where we said 'key',
whereas this function is always used with parameter dicts, so I
changed it to "Unknown parameter".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Remove the actual code
- Remove mentions of it from iallocator.rst, and use hail instead
- Also remove mentions of "etch-image" and use "debootstrap+default"
- Mention htools as the reference implementation in iallocator.rst

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cli._RunWhileClusterStoppedHelper.Call passes (self, *args) to functions
called via cli.RunWhileClusterStoppedHelper(). The code in cluster-merge
was broken by commit d8aab233.

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 70b0d2a2 broke unittests on Python 2.4 and 2.5. Turns out that
Python 2.6 and above allow classes to be passed as custom test runners,
whereas earlier versions don't.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

In cases where secondary was offline and not evacuated watcher tried
to activate-disks in an endless manner, but this is useless, as the
secondary is offline and therefore not responding to this approach.

This patch skips activation of the disk if the secondary is bad but
instance up and running.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The recent work on multi-VG support has converted LUClusterVerifyDisks
into doing serialised calls to each node, as each node can have
different VGs. This is suboptimal, especially for big clusters, where
this LU is executed by the watcher very often.

This patch changes the logic based on the observation that querying a
node for its VGs and then requesting a LV list for those VGs is
equivalent to simply asking for all LVs, without specifying the VG
name(s). So backend.py needs changes to accept an empty VG list, and
the LU itself partially reverts to the previous version.

Additionally, we do two other fixes to this LU:

- small improvement in getting the instance list from the config
- MapLVsByNode works for all disk types, hence no need to restrict to
  the DRBD template, especially as today we can "recreate" disks for
  plain volumes too (the warning message in gnt-cluster is updated
  too)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Recent multi-VG work already exports the missing LV names as vg/lv,
not simply lv. So the query and addition of the VG name in gnt-cluster
verify-disks is redundant, and even wrong for non-default-VG
instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In some cases (e.g. the hypervisor not running at all), we might want
to force disk deactivation, skipping the hypervisor checks. I believe
this is not a good thing to do all the time, so this patch adds the
force option to allow manual selection of this operation mode.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is analogue to the existing check for a responsive node daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

At least ganeti-confd was not started. It got started a few minutes
later by ganeti-watcher. Also move one pylint disable to the effective
line.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The fact that jobs don't necessarily execute in order has been source
for some confusion. Hopefully this update will clarify things.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also replace hardcoded “xenvg” with constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since we can blacklist/hide non-existing OSes (for preseeding), we
cannot query easily the OSes themselves for this status. Hence we
export the entire lists in cluster info (which should be cheaper than
gnt-os diagnose).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Also replace one UTF-8 char with the ASCII equivalent, not all Pandoc
versions support it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is because we want, whenever we use sequences of commands, to set
pipefail, otherwise detecting build failures is difficult.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This skips non-vm_capable nodes in the OS diagnose search, since such
OSes will not be used anyway on those nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Using auto_promote or auto-promote can lead to confusion on using the
user facing interfaces. While auto-promote is fine for CLI it's not for
RAPI and vice-versa. This patch should eliminate this confusion.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The if structure in CheckReread is broken, and makes partitions reread
be full of race issues (esp. after updating them).

Also fix a small message bug.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is a followup patch to the one moving GetAllocatable out to
module level.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

LVM PV storage units would always show as allocatable, even when they
weren't. For some reason I have not been able to determine, the function
parsing the attributes (“_GetAllocatable”) was not even called and the
list opcode simply returned the attribute string as the value (e.g.
“a-”).  Removing “@staticmethod” did the trick and then I just moved it
to module level.

A QA test is included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This logic error was not detected before as health has not been
implemented on the cli and therefore no QA code existed for that.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is a bug I recognized while doing tests on gnt-node health. A left
over result.Raise line causes premature abort of LUOobCommand on the
first node failing the RPC call. This is not expected behaviour for
multiple nodes.

This bug is not critical for the current use case as for a single node
this just works as expected. Though for multi node this will fail,
that's the reason why QA and tests passed as currently we just had
single node invocation. QA for multi node will be included in the
gnt-node health, as this is the first command using multi node.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This replaces a number of equal “ht.TOr(ht.TDict, ht.TNone)” checks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows a lot of simplification in the TestIcmpPing, as fping can
take multiple arguments so we don't need anymore to create many
commands joined with &&.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will change the result of this LU to a query like result. A list of
tuples with information about the state of the data.

It also includes the modification to the commands calling this opcode.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.4:
  Another fix for LUClusterVerifyDisks
  QA: also run gnt-cluster verify-disks
  Fix disk adoption breakage
  Fix typo in query2 design document
  Improve documentation for QRFS_UNAVAIL
  lvmstrap: add PV-on-partition support
  lvmstrap: abstract a little the sysfs paths
  lvmstrap: ignore small-sized partitions
  lvmstrap: add explicit test for swap backends
  lvmstrap: add an explicit test for mounted devices
  lvmstrap: add more excluded FS types
  lvmstrap: fix very old contact information
  Instance query: replace duplicates with aliases
  query: Add alias support in _PrepareFieldList
  Fix disk count check in LUSetInstanceParams
  Document iallocator change (alloc_policy)

Conflicts:
        lib/constants.py (due to QRFS→RS on master and doc on 2.4)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This patch renames QRFS_* to RS_* fields so they can be used in other
places (i.e. LUs) without confusion, as this was initially meant for
query operations.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The LVM queries should only be done for vm_capable nodes. In order to
do this, we also add a new ConfigWriter method to abstract that query.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The bug recently reported by Apollon Oikonomopoulos was missed because
we don't test this command at all.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Disk adoption is currently broken by 84d7e26b, which added multiple LVM
volume group support. This patch fixes the calls to rpc.call_vg_list,
which are multi-node calls but were handled as single-node calls in
84d7e26b.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>