This fixes the doc builds broken by commit b292b986, but is only an
ugly workaround due to our dependency chain.

opcodes.py imports (both directly and indirectly) constants.py, which
imports _autoconf.py. Hence one could say that all .py files have a
logical dependency on _autoconf.py, since you can't "import opcodes"
before _autoconf.py is built. So the sphinx rule should depend on
_autoconf.py as a prerequisite.

But we can't use a normal prerequisite, since that would cause the
html docs to be rebuilt whenever _autoconf.py gets rebuilt, i.e. even
on end-users machines after ./configure; this is bad, and it also
breaks distcheck (as the html files are not removed in distclean,
since we do want them distributed).

So we only add an order-only prerequisite, which does the right thing
(although our dependency chain is still ugly).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This replaces hardcoded lists of parameters with automatically generated
ones, making maintenance easier.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now this just registers a single new directive, “opcode_params”,
which can be used to generate opcode parameter documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These will be used in automatically generated parts of the RAPI
documentation. Not all opcodes are documented so far, but each added
opcode's parameters need to be fully documented (verified by an
assertion).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now they're all set to None, but a following patch will set descriptions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds text descriptions to all T* checks. Converting to a
string (e.g. str(ht.TNone)) will produce a user-readable text. This
will be used for Opcode parameter documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, when reinstalling an instance, especially when selecting an
OS interactively, is hard to be sure one has chosen the correct OS.

This patch adds displaying the selected OS. I'm not entirely happy, as
it's not very good English (e.g. keeping the same OS should be "but
keep the same OS", and changing should be "and also change …") but I
didn't find a better handling of all the cases:

node1# gnt-instance reinstall -o lenny-image+default instance1
This will reinstall the instance 'instance1' (and change the OS to
'lenny-image+default') which removes all data. Continue?
y/[n]/?: n

node1# gnt-instance reinstall instance1
This will reinstall the instance 'instance1' (and keep the same OS)
which removes all data. Continue?
y/[n]/?: n

node1# gnt-instance reinstall instance1 instance2
The reinstall will operate on 2 instances.
Note: this will remove *all* data for the below instances! It will
keep the same OS.
Do you want to continue?

node1# gnt-instance reinstall -o lenny-image+default instance1
instance2
The reinstall will operate on 2 instances.
Note: this will remove *all* data for the below instances! It will
change the OS to 'lenny-image+default'.
Do you want to continue?

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This allow to pass in --oob-timeout to define the timeout before the
out-of-band helper is beeing aborted.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This also has the sideeffect of presenting a table with the result

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The tty can have strange side effect as it seems to replace \n with
\r\n. In case of simply catting a file and compare it's output this
will break. This patch adds the possibility to disable tty if needed.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In commit bfceedbe a check was added to put the master at the end or
skip it completely. While this functionality works, it was done at the
wrong point because node_names was already processed to a node list
without this checks. The Exec method if the LU just used that node list.

To overcome this situation we correct the node_names before the node
list is generated.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will allow us to use this functionality also in other cli-tools
like gnt-node power where we've to operate on multiple nodes.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was stating "secondary nodes" were the keys of the dictionary, when
they are primary nodes. Also, further clarify only the node's secondary
instances are included.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds ignore_status to ignore the offline flag of nodes
and also adds a force_master option to force operations on master node
if they will make the master unavailable (for some time).

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* devel-2.4: (34 commits)
  cluster verify: add hvparams verification
  Remove dumb-allocator
  Open other clusters' config in foreign mode
  Add (unused) arg to _OfflineClusterMerge
  Fix unittest breakage on Python 2.4/2.5
  Check for duplicate RAPI URIs and handlers
  Ensure all resources are used by RAPI client
  RAPI client: De-/activating instance disks
  RAPI client: Wrap /2/redistribute-config resource
  Add unittest for RAPI client's ModifyInstance
  Watcher: Fix endless repair tries for broken secondary
  Verify disks: increase parallelism and other fixes
  gnt-cluster verify-disks: fix VG name
  Deactivate disks: allow skipping hypervisor checks
  Wait for master to become available on initialization
  Start all daemons on cluster initialization
  Clarify job processing order in admin guide
  Improve option descriptions
  Remove two unused variables
  Show hidden/blacklisted OSes in cluster info
  ...

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.3:
  Wait for master to become available on initialization
  Start all daemons on cluster initialization
  Clarify job processing order in admin guide
  Improve option descriptions
  Remove two unused variables
  Fix LUOSDiagnose and non-vm_capable nodes
  Rephrasing two error messages for auto promotion
  storage: Check that mapper is either used or None
  Fix bug in “gnt-node list-storage”
  Improve import/export timeout settings
  Increase remote import/export timeout

Conflicts:
	lib/constants.py: Trivial
	lib/objects.py: Trivial
	qa/qa_node.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the validity of the hypervisor parameters is only checked
at init/modification time, and not in the cluster verify. This is bad,
as it can lead to inconsistent state that is only detected when the
next modification (which can be unrelated) is made, leading to
unexpected error messages.

This patch adds both syntax verification (in masterd) and validity
verification on remote nodes. The downside of the patch is that on
clusters with many instances which have custom parameters, it will be
slow. A possible improvement would be to detect duplicate, identical
set of parameters, and collapse these into a single verification, but
that is left as a TODO (in case it becomes problematic).

An additional change is in utils.ForceDict, where we said 'key',
whereas this function is always used with parameter dicts, so I
changed it to "Unknown parameter".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Remove the actual code
- Remove mentions of it from iallocator.rst, and use hail instead
- Also remove mentions of "etch-image" and use "debootstrap+default"
- Mention htools as the reference implementation in iallocator.rst

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cli._RunWhileClusterStoppedHelper.Call passes (self, *args) to functions
called via cli.RunWhileClusterStoppedHelper(). The code in cluster-merge
was broken by commit d8aab233.

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 70b0d2a2 broke unittests on Python 2.4 and 2.5. Turns out that
Python 2.6 and above allow classes to be passed as custom test runners,
whereas earlier versions don't.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

In cases where secondary was offline and not evacuated watcher tried
to activate-disks in an endless manner, but this is useless, as the
secondary is offline and therefore not responding to this approach.

This patch skips activation of the disk if the secondary is bad but
instance up and running.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The recent work on multi-VG support has converted LUClusterVerifyDisks
into doing serialised calls to each node, as each node can have
different VGs. This is suboptimal, especially for big clusters, where
this LU is executed by the watcher very often.

This patch changes the logic based on the observation that querying a
node for its VGs and then requesting a LV list for those VGs is
equivalent to simply asking for all LVs, without specifying the VG
name(s). So backend.py needs changes to accept an empty VG list, and
the LU itself partially reverts to the previous version.

Additionally, we do two other fixes to this LU:

- small improvement in getting the instance list from the config
- MapLVsByNode works for all disk types, hence no need to restrict to
  the DRBD template, especially as today we can "recreate" disks for
  plain volumes too (the warning message in gnt-cluster is updated
  too)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Recent multi-VG work already exports the missing LV names as vg/lv,
not simply lv. So the query and addition of the VG name in gnt-cluster
verify-disks is redundant, and even wrong for non-default-VG
instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In some cases (e.g. the hypervisor not running at all), we might want
to force disk deactivation, skipping the hypervisor checks. I believe
this is not a good thing to do all the time, so this patch adds the
force option to allow manual selection of this operation mode.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is analogue to the existing check for a responsive node daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>