This patch adds an enable/disable option for htools (--enable-htools),
and associated tests for determining whether GHC (the compiler we use
for htools) and required libraries are present.

The method to do so is not very nice; usually, Haskell programs are
configured and compiled using cabal (http://www.haskell.org/cabal/

, a
tool similar to Python's setuptools)), but that doesn't suit itself to
nice integration with autoconf/automake, so we test for the modules
presence manually.

In the end, we set a few variables:

- GHC: the path to the ghc compiler
- HTOOLS_MODULES: command line option for ghc to select the wanted
  'parallel' module
- HTOOLS_NOCURL: set to -DNO_CURL if we don't want to enable curl (and
  thus RAPI) support in htools
- HTOOLS: set to 'yes' if we should compile/install the htools
  programs
- HTOOLS_APIDOC: set to yes if we should build/install the htools
  API documentation
- WANT_HTOOLS, WANT_HTOOLSAPIDOC: two automake conditionals for later
  use in Makefile.am

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Use constants
- Don't sleep if no delay is given
- Mark function as deprecated: it uses polling instead of waiting for changes
  (but the latter needs authentication); it can still be used
- Add unittests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Commit dd7f6776 added another call to BuildHooksEnv to provide
post-phase status variables. Since BuildHooksEnv also built the node
lists, that meant they have to be built twice. First a rather strict
check was used, but it turned out to be more tricky. Commit b423c513
had to remove the strict check again.

With this patch the function is split in two parts, one generating the
actual environment variables, and another part returning the node lists.
The former is called twice.

Unittests are updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

cfgupgrade provides a --ignore-hostname option, however consequently it
spawns config.ConfigWriter, which refuses to handle a foreign
configuration file by default.

config.ConfigWriter provides a relative keyword argument,
accept_foreign, so we tie this to the value of options.ignore_hostname.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds a new method WaitForJobCompletion that can be used for
client who are not interested in the entire job log, just in its
completion status.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

I didn't know where to put in the TOC, so I added it after the ganeti
2.3 design doc.

Also, it seems that Sphinx is a bit limited in its list nesting
(compared to Pandoc), so I had to rework a bit the indentation (it
still doesn't generate good output, but…).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This moves the man pages to their right place, removes the static
footer and header, and adds Makefile rules for the new .1 man pages.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Commit dd7f6776 added a restrictive check for the node lists returned
by BuildHooksEnv, leading to errors with some LUs, one of which was
fixed in commit 0dfa2c22. As it turns out, other LUs have similar
issues, some not easy to fix. This patch disables the restrictive check
until the BuildHooksEnv functions can be split into one part generating
the actual environment and one generating the node lists.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 8bc9e04a, not
needed anymore.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* htools/master: (605 commits)
  Update NEWS file for version 0.3.1
  Update copyright years in the man pages
  Remove obsolete env vars from the manpages
  Fix dist archive generation
  luxi backend: show attribute names in errors
  Update exit code information in the hbal man page
  Fix broken hscolour.css symlink in tarball
  Update NEWS file for the 0.3.0 release
  Remove an obsolete TODO
  Update live-test.sh w.r.t. simulation mode changes
  Prevent allocation on unallocable node groups
  Fix bug in simulation backend
  Precompute allocation nodes
  Change the AllocNodes type and functions
  Extract node pair generation from tryAlloc
  tryAlloc: restrict valid node pairs to same-group
  Cluster.hs: add a new type alias
  Rapi: read and use the vm_capable node flag
  IAllocator: read and use the vm_capable node flag
  IAllocator: replace fake policy with real one
  ...

Note: done via the following sequence of commands:

  git checkout master && git pull
  git remote add htools git://git.ganeti.org/htools.git


  git remote update
  git merge -s recursive -X subtree=htools htools/master

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In order for “git merge -s recursive -X subtree=htools” to do its
magic, an (mostly empty) directory must exist in the target tree name
‘htools’. Hence we add a temp file to it before the merge, and remove
it after that.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This broke QA (and everyone trying to add a node) by complaining about
different node lists.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When a hook is called, it is provided with a number of variables
describing the status of the instance/node/etc. before the operation.
Some opcodes provide extra variables to see modified values from hooks,
but that's not a generic solution.

This patch modifies the code calling hooks to generate the environment
once before and once after an opcode has been executed. Doing so should
be safe—I did not find any LU.BuildHooksEnv modifying LU instance
attributes. The values collected after running the opcode are prefixed
with “GANETI_POST_”, as opposed to “GANETI_” for pre-execution
variables. The latter are still provided for backwards compatibility.

Environment variable examples:

gnt-instance start $instance:
GANETI_INSTANCE_STATUS=down
GANETI_POST_INSTANCE_STATUS=up

gnt-instance modify -B memory=512 $instance:
GANETI_INSTANCE_BE_memory=768
GANETI_POST_INSTANCE_BE_memory=512

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also replace explicit loop with dict.update.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Dictionary indentation
- Add empty lines for readability
- Simplify conditional code

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- Access is only permitted for authenticated clients (queries can return
  sensitive data)
- Filters can be specified when sending a PUT request
- Updates RAPI client, documentation and tests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Instead of going via cmdlib and using special cases for different
resources, the list of fields is used directly.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Commit 28b71a76 added a list of resources which can be queried using
LUXI. Unfortunately the variable was named “QR_OP_LUXI”, which can be
confusing. This patch renames “QR_OP_QUERY” to “QR_VIA_OP”, “QR_OP_LUXI”
to “QR_VIA_LUXI” and adds a new variable, currently equal to
“QR_VIA_LUXI”, for remote API queries (mostly for documentation).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Previously we failed later with a rather useless error message. This
patch fixes this and tells the user to activate-disks if replace-disks
is in the need of activated disks rather than abort with a cryptic error
message.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The behaviour of LUOsDiagnose needs special treatment. Commit d22dfef7
changed it to not return hidden, blacklisted or invalid OSes if the
respective field is not requested. This behaviour needs to be preserved
for users of the old opcode, but since it provides an implicit
filtering, it should not be done for LUQuery (the generic way to query
for resources) and is therefore moved from the underlying implementation
to LUOsDiagnose. Users of the new opcode should pass an explicit filter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It's not perfect, but at least some more.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Allows converting the value of a column before returning it. Useful for
sorting while still using one of the other generic functions for
retrieving the value.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Remove duplication by merging two MatchNameComponent into a wrapper
- Reduce lock duration by getting list of names under lock and then
  matching names without the lock
- Also, ExpandNodeName's docstring is fixed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Commit 16c2369c removed support for using environment variables for
selecting the text backend (since now it's just one option), but
didn't update the man pages.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Patches db366d9a and aac4511a added support for EXT_MIRROR instances,
but inadvertently introduced a bug: for INT_MIRROR cases, we don't
need (actually we can't support) neither an iallocator nor a target
node.

To fix this, we move the iallocator/node checks in CheckPrereq (or
respectively in the tasklet CheckPrereq), where we have access to the
instance configuration, and additionally we check for and prevent
passing either of these two for INT_MIRROR instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>