These are just changes from hlint suggestions. Still compiles and
passes unittests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Requested-by: Iustin Pop <iustin@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a design document describing how jobs can be submitted
from within LUs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since we don't test for now the RAPI backend directly, we can simply
skip the import. Later we can make a conditional import if needed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Use m4_normalize to make single-line strings while removing
  unnecessary spaces
- Wrap lines longer than 80 characters

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This documents the needed libraries for Haskell development. It also
fixes a tiny typo in devnotes.rst.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Based on Michael's suggestion, this patch partially reverts my
changes. The new directories are:

- doc/api/py
- doc/api/hs
- doc/coverage/py
- doc/coverage/hs

Basically the Python-specific output moves one level down (into py/)
compared to the original location, and the Haskell stuff goes into
hs/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also add mention about it being obsolete.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should be more readable now. I wanted to even use the nicer
_SOURCES, but _SOURCES is special in Automake (again), so _SRCS it is.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Only one target wasn't ported over (the TAGS one), as hasktags is not
available easily in distributions, so it doesn't make sense to enable
it for all developers.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since we do have errors currently, this is not enabled from the main
'make lint' rule. That will get cleaned up later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This was a bit tricky, as the compilation from the top-dir changes the
paths in the .tix/.mix files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Previously, Python api doc was under doc/api (which didn't match with
the target rule, apidoc). After this patch, we have the following:

- make py-apidoc generates Python api doc under doc/py-apidoc
- make hs-apidoc generates Haskell api doc under doc/hs-apidoc
- make apidoc does both (if hs-apidoc enabled at configure time)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since we have two sets of coverage data, we make the naming
consistent:

- doc/py-coverage for Python coverage
- doc/hs-coverage for Haskell coverage

We also need to exclude some more dirs from the list of checked
directories.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We do it via per-target-binary .o/.hi files. There are a few other
options, in the end this was chosen as different binaries can have
different options/components (e.g. hail won't use RAPI, etc.) and that
the unittests need different compilation options.

We add a wildcard *.o/*.hi to CLEANFILES, so that it removes all
per-target build artifacts.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

RAPI needs the curl module, which is a binding to the libcurl C
library. This is more complex (can be compiled either against gnutls
or openssl, etc.) and it's not of much help when the tools are used on
the cluster itself, we disable it by default.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instead of bindir. We do this by adding a new allocdir variable, and
redirecting hail to it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds some (not very nice) rules for htools
compilation. Since automake only knows about some languages, and
doesn't have a real extension mechanism, I cheat and declare them as
"buildable scripts", instead of programs. Rationale:

automake support for dir_PROGRAMS is limited to a few languages; in
all cases, it assumes that translation from sources to binaries occurs
in two steps, compilation and linking; it's not easy to change its
assumption.

In our case, where “ghc --make” does both steps, it's hard to force it
into the automake model.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds an enable/disable option for htools (--enable-htools),
and associated tests for determining whether GHC (the compiler we use
for htools) and required libraries are present.

The method to do so is not very nice; usually, Haskell programs are
configured and compiled using cabal (http://www.haskell.org/cabal/

, a
tool similar to Python's setuptools)), but that doesn't suit itself to
nice integration with autoconf/automake, so we test for the modules
presence manually.

In the end, we set a few variables:

- GHC: the path to the ghc compiler
- HTOOLS_MODULES: command line option for ghc to select the wanted
  'parallel' module
- HTOOLS_NOCURL: set to -DNO_CURL if we don't want to enable curl (and
  thus RAPI) support in htools
- HTOOLS: set to 'yes' if we should compile/install the htools
  programs
- HTOOLS_APIDOC: set to yes if we should build/install the htools
  API documentation
- WANT_HTOOLS, WANT_HTOOLSAPIDOC: two automake conditionals for later
  use in Makefile.am

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Use constants
- Don't sleep if no delay is given
- Mark function as deprecated: it uses polling instead of waiting for changes
  (but the latter needs authentication); it can still be used
- Add unittests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Commit dd7f6776 added another call to BuildHooksEnv to provide
post-phase status variables. Since BuildHooksEnv also built the node
lists, that meant they have to be built twice. First a rather strict
check was used, but it turned out to be more tricky. Commit b423c513
had to remove the strict check again.

With this patch the function is split in two parts, one generating the
actual environment variables, and another part returning the node lists.
The former is called twice.

Unittests are updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

cfgupgrade provides a --ignore-hostname option, however consequently it
spawns config.ConfigWriter, which refuses to handle a foreign
configuration file by default.

config.ConfigWriter provides a relative keyword argument,
accept_foreign, so we tie this to the value of options.ignore_hostname.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds a new method WaitForJobCompletion that can be used for
client who are not interested in the entire job log, just in its
completion status.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

I didn't know where to put in the TOC, so I added it after the ganeti
2.3 design doc.

Also, it seems that Sphinx is a bit limited in its list nesting
(compared to Pandoc), so I had to rework a bit the indentation (it
still doesn't generate good output, but…).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This moves the man pages to their right place, removes the static
footer and header, and adds Makefile rules for the new .1 man pages.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Commit dd7f6776 added a restrictive check for the node lists returned
by BuildHooksEnv, leading to errors with some LUs, one of which was
fixed in commit 0dfa2c22. As it turns out, other LUs have similar
issues, some not easy to fix. This patch disables the restrictive check
until the BuildHooksEnv functions can be split into one part generating
the actual environment and one generating the node lists.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 8bc9e04a, not
needed anymore.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* htools/master: (605 commits)
  Update NEWS file for version 0.3.1
  Update copyright years in the man pages
  Remove obsolete env vars from the manpages
  Fix dist archive generation
  luxi backend: show attribute names in errors
  Update exit code information in the hbal man page
  Fix broken hscolour.css symlink in tarball
  Update NEWS file for the 0.3.0 release
  Remove an obsolete TODO
  Update live-test.sh w.r.t. simulation mode changes
  Prevent allocation on unallocable node groups
  Fix bug in simulation backend
  Precompute allocation nodes
  Change the AllocNodes type and functions
  Extract node pair generation from tryAlloc
  tryAlloc: restrict valid node pairs to same-group
  Cluster.hs: add a new type alias
  Rapi: read and use the vm_capable node flag
  IAllocator: read and use the vm_capable node flag
  IAllocator: replace fake policy with real one
  ...

Note: done via the following sequence of commands:

  git checkout master && git pull
  git remote add htools git://git.ganeti.org/htools.git


  git remote update
  git merge -s recursive -X subtree=htools htools/master

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In order for “git merge -s recursive -X subtree=htools” to do its
magic, an (mostly empty) directory must exist in the target tree name
‘htools’. Hence we add a temp file to it before the merge, and remove
it after that.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This broke QA (and everyone trying to add a node) by complaining about
different node lists.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When a hook is called, it is provided with a number of variables
describing the status of the instance/node/etc. before the operation.
Some opcodes provide extra variables to see modified values from hooks,
but that's not a generic solution.

This patch modifies the code calling hooks to generate the environment
once before and once after an opcode has been executed. Doing so should
be safe—I did not find any LU.BuildHooksEnv modifying LU instance
attributes. The values collected after running the opcode are prefixed
with “GANETI_POST_”, as opposed to “GANETI_” for pre-execution
variables. The latter are still provided for backwards compatibility.

Environment variable examples:

gnt-instance start $instance:
GANETI_INSTANCE_STATUS=down
GANETI_POST_INSTANCE_STATUS=up

gnt-instance modify -B memory=512 $instance:
GANETI_INSTANCE_BE_memory=768
GANETI_POST_INSTANCE_BE_memory=512

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also replace explicit loop with dict.update.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>