The first dot files we added way back in the 2.0 release are managed
manually: we build the .pngs ourselves, track them in .gitingore,
etc. Since then, we have switched to allow sphinx to build/manage them
itself, so it makes sense to cleanup the build system and covert
everything to this simpler method.

This patch simply converts from "image: x.png" to "graphviz: x.dot",
as an externally-referenced dot file, and removes all the custom
makefile rules for png conversion. The next patch will bring the
conversion further (by inlining then).

Additionally, while checking that the patch doesn't change the html
output, I saw that the use of "shape=diamond" with non-trivial labels
results in very ugly diagrams; so we split the question in such
decision boxes to make them more reasonably looking.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch makes the gnt-cluster and gnt-node man page more consistent
with the other man pages, where the tag commands are listed in a
separate section at the end.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This makes the (very dull) html version of the man pages a bit more
readable, until such a time that we can switch over to sphinx fully.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

I don't remember why I introduced these when switching to pandoc, but
they are not absolutely needed. Instead of pre-generating them (which
might save some trivial cpu time), we can simply pass footer.rst as an
additional input file to pandoc.

The advantage in doing so is that semantic information is preserved;
when pre-generating and including them as-is, the entire structure is
lost, as pandoc won't reinterpret them, so things like a potential
table of contents will simply ignore the footer.

The patch also adds a new target to rebuild all man pages (both groff
and html formats), for testing changes to the pages easier.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The hinotify library was missing any versions, and finding one that
works with ghc 6.12 (base 4.2.x) was not obvious. The
test-framework-quickcheck2 recommended version didn't actually work
with the test-framework version, so fix that as well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As Constantinos observed, the gnt-os man page is not consistent with the
others. This patch adds the missing sections.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

"Tags" is a section, not a command, so let's title-case it (as opposed
to all upper-case). Furthermore, the RST markup was wrong for the
gnt-network man page.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The "list-fields" and "change-group" commands were mistakenly listed
as sections; we also rename the "Recovery" section to match better its
content.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Completion information for multi-command binaries were not generated in
the correct way, so the build-bash-complation script failed when such a
binary was actually found.

This commit fixes this issue, and a related docstring typo as well.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When working on documentation it can be helpful to use a browser. In
some environments it's not possible to access the files directly from a
graphical browser. This trivial webserver exports all files in the
current directory and unlike alternatives such as thttpd, doesn't check
for world-readable permissions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

When the man pages' reST is built using Sphinx, “bash” must be used
instead of “Bash”. Pandoc works with both.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was extracted from a patch by Dimitris Aragiorgis with the subject
“Add locking to _NetworkQuery”. The rest of the patch did no longer
apply and will require more work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit acaf3ca7 missed this space before a backslash.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The mon-collector binary was erroneously installed twice: once in
/usr/bin/ and once in /usr/lib/ganeti/.
With this modification, it is now installed in /usr/lib/ganeti/ only,
as intended.

In order to do this in a proper way, some parts of the Makefile.am where
refactored.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If a received reply is valid and there is still no valid reply,
keep it, no matter how new it is.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Small updates to the extstorage design document and interface.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This makes the man page more consistent with the others (except
gnt-os, that is).

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While trying to understand why some code was not being tested, I
realised that we have a bad data type in Hcheck.hs.

We have "data Level = GroupLvl | ClusterLvl", but then we need to pass
the group name/index as well, so we have functions that look like the
following:

  f :: Level -> Maybe String -> …
  f GroupLvl (Just gname) = …
  f GroupLvl Nothing = … -- never used
  f ClusterLvl _ = … -- wrong, since a (Just x) could be passed

It's clear that we just need to move the group name in the "GroupLvl"
constructor, and simplify the call chain.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in
the future).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- gen-coverage → gen-py-coverage
- coverage-htools.tix → coverage-hs.tix

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, both the unit test and the shell tests are run (manually)
from this rule. This has two downsides: you can't (compile and) run
just one of them easily, and make can't run them in parallel.

By splitting the rules, we can both of the above easily.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch unifies the functions that generated the tree
reorganisation discussion in the first place :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The current base name of 'test' means that this binary will generate
profiling, coverage, etc. files with the name 'test.*', result in
tab-completion conflicts with the test directory. Let's slightly
change its base name for typing easiness.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

htest/data becomes test/data/htools (basically reverting commit
8feabc89), and htest/* becomes test/hs/*.

Most changes beside the rename are trivial s/…/…, with the exception
of autotools/run-in-tempdir, which needed some more changes now that
test/ is not just Python files:

- test/py is still being copied
- test/hs moves from individual symlinks to entire dir symlink
- test/data is symlinked in its entirety

Checked with make distcheck, pep8 and pylint, so at least VPATH builds
are OK.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is the first step of the test files reorganisation: moving test/*
(except test/data) to new directory test/py/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently only the exit code is returned, which is somewhat pointless:
for failures, we already know it's ≠ 0, so we don't have enough
information (reported on the mailing list). By logging the output as
well (if any), the user will have more information available for
debugging.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This continues the cleanup of the 'htools' name, by replacing 'htools'
with 'hs' in configure and makefile rules.

There is still 'WANT_HTOOLS' left as it is correct, but some of the
conditionals in Makefile.am will need some cleanup (in a different
patch).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Recent iallocator changes (commit fb60bc6a, “iallocator: Add node
whitelist”) broke the parameter validation for LUInstanceRecreateDisks
(the other LUs were manually fixed already):

  $ gnt-instance recreate-disks -I hail --dry-run instance3
  Failure: prerequisites not met for this operation:
  error type: wrong_input, error details:
  Request is missing 'node_whitelist' parameter

By just defining the new parameter, the LU works normally (“INFO:
Selected nodes for instance instance3 via iallocator hail: node2”).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since we call into Python for opcodes and other type compatibility
checks, we need to ensure that the python sources are
built. Otherwise, full parallel make might or might not break,
depending on timing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

I made a mistake when adding support for htools to run-in-tempdir, by
needlessly copying the htools binaries, when it's enough to link
them. The copying is only really needed for python code, so we change
the Haskell binaries to linking and also a few other directories.

Difference:
- before: 0.8s, 95MB copied
- after: 0.6s, 6.2MB copied

And, by the way, can't wait for Python 2.6 as minimum supported
version…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unless this is run under run-in-tempdir, it will create as usual the
.pyc files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

 * ganeti-extstorage-interface man page
 * gnt-storage man page

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.

For now, only two commands are supported: 'info' and 'diagnose'.

'diagnose' calculates the node status of each provider on each node,
similarly to gnt-os diagnose. Furthermore, for every provider, it
calculates it's nodegroup validity for each nodegroup. This is done
inside the LU and not the client (marked as 'TODO' for the  global
validity of gnt-os diagnose).

In the future, gnt-storage can be used to manage storage pools,
or even be extended to diagnose other storage types supported by
Ganeti, such as lvm, drbd (INT_MIRROR) or rbd (EXT_MIRROR).

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed Haskell compatibility and style fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

This adds a few new hcheck tests, to extend the coverage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

After this patch, almost all of hbal except the actual Luxi execution
is being covered by unit- or shelltests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* devel-2.6:
  Final update of NEWS file for 2.6.2 and version bump
  Fix job completion with big job queues
  confd: reduce noise during normal config reload
  Change hbal handling of !auto_balance instances
  Small corrections in man pages
  Fix gnt-instance(8) w.r.t. growable disk templates
  Stop verifying opcode results in dry_run mode
  Update NEWS for 2.6.2
  Fix TypeError when unsetting OS parameters
  cmdlib.py: don't use GetHypervisor

Conflicts:
        NEWS (trivial, integrated the 2.6.2 news)
        autotools/build-bash-completion (trivial: pathutils)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This fixes:

htest/Test/Ganeti/Rpc.hs:50:14:
    Couldn't match expected type `Rpc.RpcCallNodeInfo'
           against inferred type `Data.Map.Map String Bool
                                  -> Rpc.RpcCallNodeInfo'
    In the first argument of `(<$>)', namely `Rpc.RpcCallNodeInfo'
    In the first argument of `(<*>)', namely
        `Rpc.RpcCallNodeInfo <$> arbitrary'

by adding the new parameter (empty map for now), and also:

ganeti/constants.py:432:5: E123 closing bracket does not match
   indentation of opening bracket's line

By switching to 2-char indentation and (for consistency) also replaces
frozenset(…) with our unique checker (UniqueFrozenset).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

No conflicts, really trivial.

* stable-2.6:
  Final update of NEWS file for 2.6.2 and version bump
  Fix job completion with big job queues
  confd: reduce noise during normal config reload
  Change hbal handling of !auto_balance instances
  Small corrections in man pages
  Fix gnt-instance(8) w.r.t. growable disk templates
  Stop verifying opcode results in dry_run mode
  Update NEWS for 2.6.2
  Fix TypeError when unsetting OS parameters
  cmdlib.py: don't use GetHypervisor

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This happens when the exclusive_storage flag is set, and the requested
operation is incompatible with the flag or not yet supported.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>