htest/data becomes test/data/htools (basically reverting commit
8feabc89), and htest/* becomes test/hs/*.

Most changes beside the rename are trivial s/…/…, with the exception
of autotools/run-in-tempdir, which needed some more changes now that
test/ is not just Python files:

- test/py is still being copied
- test/hs moves from individual symlinks to entire dir symlink
- test/data is symlinked in its entirety

Checked with make distcheck, pep8 and pylint, so at least VPATH builds
are OK.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is the first step of the test files reorganisation: moving test/*
(except test/data) to new directory test/py/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently only the exit code is returned, which is somewhat pointless:
for failures, we already know it's ≠ 0, so we don't have enough
information (reported on the mailing list). By logging the output as
well (if any), the user will have more information available for
debugging.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This continues the cleanup of the 'htools' name, by replacing 'htools'
with 'hs' in configure and makefile rules.

There is still 'WANT_HTOOLS' left as it is correct, but some of the
conditionals in Makefile.am will need some cleanup (in a different
patch).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Recent iallocator changes (commit fb60bc6a, “iallocator: Add node
whitelist”) broke the parameter validation for LUInstanceRecreateDisks
(the other LUs were manually fixed already):

  $ gnt-instance recreate-disks -I hail --dry-run instance3
  Failure: prerequisites not met for this operation:
  error type: wrong_input, error details:
  Request is missing 'node_whitelist' parameter

By just defining the new parameter, the LU works normally (“INFO:
Selected nodes for instance instance3 via iallocator hail: node2”).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since we call into Python for opcodes and other type compatibility
checks, we need to ensure that the python sources are
built. Otherwise, full parallel make might or might not break,
depending on timing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

I made a mistake when adding support for htools to run-in-tempdir, by
needlessly copying the htools binaries, when it's enough to link
them. The copying is only really needed for python code, so we change
the Haskell binaries to linking and also a few other directories.

Difference:
- before: 0.8s, 95MB copied
- after: 0.6s, 6.2MB copied

And, by the way, can't wait for Python 2.6 as minimum supported
version…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unless this is run under run-in-tempdir, it will create as usual the
.pyc files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

 * ganeti-extstorage-interface man page
 * gnt-storage man page

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.

For now, only two commands are supported: 'info' and 'diagnose'.

'diagnose' calculates the node status of each provider on each node,
similarly to gnt-os diagnose. Furthermore, for every provider, it
calculates it's nodegroup validity for each nodegroup. This is done
inside the LU and not the client (marked as 'TODO' for the  global
validity of gnt-os diagnose).

In the future, gnt-storage can be used to manage storage pools,
or even be extended to diagnose other storage types supported by
Ganeti, such as lvm, drbd (INT_MIRROR) or rbd (EXT_MIRROR).

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed Haskell compatibility and style fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

This adds a few new hcheck tests, to extend the coverage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

After this patch, almost all of hbal except the actual Luxi execution
is being covered by unit- or shelltests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* devel-2.6:
  Final update of NEWS file for 2.6.2 and version bump
  Fix job completion with big job queues
  confd: reduce noise during normal config reload
  Change hbal handling of !auto_balance instances
  Small corrections in man pages
  Fix gnt-instance(8) w.r.t. growable disk templates
  Stop verifying opcode results in dry_run mode
  Update NEWS for 2.6.2
  Fix TypeError when unsetting OS parameters
  cmdlib.py: don't use GetHypervisor

Conflicts:
        NEWS (trivial, integrated the 2.6.2 news)
        autotools/build-bash-completion (trivial: pathutils)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This fixes:

htest/Test/Ganeti/Rpc.hs:50:14:
    Couldn't match expected type `Rpc.RpcCallNodeInfo'
           against inferred type `Data.Map.Map String Bool
                                  -> Rpc.RpcCallNodeInfo'
    In the first argument of `(<$>)', namely `Rpc.RpcCallNodeInfo'
    In the first argument of `(<*>)', namely
        `Rpc.RpcCallNodeInfo <$> arbitrary'

by adding the new parameter (empty map for now), and also:

ganeti/constants.py:432:5: E123 closing bracket does not match
   indentation of opening bracket's line

By switching to 2-char indentation and (for consistency) also replaces
frozenset(…) with our unique checker (UniqueFrozenset).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

No conflicts, really trivial.

* stable-2.6:
  Final update of NEWS file for 2.6.2 and version bump
  Fix job completion with big job queues
  confd: reduce noise during normal config reload
  Change hbal handling of !auto_balance instances
  Small corrections in man pages
  Fix gnt-instance(8) w.r.t. growable disk templates
  Stop verifying opcode results in dry_run mode
  Update NEWS for 2.6.2
  Fix TypeError when unsetting OS parameters
  cmdlib.py: don't use GetHypervisor

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This happens when the exclusive_storage flag is set, and the requested
operation is incompatible with the flag or not yet supported.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

All the checks have been moved to a private method, so as to make
CheckPrereq() smaller and more readable (pylint was complaining about it).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When exclusive_storage is set, nodes report only the space that could be
used to create new disk according to the rules set in the
design-partitioned doc (i.e., non-empty disks are ignored and the space
reserved for rounding/overhead is not reported).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The flag is read from the configuration and passed to the RPC.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The flag is passed to the low-level functions that need it, but it's not
yet used. Also, RPCs get just a default value for now.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Requested in issue 337.

The parameter “bridge” was not documented and is therefore silently
replaced with “master-netdev”. A note is added to “qa-sample.json”
describing how comments work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit ec0652ad (June 2009) removed the option.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For people writing query filters (documented in ganeti(7)) knowing a field's
type can be useful.

$ gnt-instance list-fields name be/memory
Name      Type         Title        Description
name      Text         Instance     Instance name
be/memory Storage size ConfigMaxMem The "maxmem" backend parameter

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Of course I forgot to add this to TEST_FILES… and only distcheck
catches it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In issue 240 this assertion fails. While reproducing was not possible
with the master branch, having a more verbose message can't hurt.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Hiding the “connect” and “disconnect” commands below the tag operations
isn't good. The same is done in “gnt-instance.rst”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit a04cc7a4 added “gnt-network.rst”, but didn't copy the editor
settings from another reST file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

When exclusive_storage is set, PVs are allocated according to the
design-partitioned design doc.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The flag is read from the configuration and passed to the RPC.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The flag is passed to the low-level functions that need it, but it's not
yet used.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This makes the code more readable and easier to upgrade.
bdev.LogicalVolume.GetPVInfo and the code that depends on it have been
refactored to use the new class.

utils.CheckVolumeGroupSize() has been moved to lib/utils/lvm.py, where more
functions will be added.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently only --help and --version are tested; the patch extends this
with testing parsing of an empty cluster, of a cluster with one group,
and other command line behaviours.

It also adds testing for --help-completion for all htools binaries.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds an ignore for an hlint warning that wants camelcase to be used
in the name of a "case" test.
But if camelcase is used, it is not a valid test name anymore.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also add a missing parenthesis at the end.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The original implementation reserved the name “all” to specify all node
groups. That would prevent the use of a node group actually named “all”.

This patch changes the order and definition of the command line
arguments for “gnt-network connect” and “… disconnect” to have the
group(s) as the last argument(s). If no groups are specified all groups
are used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It is not implemented after all.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Use unpacking instead of addressing indivdual list items
- Add “--submit” and “--priority” support where possible
- Re-use LUXI client where possible
- Fixed some typos

There is still an issue that a node group named “all” can not be
connected or disconnected. More changes are needed for that.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Similar to commit 572a277b. Directly iterate over “self.wanted” instead
of iterating through all networks and checking if they're wanted.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>