Currently, this tests and its helper function 'isAlmostEqual' uses
plain booleans to signify failures, which means you can't really debug
a failed test. The patch changes the call chain to use annotated
properties all through, which results in messages like:

  Failing almost equal check
  Delta 3.725290298461914e-9 not smaller than 1e-12
  expected: 2.147785408767952e7
   but got: 2.1477854087679517e7

which (IMHO) it's much more readable.

The patch also replaces a 'fail' with 'failTest' in another test.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This implements QffHostname and fixes the node listing (as well as
export listing when filtering on node name).

This bug was hidden by the fact that node listing with "gnt-node list
aa" works if you don't have live queries (as it was originally), as
the choosing of wanted nodes out of the config based on short names
works. What didn't work was later post-filtering based on such short
names (kind of duplicate, but that's how the code path is).

By implementing QffHostname, we can have custom equality checks, like
in the Python code. What I don't like is how convoluted the testing on
various left/right combinations is, but I didn't find an easier way.

The included unittest tests the partial filtering behaviour, and fails
if the node name flag is not set to QffHostname.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since we use the primitive string type for group UUIDs, the group
fields have a bug where we pass the group name as filter for node
tests, whereas the nodes themselves use the group UUID. This results
in zero node count, empty node list, and no instances being reported
as assigned to groups.

The patch fixes this and adds a test for the node count. It does some
test generation improvement, and also cleans up whitespace issues in
Test/G/Q/Query.hs (the functions case_queryNode_allfields,
prop_queryGroup_noUnknown and case_queryGroup_allfields are unchanged
but simply have indentation fixed).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

After implementing a few of the query executor functions, it turns out
that we have the same general pattern:

- compile the filter
- extract the selected fields
- determine whether we need to run collectors
- do a first pass filtering
- run the collector if needed
- compute the final fields

For pure config queries, the collector/final computation is not
needed, but otherwise the code flow is the same.

We can therefore abstract all the queries that originate in the config
(i.e. except the job query) and have a single code path for all of
them, just with different parameters.

To do this, we add some stub collectors for group/network queries,
which don't have live data.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The fields are not yet complete, but at least we can enable the
list-field query to see what is there already.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Following the new naming style introduced in Exports.hs, this patch
renames the other resources to export non-qualified names (fieldMap as
opposed to nodeFieldMap), and to use qualified module imports.

Also fixes a haddock issue in a docstring.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is an initial implementation of diskless instances.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If a value was set to numeric 0, the parameter wouldn't be passed to the
command.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This makes it easier to know what options are available.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Make it match the style guide.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If “instance-add-drbd-disk” is not enabled it usually also means to not
use DRBD at all.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This was meant to be since "the beginning" but was never submitted as
somehow it failed with python 2.4. Now that the minimum python version
has been increased it can be added.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The QA script connects to the RAPI daemon. In virtual cluster setups the
daemon is usually listening on a local interface not reachable from the
outside.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The path to the queue drain file must use the virtual node path.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This test will need some more work for virtual clusters. Disable it in
the meantime.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

They do not work properly at the moment.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The file is located below the virtual node directory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The virtual node prefix must be added to the path.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The OS definitions are not virtualized. Tests configuring different OS
definitions on different nodes don't work properly.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Use path of virtual node when retrieving ssconf's instance list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Use virtual node path when removing the watcher status file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

“qa_utils.BackupFile” is always called with the path for a normal node.
In virtual clusters the node prefix path must be added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

In virtual clusters, all “gnt-*” and “ganeti-*” commands require
environment variables to be set for the virtual cluster root directory
and node name. The “cmd” script generated by “vcluster-setup” is a
wrapper setting those variables.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Until now all tests depended on simple AND/OR conditions. With the
introduction of virtual cluster support, some tests can only run if
virtual clusters are not in use (e.g. not yet supported or not
possible). Instead of introducing more logic for enabling/disabling
tests, callable values are supported instead. “ht.WithDesc” should be
used on them to continue to provide a readable description.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If these new options are set, a virtual cluster is created and used.
Virtual clusters consist of 1..N virtual nodes on a single physical (or
virtual) node. The design of virtual clusters is described in
“doc/design-virtual-clusters.rst”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Reviewed-by: Michele Tartara <mtartara@google.com>

Many resources, but not all, list their supported methods. With a new
reST directive, this list is now automatically generated and also lists
the required permissions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Accept the -G option, and if it's passed require that it matches a
nodegroup, then only output nodes belonging to that group.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

To make the graphs work even when instances live on offline nodes (eg.
because we're offlining them just to exclude them, or because they have
instance still on them) we just filter them out at the end, when we're
going to print out the result.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We have no way to test Python 2.4 compatibility anymore, and that
version is long deprecated. From version 2.8+ require Python 2.6 or
above (but still not 3.0).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In some situations after restarting the SSH daemon by
“prepare-node-join” it's not immediately reachable. Like with the master
and node daemons, waiting for it to become reachable instead of failing
hard can help in such situations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Log every attempt to reach a port on another node. Don't silently return
False if a target not recognized as an IP address is given (e.g. a
hostname).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In issue 353 Sascha Lucas reported that disks are not wiped when added
through “gnt-instance modify”. This patch adds this functionality and
updates the docstring for “_WipeDisks”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This directory is required by “cfgupgrade” and some other tools.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Moving the SSH/SCP constants around was necessary for this.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

- Commit 4a90bd4f contained a rather large number of style violations:
  - Indentation/sequence formatting
  - Wrapping of long lines
  - Docstrings whose first line was wrapped
  - A stray backslash in a docstring
- opcodes: Don't check for list or tuple. None of the other opcodes does
  it explicitely. As long as the length and the items match the value is
  accepted.
- server/noded: “if variable” doesn't test for None, but False

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Until now many resources, but not all and not consistently list their
supported methods (e.g. “Supports the following commands: ``GET`` …”).
Not only is it easy for this list to get out of date, but it would also
be nice if it listed the required access permissions.

This patch adds a new directive to insert a table listing all supported
methods and their access permissions for a resource.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The list of resources and handlers will also be used for generating
a per-resource table.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>