This commit adds a Group.allTags field to store the tags of node groups,
and teaches each loader backend in HTools to populate it (additionally, the
IAllocator class in lib/cmdlib.py now includes tags for groups too). Test
data is updated to include an empty set of tags for node groups in all
affected test cases.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This takes care of a FIXME; 2.6 already uses the new method, so we're
good during upgrades.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Per a conversation on ganeti@googlegroups.com:

  - gnt-os diagnose ; gnt-os list take in consideration blank lines in
   /etc/ganeti/instance-image/variants.list that could be confusing.

Let's fix this and also let's ignore comment lines.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

New parameters, “--pending”, “--queued” and “--waiting”, are added to
select all jobs in the respective state. If one of those options is used
and “--force” is not given, the user is asked to confirm the operation.
Unit tests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This constant contains the job status' “queued”, “waiting” and
“cancelled”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A socket shouldn't have its executable bit set.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also fix one small mistake in the docstring for QuitGanetiException.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds support for Xen's CPU scheduler 'cpu_cap' and
'cpu_weight' parameters.

Ganeti default values (cap: 0=unlimited, weight: 256) are Xen defaults.

cpu_cap is not validated correctly because of actual Ganeti limitation
on checks. I'd need check methods to get beparams or vcpus to get moved
to hvparams.

Signed-off-by: Sébastien Bocahu <sebastien.bocahu@nuxit.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The check for file consistency didn't properly handle virtual paths
in case of a virtual cluster. This didn't cause any breakage as in
a standard virtual cluster setup with only one node all files are
visible for every node.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This switches gnt-node/gnt-group (and their equivalent RAPI resources)
to go over the query socket.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

/etc/hosts is a bit special as it's a system-wide file and the virtual
cluster/node root doesn't apply. The modification of /etc/hosts should
be disabled in virtual clusters. If it isn't, however, the vcluster
functions would raise an exception complaining about a path outside of
the virtual node root. This patch adds a whitelist to exclude
/etc/hosts.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There is a constant for this purpose.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Needed for coming patches.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This provides a convenience option to include archived jobs in the
output list. It's equivalent to using “-o +archived”, but tab completion
is nicer.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If requested via a filter or by including the “archived” output,
archived jobs will be loaded and shown. This is significantly slower
than just listing normal jobs, therefore by default they are not loaded
at all.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

All data kinds (used to restrict the data collected) referenced in a
filter can be requested once it's been “compiled”. However, the kinds
of fields used in boolean expressions (e.g. ["?", "xyz"]) were not
recorded. This patch changes the code accordingly and provides a unit
test update.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This attribute is set to True for jobs which were restored from an
archived file. A new filter will act on this field.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The description was not accurate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Example output:
$ gnt-cluster verify-disks
Submitted jobs 4327
Waiting for job 4327 ...
No disks need to be activated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds some unit tests for “gnt-cluster epo”. Not everything is
covered, but at least the bug fixed in the previous patch is.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The usage of that variable was removed in 45a36f36, but accidentally
the enumerate() was left in.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Updating the “node_query_list” variable would fail if no arguments were
passed and the “--all” option wasn't specified. A follow-up patch will
add unit tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This exception is documented to have three arguments, but in one case
we raise it with a simple string argument. Let's fix that.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It seems a few of the error classes are no longer used:

- LVMError, deprecated in 8c5533a5 (before ganeti 1.2.2!)
- ConfdRequestError, deprecated in b0dcdc10
- SshKeyError, introduced in the initial open source commit but never used (⁈)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

utils.IsBelowDir is actually tested and doesn't allow writes to
“…/queue*”, like the old code here did.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

An error like “invalid literal for int() with base 10” can be quite
confusing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

My local pylint didn't complain.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The man page and the bultin-in help for gnt-instance replace-disks were
inconsistent. Also fixed some typos in man pages.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This makes differences show up in “gnt-cluster verify”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

"gnt-instance replace-disks" now behaves like the other commands, and uses
the default iallocator when "." is passed as the iallocator parameter.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now "gnt-instance recreate-disks" uses the default iallocator when "." is
specified as the iallocator. For uniformity, the same behavior applies to
these commands:
  gnt-node evacuate
  gnt-instance migrate
  gnt-instance add
"." is used instead of "default" becuse the latter could be a valid name for
an iallocator.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- LoadAllowedFileStoragePaths: Loads a list of allowed file storage
  paths from a file
- CheckFileStoragePath: Checks a path against the list of allowed paths

The unit test for “utils.IsBelowDir” is updated with cases which weren't
tested before.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

First: This enables the use of “gnt-job watch $id” for archived jobs.

Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit 04569469)

First: This enables the use of “gnt-job watch $id” for archived jobs.

Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If normal file storage was disabled but shared storage enabled,
“_TransformFileStorageDir” would still throw an exception.

in “opcodes._CheckStorageType” there's also a check, but I wasn't quite
sure what the correct way of handling it was, so I added a TODO comment.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We don't use generators often and lists are easier to re-use.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This patch adds code to wipe newly added disk space when growing disks
using “gnt-instance grow-disk”. “New disk space” is defined as the delta
between the old block device size (not necessarily equal to the amount
recorded in the configuration) and the new recorded size. Extra caution
is taken to avoid overwriting existing data. Unit tests are included.

A small typo in gnt-cluster(8) (“it's”) is also fixed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- LogInfo takes *args, no need to replace values right away
- Don't overwrite wipe_chunk_size right after it's been set

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This will also be used for verifying the file storage directory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>