Instead of being given the priority for acquiring locks by means of a
parameter, mcpu will now call back. This is in preparation for
implementing a command to change a job's priority on the fly and allows
to change it while locks are being acquired (taking effect on the next
lock acquire).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

A newly added test for RAPI will also verify the returned headers. A
test in ganeti.rapi.client_unittest.py is split into smaller stand-alone
tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit f0d22861 changed the logic of
gnt_instance._ConvertNicDiskModifications to also allow a parameter
named “modify”. Unfortunately the corresponding unittest was not
updated. An “if”/“else” condition is also merged.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This replaces direct calls to “os.access” and
“os.path.exists”/“os.path.isfile”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Up until now “IsBelowDir("/", …)” would never return True. The reason
was that an additional slash was added to the root path resulting in
“//", which is “implementation-defined” in posix and treated specially
by “os.path.normpath”.

This patch fixes the behaviour for this special case and adds tests
(also for IsNormAbsPath). A typo in the docstring is fixed. Calls to
“assert_” and “assertFalse” are changed to pass a message by keyword
argument.

It is a bit of a mess, but I hope the resulting behaviour is correct.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dato Simó <dato@google.com>

When a job is still waiting for locks and the queue is shutting down,
they should be returned and not actually start processing. Until now
jobs which transitioned from “queued” to “waiting” were already
considered to be running as far as the shutdown code was concerned.

This fixes issue 296.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 52c47e4e (July 2010) added the exact test twice, probably due to
a copy & paste error.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When file storage is used this file is now mandatory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With the move away from “setup-ssh”, Paramiko is no longer necessary to
configure SSH on nodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This new function returns the file paths for all of a user's SSH-related
files (RSA, DSA and authorized_keys).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch changes “utils.RunCmd” to accept a file-like object or a
numeric file descriptor which will be used as the command's standard
input. One use-case will be to pass all necessary data to
“prepare-node-join”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Other places, such as “ssh.GetUserFiles”, use a structure where the
private key comes before the private key. Until now prepare-node-join
did the opposite, that is the public key came first. To avoid confusion
and potential bugs, this is changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A public key already includes the necessary prefix (“ssh-rsa” or
“ssh-dss”), so there is no need to add it again.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Without this parameter, either an error would be raised or “.ssh” would
have to be created. Now it is possible to retrieve the paths without
requiring the “.ssh” directory to exist.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Note that this commit has no Makefile.am changes, as the files were
not actually used. So it's better to actually remove them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

An earlier version of this patch series verified all paths in cmdlib in
the master daemon. With this change all that verification code is moved
to bdev to run inside the node daemon. The checks are much stricter
now--it is no longer possible to use forbidden paths (e.g. /bin) to
manipulate file storage devices (once these checks are being used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is a new tool as per the design document “design-ssh-setup”. It
receives a JSON data structure on its standard input and configures the
SSH daemon and root's SSH keys accordingly. Unit tests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch changes “ssh.GetUserFiles” to support two different kinds of
SSH keys, RSA and DSA. Before it would always use DSA. Newly written
unit tests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For simple SSH keys, that is those without options such as
“command="…"”, only the first two parts need to be compared. The third
field is a free-form comment.

This patch changes the comparison used in
AddAuthorizedKey/RemoveAuthorizedKey to take this into account. Lines
with options are still compared in full.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There might be more, but at least replace all these low-hanging fruits.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This commit adds a Group.allTags field to store the tags of node groups,
and teaches each loader backend in HTools to populate it (additionally, the
IAllocator class in lib/cmdlib.py now includes tags for groups too). Test
data is updated to include an empty set of tags for node groups in all
affected test cases.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In preparation for future modifications in the exclusion tags field, add a
test that verifies that exclusion tags are being honored: in a test cluster
with two instances of the same exclusion group in each node, hbal should
shuffle instances around to improve the score.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

New parameters, “--pending”, “--queued” and “--waiting”, are added to
select all jobs in the respective state. If one of those options is used
and “--force” is not given, the user is asked to confirm the operation.
Unit tests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There is no need for the “_IsUniqueSequence” function anymore, it can
easily be replaced by utils.FindDuplicates. Also, pass the message as a
keyword parameter and use the more commonly used assert* functions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This constant contains the job status' “queued”, “waiting” and
“cancelled”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

/etc/hosts is a bit special as it's a system-wide file and the virtual
cluster/node root doesn't apply. The modification of /etc/hosts should
be disabled in virtual clusters. If it isn't, however, the vcluster
functions would raise an exception complaining about a path outside of
the virtual node root. This patch adds a whitelist to exclude
/etc/hosts.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

All data kinds (used to restrict the data collected) referenced in a
filter can be requested once it's been “compiled”. However, the kinds
of fields used in boolean expressions (e.g. ["?", "xyz"]) were not
recorded. This patch changes the code accordingly and provides a unit
test update.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This attribute is set to True for jobs which were restored from an
archived file. A new filter will act on this field.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds some unit tests for “gnt-cluster epo”. Not everything is
covered, but at least the bug fixed in the previous patch is.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

An error like “invalid literal for int() with base 10” can be quite
confusing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As I wrote during/after the review on commit 2958c56e, “ganeti-cleaner:
Separate queue cleaning code”, while I appreciated the permission
separation, I didn't like too much the file-based approach:

- it is a very simple script, and lots of the code is duplicated
  between the two; I wouldn't like to see "ganeti-vmcapable-cleaner",
  "ganeti-master-candidate-cleaner", etc. in the future
- ganeti-master-cleaner "pollutes" the namespace, creating
  tab-completion conflicts with ganeti-masterd

This patch simply merges the master-cleaner back into cleaner, while
keeping the separate user permissions scheme, separate log files, etc.

Additionally, it fixes two bugs in the unit-test (not run with set -u
and wrong path in the master-cleaner log files test; yay for even
worse safety than Python?).

And finally, since we have now support for --help-completion, it adds
bash completion support for this script :) (needs to be applied on top
of my argument support patch series).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now "gnt-instance recreate-disks" uses the default iallocator when "." is
specified as the iallocator. For uniformity, the same behavior applies to
these commands:
  gnt-node evacuate
  gnt-instance migrate
  gnt-instance add
"." is used instead of "default" becuse the latter could be a valid name for
an iallocator.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- LoadAllowedFileStoragePaths: Loads a list of allowed file storage
  paths from a file
- CheckFileStoragePath: Checks a path against the list of allowed paths

The unit test for “utils.IsBelowDir” is updated with cases which weren't
tested before.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We don't use generators often and lists are easier to re-use.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

- Don't hardcode node name in some places
- Don't define functions inside functions
- Simplify code for testing with and without offset, this is now in two
  separate tests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This patch adds code to wipe newly added disk space when growing disks
using “gnt-instance grow-disk”. “New disk space” is defined as the delta
between the old block device size (not necessarily equal to the amount
recorded in the configuration) and the new recorded size. Extra caution
is taken to avoid overwriting existing data. Unit tests are included.

A small typo in gnt-cluster(8) (“it's”) is also fixed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is in preparation for adding disk wipe on growing disks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

I somehow forgot this in the previous patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This patch is somewhat longer than was anticipated. Before, commands like
“gnt-instance --help” would exit with the status code 1 (failure). With
this patch, those commands exits with 0 (success) while unknown commands
still print the usage screen and exit with status 1.

To achieve this goal without too much ugly code some refactoring on
cli._ParseArgs was necessary. Printing the version and usage screen was
separated. At the same time this allows for some unit testing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This code does not need to run as root, therefore it's better to split
it out. It is now run with the same permissions as the master daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>