This patch adds a mock RAPI client to test input values to methods. All
methods either raise an exception if there was a problem or return None.
Third-party code can use this to test their input values to the RAPI
client.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this patch headers are constructed from the PycURL options
and passed to the mock implementation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is preparation for the mock system, where we need the same cURL
mock.

Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This solves one case where locks are acquired during LUXI queries.
Pretty late into the transition I noticed that OpBackupQuery had a
“use_locking” parameter for a long time, but didn't use it. Since
most of the other changes were already and this allows exports to
be listed via RAPI (/2/query) I decided to finish.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Before this patch, a node evacuation submitted with high priority would
only compute the solution at that priority, but the actual evacuation
ran at normal priority.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

I missed these among some shelltest-related failures.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This rather inefficient implementation (fields are evaluated on every
call to GetInfo) is not good for WaitForJobChanges and doesn't support
filters, but that will be rectified in later patches.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Jobs don't have a “name” field, so we must be able to control
the field used for simple filters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

… to some degree at least. Unittests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also include unittest for LOCK_ATTEMPTS_TIMEOUT.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch moves the “call_version” to a new RPC client definition and
then adds a new runner using the DNS resolver for getting the host
address.

The standard “BootstrapRunner”, where the call was before, tries to
resolve node names using ssconf first, which doesn't work properly when
re-adding a node with a new primary IP address.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While testing some other stuff, I realised that the gnt-* commands
could be broken (as in, the script fails with syntax errors), but make
check doesn't detect it. Since we have shelltest, we can now add
trivial tests for this case.

One downside is that starting the scripts seems to be much slower
than the htools binaries, so we can't add as many tests.

The other downside is that shelltest is now required for all
development work, but I think this is a small disadvantage compared to
the increased testing possibilities.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

While debugging another issue we realized that LUClusterQuery forks.
This turned out to be the “platform.architecture” function from the
Python library. It uses the “file” command to determine the architecture
of the Python binary.

This patch adds two new functions to the “runtime” module to get this
information once per process instead of doing it every single time
LUClusterQuery is used. Forking is a no-go in a multi-threaded
environment anyway.

A future change will also have to change the terminology in “gnt-cluster
info”: it reports the binary architecture simply as “architecture”, when
it's actually the binaries' architecture. Kernel and userland can be
different.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This is more of a RFC… Basically most of the shell-based tests are
converted from exec+grep to shelltestrunner.

Things are not all fine and nice though:

- we have dependencies between tests, as some generate some data files
  needed later; this is not nice, and we depend on serial execution in
  testrunner
- we can still fail with no so nice messages in the offline-test
  script (when we generate most of the data)

But overall, I think the tests are much nicer to
define/read/debug:

- each test is standalone, with the only dependency being an optional
  input data file; this is much better than a single monolithic shell
  script
- in case of failures, the failure is clearly shown by shell test,
  both for exit code and stdout/stderr
- shelltest can run in --debug mode, where the exact details are shown
  much better than the alternative of "set -x" for the shell script

Comments welcome!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was already a TODO since the implementation of lock priorities in
September 2010. Under certain conditions a waiting acquire can be
notified at a time when it can't actually get the lock. In this case it
would try and fail to acquire the lock and then return to the caller
before the timeout ends.

While this is not bad (nothing breaks), it isn't nice either. A separate
patch will prevent unnecessary notifications when shared locks are
released.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

While working on another SharedLock fix I realized timeouts on lock
deletion don't work very well if the timeout actually expires. This
patch fixes the issue and adds a new unittest.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

(cherry picked from commit f8326fca)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Using some hand-crafted files (manually modified from a real cluster
running master branch), we test that hail behaves as we expect:
doesn't load obvious wrong data, can do relocations/allocations, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)
have fixed http://bugs.python.org/issue12856 which we worked around
ourselves.

This means two things:

- we don't need to manually reset the module
- we can't test for the no-reset case

Unfortunately current Debian Sid has the 2.7.3 RC but still labeled as
2.7.2+ ☹️

, so we have to manually do an extra check (in the unit test
only); I expect Debian will update to official 2.7.3 as soon as it's
released, and then we can remove this override.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

On my workstation, if I run the following two tests in this exact
order:

- TestSingleFileEventHandler.testError
- TestSingleFileEventHandler.testReplace

the second test will fail, because there is no "unregister" of
previous tests (and the poll() done by asyncore will fire on the
previous files which are now removed, etc. etc.).

While investigating this, I realised that we don't do any cleanup of
the various handlers/notifiers we setup for this test, so if we simply
add a proper cleanup, the tests will work nicely (in any order :).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Also change the meaning of group ndparams to mean the actual (not
custom) nd params, and add custom_ndparams for the current meaning of
ndparams.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently this is not handled by Ganeti, just recorded.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Only whitelisted opcodes may not have a result check.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This check verifies “iv_name” of all instance disks. If one is wrong
(which shouldn't happen in the first place), cluster verification will
warn:

“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:
name of disk 1 should be 'disk/1', but is 'disk/4'”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

To remove the last disk, or to add a NIC to the end, one can use the
index -1. This wouldn't work as intended as “-” is a special prefix.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The function adds an ellipse if the string was actually truncated. Also
start using it in mcpu for result checks (where the message is also
slightly changed to use a colon).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

When modifications are made, disks may not have the same index anymore.
Updating all disks fixes this.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

When adding an item the index given to the callback function would be
incorrect under certain conditions. This patch also adds assertions and
more tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This patch adds support for adding/removing NICs/disks at arbitrary
indices on the command line. To add a disk at a specified index, use
“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”.
Unittests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The vgreduce command has changed behaviour from when we initially
wrote the code (2.02.02 versus 2.02.66, 4 years delta):

- if there are LVs which will be impacted, it requires --force
- otherwise refuses to proceed, but it still returns exit code 0

We handle this by looking to see if it returns "Wrote out consistent
volume group" (behaviour unchanged), or if it complains about
"--force"; in the case it didn't complete, we retry the operation.

We improve a bit the checking of "vgs", as it uses to fail silently
and we didn't detect it.

New tests for this function should test, I believe, all the expected
variations; at the least we now have data files with the expected
output.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is in preparation to de-duplicating significant chunks of code in
cmdlib._GenerateDiskTemplate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>