Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The following (blockdev) RPC calls are not converted yet (as they are
not straight forward or need more research):

* bdev_sizes
* blockdev_remove
* blockdev_shutdown
* blockdev_removechildren
* blockdev_close
* blockdev_getsize
* drbd_disconnect_net
* blockdev_rename (has already a special encoder, needs further research
  if needed at all)
* blockdev_getmirrorstatus (not sure if we have everywhere a clear link
  to the instance the disk belongs)
* blockdev_getmirrorstatus_multi (same here, further research)

Then special cases where we take care later in the patch series:

* blockdev_create (special cased)
* blockdev_find (special cased, like blockdev_create)

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We change the cmdlib side on the next patch.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently we allow for up to NODE_MAX_CLOCK_SKEW time difference
between nodes in some operations, but not everywhere: SSL certificate
verification (import/export, both intra and inter-cluster) has a zero
limit (downwards), and a week upwards. This can cause even
intra-cluster backup problems, if the source node has a time even two
seconds in the future.

To fix this, when we verify certificates compare with a time offset
with the max skew, which fixes the lower bound and reduces the upper
bound by an insignificant amount (0.04%).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Two Luxi calls have inconsistent an name/value mapping (in the Python
code):

- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)
- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)

While these are only a consistency issue, let's fix them so that the
Haskell code (which uses the auto-generated camel-case form) doesn't
need to handle them case specially, and looks more like the Python
code (hah, joke!).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The reason why grow-disk was doing:

$ gnt-instance grow-disk instance3 0 -64
Unhandled Ganeti error: Invalid format

Is because it does it's own ParseUnit call, and doesn't transform that
into a nicer message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

So while testing my previous patch I run setup-ssh manually without
any arguments, and it did nothing (no complains, no messages,
anything). That was very surprising, so let's make it behave better if
no nodes are passed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a mock RAPI client to test input values to methods. All
methods either raise an exception if there was a problem or return None.
Third-party code can use this to test their input values to the RAPI
client.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this patch headers are constructed from the PycURL options
and passed to the mock implementation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is preparation for the mock system, where we need the same cURL
mock.

Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This exception is raised to abort before actually sending a LUXI call
(there is no LUXI server involved in the test). The testing utilities
catch the exception to report a success (i.e. the code didn't throw
an exception before due to invalid types, etc.). To allow the exception
to be thrown all the way to the test utilities, the HTTP server library
must ignore it. Also some overly generic exception handling is removed
from the RAPI request handler.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will be used in a unittest for RAPI testing utilities.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

By moving the code loading RAPI users from a file into a standalone
function it becomes easier to use the RAPI request handler in
tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This will be used for providing a type-checking RAPI client for tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This also improves the log messages for the (default) relative mode
("by %s to %s").

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The CLI currently doesn't permit this, but a direct job submission
would allow it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This logs all external commands in normal (non-debug) mode. This will
leak the DRBD secrets, but in any case we do log failed commands, so
it's not a significant change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Chris Schrier <schrierc@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fixes QA after commit cfdf561d.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Chris Schrier <schrierc@google.com>

Short description: This fixes an issue whereby masterd would become
unresponsive on the LUXI socket, leading to client timeouts. While made
worse in 2.5, the underlying issue was already present in 2.4.

Longer description: Until now all LUXI queries would acquire the BGL
(big Ganeti lock) in shared mode. With the exception of OpNodeAdd and
OpNodeRemove, this was also the case for all opcodes before version 2.5.
In 2.5 we split OpClusterVerify into multiple opcodes, one of which
(OpClusterVerifyConfig) now acquires the BGL in exclusive mode. Whether
or not doing so is good is a separate discussion: OpNodeAdd and
OpNodeRemove, as of this writing, still require an exclusive BGL.
OpClusterVerifyConfig is run more often than OpNodeAdd or OpNodeRemove
in normal clusters, which is why we only recognized this issue in 2.5.

What would happen is that once OpClusterVerifyConfig tried to acquire
its exclusive BGL while it was actually held by other opcodes (e.g.
OpInstanceReplaceDisks), the locking code would not grant shared
acquires for the BGL, even when the exclusive acquire is removed from
the queue for a short amount of time after a timeout. This is necessary
to prevent lock starvation.

In this situation further LUXI queries requiring the BGL in shared mode,
e.g. OpClusterQuery, would block and the client eventually time out.
Over time they fill the client request workerpool's queue and at that
point even requests not requiring the BGL stop working. Once the
long-running operation(s) holding the BGL in shared mode finished,
OpClusterVerifyConfig gets it in exclusive mode and everything returns
to normal. LUXI recovers very soon too.

I'd like to thank Bernardo Dal Seno for his contribution to this bugfix.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 0fa753ba)

Also mark it as deprecated in NEWS as normal queries can be used
instead.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The main intention of this patch is to make it possible to retrieve
cluster tags via query2. While at it I decided to convert
LUClusterConfigQuery right away. Some of the values returned by
LUClusterQuery are also included, but the conversion of LUClusterQuery
is not yet complete.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These are not used anywhere in Python or Haskell.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This solves one case where locks are acquired during LUXI queries.
Pretty late into the transition I noticed that OpBackupQuery had a
“use_locking” parameter for a long time, but didn't use it. Since
most of the other changes were already and this allows exports to
be listed via RAPI (/2/query) I decided to finish.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

str.isdigit is about 4x faster than using a regular expression ("\d+").
This is in the inner sorting code so speed matters.

$ python -m timeit -s 'import re; s = re.compile("^\d+$")' \
's.match(""); s.match("Hello World"); s.match("1234")'
1000000 loops, best of 3: 0.937 usec per loop

$ python -m timeit '"".isdigit(); "Hello World".isdigit(); "1234".isdigit()'
1000000 loops, best of 3: 0.218 usec per loop

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Chris Schrier <schrierc@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It was only used in one place and wasn't really necessary.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUXI queries are processed without callbacks (see
server.masterd.ClientOps._Query). With commit 07923a3c the logic for
checking an opcode's result for jobs to submit was changed and
subsequently raised an exception (“'NoneType' object has no attribute
'SubmitManyJobs'”) in such a case. Before said commit the exception
would also have been raised if an opcode used by a query submitted jobs.

This patch changes the logic to only resolve the method if callbacks are
defined and to use an exception-raising implementation otherwise.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds support for "show" as an alias for "info" to
gnt-(cluster|instance|node|os). It already exists in gnt-job.

Signed-off-by: Alexander Schreiber <als@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>