Currently the RAPI client uses the urllib2 and httplib modules from
Python's standard library. They're used with pyOpenSSL in a very fragile
way, and there are known issues when receiving large responses from a RAPI
server.

By switching to PycURL we leverage the power and stability of the
widely-used curl library (libcurl). This brings us much more flexibility
than before, and timeouts were easily implemented (something that would
have involved a lot of work with the built-in modules).

There's one small drawback: Programs using libcurl have to call
curl_global_init(3) (available as pycurl.global_init) while exactly one
thread is running (e.g. before other threads) and are supposed to call
curl_global_cleanup(3) (available as pycurl.global_cleanup) upon exiting.
See the manpages for details. A decorator is provided to simplify this.

Unittests for the new code are provided, increasing the test coverage of
the RAPI client from 74% to 89%.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix the sentence to say what it means.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This test moves an instance on the same cluster and, if successful,
moves it back. While not testing a real move between two clusters,
this is certainly better than nothing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Two new resources are added:
- /2/instances/$name/prepare-export
- /2/instances/$name/export

The documentation for the existing resource for creating instances is updated
for remote imports. The RAPI client is extended for the new resources.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The RAPI tests will depend on the certificate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Add support for sending JSON-encoded body to RAPI daemon
  (required to test more complex resources)
- Expand names when testing resources against config

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this, the burnin sequence on big clusters should be (much?) faster.
The patch also fixes the parallel setting (before it was always
enabled).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Just using “echo” would use whatever umask is set in the shell,
in addition to potentially showing the password in the process
list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Since the original instance still exists, we need to use a different MAC
for its NIC.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch makes sure that the Ganeti RAPI credentials are setup,
if any, on cluster init time.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* This also adds support for authenticated RAPI calls
* Other HTTP methods than GET/POST

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Information exchanged between different clusters via untrusted
third parties (e.g. for remote instance import/export) must be
signed with a secret shared between all involved clusters to
ensure the third party doesn't modify the information.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” or
simply “HMAC key” everywhere. With the implementation of inter-cluster
instance moves, another HMAC key will be introduced for signing critical
data. They can not be the same, so this patch clarifies the purpose of the
“cluster HMAC key” by renaming it. The actual file name is not changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The new test depends on the drbd type tests being enabled, and test
conversion to plain and back to drbd.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Recent changes to “gnt-cluster verify” made it complain on expiring SSL
certificates. While it was possible to replace the SSL certificates and
other cluster secrets manually before, doing so was cumbersome. Cluster
certificates, keys and secrets can now be replaced easily.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This adds basic qa tests for gnt-os modify

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When not initialising the cluster, consider all nodes are added, so that
multi-node tests (e.g. export, replace) work correctly (if there are
nodes, of course).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This also adds the new nic.links query field.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Before the arguments weren't valid.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This option is gone since 1ac0f5e6d2f06a5bf12815e1b92d1338af32833c,
fixing the QA suite to deal with this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently rapi is the only daemon which accepts a port option, rather
than querying its own port from services, and failing back to the
default if not found. Changing this to conform to what other daemons do.

Also update the ganeti-rapi(8) manpage

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This constant is unused, except in qa. Removing it since it's always True.

This patch also removes the unused qa_rapi.PrintRemoteAPIWarning
function, and removes a comment about temporary constants "until we have
cluster parameters".

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also, since Ganeti 2.1 will be compatible with both 10 and 15, change
the OS_API_VERSION constant to be an OS_API_VERSIONS set, and update the
places in the code that used that constat to use something else.

In particular:
  - in the qa for now we just create a fake version 10 OS
  - in the os environment we use the highest common version
    (which means we need to pass in the os to OSEnvironment)
  - when loading an OS any common version will do

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In Ganeti 1.2, “none” was used to signify no initrd. In 2.0 we have
changed to “no_” as a prefix (i.e. “-H no_initrd_path”) and thus we
document in the manpage this.

The QA suite is changed accordingly.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds support for optionally doing the rename burnin test, and
adds an example to the sample QA file. To disable, either remove or
specify an empty rename target.

Reviewed-by: imsnah

This was skipped in the previous QA patch.

Reviewed-by: imsnah

Since we by default now use SSL for RAPI, we need to switch the QA
tests to SSL too.

Reviewed-by: amishchenko

This patch fixes the RAPI QA tests to work with today's RAPI code and
also does some other minor improvements:
  - QA: only create the cluster if so configured (‘create-cluster’ key),
    this allows running parts of the QA suite against existing clusters
  - export the “hvparams” for instances in RAPI

Reviewed-by: imsnah

This patch adds burnin parameters for --parallel and --http-check
options to the burnin script.

Reviewed-by: ultrotter

There are still issues, especially with "gnt-instance modify" and
resetting values. However, this is a start.

Reviewed-by: ultrotter