GitLab

This is an extract of commit 1166 on the 1.2 branch (Add a rpc call for
drbd network reconfiguration), but only the blockdev_close part.

The patch changes the blockdev_close call to take the instance so that
it can remove the symlinks of the instance.

Originally-Reviewed-by: imsnah

Reviewed-by: ultrotter

In the iteration we don't care about the node names, so we change the
for loop to be over the values (and not itervalues).

Reviewed-by: amishchenko

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

Adding compression to larger amounts of data is more efficient than
transferring it (len(nodes) - 1) times over the network without
compression. We were able to compress a 800KB config file to about
30 KB, which is about 40 KB with Base64 encoding (required due to
the way SimpleJson handles strings).

Reviewed-by: ultrotter

This patch changes the _MultNodeCall and _SingleNodeCall helpers to not
actually make calls to offline nodes, but instead generate fake
responses which have a parameter caller 'offline' set so that callers
can check for this value if they want (otherwise, it's just a failed RPC
call).

Reviewed-by: ultrotter

This patch adds a simple rpc which makes a backup of the config file and
then removes it. This is done so that cluster verify doesn't complain
immediately after demoting a node.

Reviewed-by: imsnah

Reviewed-by: amishchenko

For a long time we had the problem that both RPC-layer errors and
results from the remote node share the same "valuespace". This is
because we shouldn't raise an exception when only one node failed
(and lose the results from the other nodes).

This patch attempts to address this problem by returning a special
object from RPC calls, which separates the rpc-layer status and the
remote results into different attributes.

All the users of rpc (mainly cmdlib, but also bootstrap and the
HooksMaster in mcpu) have been converted to this new model. The code has
changed from, e.g. for boolean return types:

  if not self.rpc.call_...

to

  result = self.rpc.call_
  if result.failed or not result.data:
     ^ rpc-layer error    |
                          - result payload

While this is slightly more complicated, it will allow cleaner checks in
the future; right now the code is just a plain port, without
optimizations.

There's also a "result.Raise()" which raises an OpExecError if the
rpc-layer had errors.

One side-effect of the patch is that now all return types from the
rpc.call_ functions are of either RpcResult (single-node) or dicts of
(node name, RpcResult); previously, some functions were returning
different object types based on error status.

The code passes burnin (after many retries :).

Reviewed-by: imsnah

Unfortunately, a rpc.Client object was passed as the first parameter,
causing the function to always fail.

Found during QA testing.

Reviewed-by: ultrotter

This patch removes the duplicate serialization and calls to
utils.GetNodeDaemonPort in rpc.Client, and instead moves them to calling
functions (the _*NodeCall ones recently introduced).

Reviewed-by: ultrotter

Instead of parsing the configuration on the node, we pass the ssconf
values from the master.

Reviewed-by: iustinp

There's no need to read the SSL certificate and key for every node
in a request. Also add a TODO for better error reporting.

Reviewed-by: amishchenko

This patch enables SSL between masterd and noded.

Reviewed-by: iustinp

With the new SSL client certificate stuff it's no longer needed.

Reviewed-by: iustinp

ganeti-masterd: Add initialization and shutdown of RPC pool. It needs
to be shutdown before forking.

ganeti.cli: Add decorator function to initialize and shutdown RPC pool.

ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw
exception when used without proper initialization.

gnt-cluster, gnt-node: Use decorator function to initialize and shutdown
RPC pool.

Reviewed-by: iustinp

Reviewed-by: iustinp

Before we had lots, lots and lots of code duplication. This patch
changes the code to use four central functions.

Reviewed-by: iustinp

Reviewed-by: iustinp

This is one of the last places where the simplejson module is
used directly.

Reviewed-by: iustinp

Currently, HttpClientManager is instantiated for every RPC call. This
will be changed with another patch, as will the use of SSL. The “Run”
method is no longer needed.

Reviewed-by: iustinp

We want to export the disk index as some OSes will only want to export
the first disk (or the second one, etc.), even if we have multiple
disks.

The patch also updates the backend.ExportSnapshot docstring.

Reviewed-by: ultrotter

- Change ImportOSIntoInstance not to get any "os_disk" and "swap_disk"
  arguments but to accept multiple target images to import, and to
  return a list of booleans with the result of each import
- Change the relevant rpc call and the only caller to conform
- Pass arguments to the import script through the environment
- Run one import os script for each disk image, passing an IMPORT_DEVICE

Reviewed-by: iustinp

The two main multi-node job queue RPC calls (jobqueue_update,
jobqueue_rename) are converted to address-based calls, in order to speed
up queue changes. For this, we need to change the _nodes attribute on
the jobqueue to be a dict {name: ip}, instead of a set.

Reviewed-by: imsnah

This patch allows rpc.call_upload_file to use addresses (if passed), and
also converts the ConfigWriter._DistributeConfig to pass them, since
this is an often-done operation.

Reviewed-by: imsnah

This patch converts all non-staticmethods rpc calls to pass addresses to
the Client calls.

This is done via two new small helpers. These functions also deal with
nodes that are not (yet) in the configuration file, and pass None for
the address instead (which allow NodeControllers to do resolving).

Reviewed-by: imsnah

The patch adds support for passing node addresses to NodeController
objects, which should speed up the connect sequence.

The patch also adds another very small optimization of computing the
content-length header value in the parent Client class instead of each
NodeController.

Reviewed-by: imsnah

This patch renames some internal function to camel case, for compliance
with the style guide.

It also fixes the module docstring (how old was that?).

Reviewed-by: imsnah

Reviewed-by: imsnah

- remove now unused osdev and swapdev arguments from backend, noded,
  rpc, cmdlib
- convert docstrings to epydoc

Reviewed-by: iustinp

A new multi-node call is added that sets/resets the drain flag.

Reviewed-by: imsnah

As for the hvparams, we need to replace the sent instance beparams.

Reviewed-by: ultrotter

We fill the instance hvparams with cluster defaults, and send a modified
dict over the wire to the node methods - they don't know anything about
cluster defaults versus instance parameters.

Reviewed-by: ultrotter

The newly-added node-specific ValidateParams hypervisor method is
exported over RPC, using the semi-standard (success, message) return
value. Multi-node call, so that we call on both primary and secondary at
once.

Reviewed-by: ultrotter

This fixes:
  - whitespace change, double lines between methods
  - duplication of call_upload_file, introduced by mistake in rev 1795
    and which went undetected because of the many changes in that ref
    (only diff -b shows it clearly)
  - call_instance_info didn't pass the hypervisor name parameter, but
    the backend requires it

Reviewed-by: ultrotter

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to
change the implementation.

The patch abstracts this into a separate utils.OwnIpAddress(addr)
function. We add a rpc call for it, which we use instead of the
(single-use of) call_node_tcp_ping. We leave node_tcp_ping in, as seems
useful and eventually it should be removed in a separate patch.

Reviewed-by: imsnah

Reviewed-by: iustinp

This big patch changes the call model used in internode-rpc from
standalong function calls in the rpc module to via a RpcRunner class,
that holds all the methods. This can be used in the future to enable
smarter processing in the RPC layer itself (some quick examples are not
setting the DiskID from cmdlib code, but only once in each rpc call,
etc.).

There are a few RPC calls that are made outside of the LU code, and
these calls are left as staticmethods, so they can be used without a
class instance (which requires a ConfigWriter instance).

Reviewed-by: imsnah

This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains
the ‘enable_hypervisors’ attribute, and instances can be created with
any of the enabled ones (no provision yet for changing that attribute).

The many many changes in the rpc/backend layer are due to the fact that
all backend code read the hypervisor from the local copy of the config,
and now we have to send it (either in the instance object, or as a
separate parameter) for each function.

The node list by default will list the node free/total memory for the
default hypervisor, a new flag to it should exist to select another
hypervisor. Instance list has a new field, hypervisor, that shows the
instance hypervisor. Cluster verify runs for all enabled hypervisor
types.

The new FIXMEs are related to IAllocator, since now the node
total/free/used memory counts are wrong (we can't reliably compute the
free memory).

Reviewed-by: imsnah

Currently the call_instance_migrate call only passes the instance name;
we need to pass the whole object for the hypervisor_type changes (all
the other individual instance rpc calls already pass the instance
object).

Reviewed-by: imsnah