ganeti-masterd: Add initialization and shutdown of RPC pool. It needs
to be shutdown before forking.

ganeti.cli: Add decorator function to initialize and shutdown RPC pool.

ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw
exception when used without proper initialization.

gnt-cluster, gnt-node: Use decorator function to initialize and shutdown
RPC pool.

Reviewed-by: iustinp

Reviewed-by: ultrotter

Reviewed-by: iustinp

Also fix an error in the CheckPrereq.

Reviewed-by: imsnah

This function will be used to write ssconf files from the node daemon.
By creating a lock file, we synchronize different child processes of
ganeti-noded to not overwrite each other's changes. Also, external
processes could use this lock to make sure they don't read ssconf
files while changes are going on.

The socket module is not used (anymore) in ganeti.ssconf, removing it.

Reviewed-by: iustinp

This patch changes the drbd8 replace disk only (no secondary change) to
work in with multi-disk. This mode of replaces works correctly with
replacing only a subset of disks.

Reviewed-by: imsnah

This patch adds support for mult-disk/multi-nic in:
  - instance add
  - burnin

The start/stop/failover/cluster verify work as expected. Replace disk
and grow disk are TODO.

There's also a change gnt-job to allow dictionaries to be listed in
gnt-job info.

Reviewed-by: imsnah

This adds some more listing cases (useful for scripting/rapi):
  - disk.sizes for a list of all sizes
  - nic.(ips|macs|bridges)

Reviewed-by: imsnah

Currently the iv_name is very linux-specific, and will break with the
multi-disk changes.

The patch changes this to generate sdX names based on the disk index in
the disks structure, instead of relying on the iv_name.

Reviewed-by: imsnah

Before we had lots, lots and lots of code duplication. This patch
changes the code to use four central functions.

Reviewed-by: iustinp

This allows a single HttpClientManager to be used from more than one
thread at the same time. We discussed having one HttpClientManager
per job queue thread. Assuming there should be one HTTP thread per
node, this would mean quadratic growth with the number of nodes. By
being able to reuse the manager, this problem is a defused a bit.

This patch also fixes a typo in HttpClientRequestExecutor.

Reviewed-by: iustinp

Reviewed-by: imsnah

Split parameters filter to the separate function and reuse it in instance creation.

The waiting sharer blocks exclusive is another not-possible to test
right now.

Sorry for missing it the first time; there are no other testWaiting...
tests.

Reviewed-by: ultrotter

This patch adds support for querying in gnt-instance list of:
  - disk.count
  - nic.count
  - disk.size/$N
  - nic.(ip|mac|bridge)/$N

The patch also disables the exception raised when the header description
is missing, that should be fixed in the future (together with better
handling of variable numeric fields).

Reviewed-by: imsnah

This patch converts the current usage of _CheckOutputFields to the
FieldSet class, but it doesn't start to use its variable matching
features.

Reviewed-by: ultrotter

This patch adds a _FieldSet class that can be used for the new variable
parameter sets: e.g. the sda_size will change to disk/0.size (or
similar) and we need to both check validity and extract the index of the
parameter easily.

The patch also sorts the unittest list in Makefile.am.

Reviewed-by: ultrotter

Until we have a way to properly test this, we'll skip the test.

Reviewed-by: imsnah

This patch removes the client-api.txt document (since it's obsoleted by
the documentation inside the design-2.0.rst document) and adds many
updates to the latter.

Reviewed-by: imsnah

Adding a new filetype/make rule requires more work than the previous
patch. Sorry! Now pases distcheck.

Reviewed-by: imsnah

This adds a simple diagram for the cluster architecture and also adds a
dependency on graphviz (both doc and in configure).

Reviewed-by: imsnah

This patch combines all the design documents for 2.0 except the
security one into a single document, in order to ease reading and reduce
duplication of information.

Future patches will start removing wrong pointers to old document names
and some better integration between the sections.

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: iustinp

Reviewed-by: iustinp

If a logging function is called before the logging module is setup
using utils.SetupLogging, it calls logging.basicConfig, which adds
a StreamHandler, on its own. If we leave it in, all log output will
be printed twice on stderr. This patch makes sure to remove all
handlers before adding our own.

Reviewed-by: iustinp

"from ganeti.rpc import RpcRunner" does not conform to the style guide.

Reviewed-by: iustinp

"from ganeti.rpc import RpcRunner" does not conform to the style guide.

Reviewed-by: iustinp

This is one of the last places where the simplejson module is
used directly.

Reviewed-by: iustinp

Currently, HttpClientManager is instantiated for every RPC call. This
will be changed with another patch, as will the use of SSL. The “Run”
method is no longer needed.

Reviewed-by: iustinp

Currently the locking tests are using timeouts to ensure an event will
'never happen'. However, this is suboptimal.

The patch converts all of these to instead: not wait, but sequence the
operations logically and expect that they execute as such. In case of
not so, they will timeout with a big (60s) timeout.

The 'never happen' is tested by multiple repetitions of the same test -
this is not perfect, but again 'X will never happen' is not actually
testable.

This patch reduces the runtime of the tests from ~5.4 to ~0.8 seconds
(with 8 repetitions of each test thas has 'never happen' situations).

Reviewed-by: imsnah

Rev 1978 introduced a breakage on the SIGKILL finall signal to the
process, due to mistyped variable.

Reviewed-by: francis.perron

Implementing SSL for the HTTP client required more work than I expected.
For correct error handling, quite a lot of code is needed. To avoid
code duplication, I moved a lot of the socket handling code into a
single function named _SocketOperation. It takes care of the polling
and error handling for OpenSSL. At the same time, the functions calling
it become a lot smaller.

This patch brings a few more changes:
- Error handling for socket.connect_ex (e.g. name resolution failures)
- Rename "ResponseError" exception class to "_HttpClientError" --
  it's an internal class anyway

Reviewed-by: iustinp

This is a fallout from my work on the HTTP client class. Repeating
timers are prepared, but not yet implemented.

Reviewed-by: iustinp

We change two functions to use RunCmd without shell, and the other
(which needs a ssh command line) is changed to the '>... 2>&1' syntax.

Reviewed-by: imsnah

Currently the dir creation in ganeti-noded is in the main function. This
is not nice: we move it into a separate function and also add creation
of the OS_LOG_DIR (with different permissions, but in the same way).
This will permit cleanup of the creation of the OS_LOG_DIR from the
backend module (it's done multiple places currently).

Reviewed-by: imsnah

Based on the discussion on the mailing list, we remove the special
casing of zombie processes.

Note we don't use kill -0 since that has a different meaning than "check
process is alive", so this function is still linux-specific.

Reviewed-by: imsnah

In case the process we want to signal is our own process, and the signal
we send is a deadly one, we should also cleanup after the process.

This patch adds a new parameter waitpid to this function that does this,
and changes its unittest to pass this new parameter.

Reviewed-by: imsnah

As can be seen from the patch, we remove from the locking manager
inst.name. However, we do this right after cfg.RenameInstance, which
changes inst.name to the new name. So we need to remove old_name not
the new name.

The real question is why does the glm allow me to remove a not-existing
instance name...

The symptom before the patch was: run rename, then:
$ gnt-instance list
Unhandled protocol error while talking to the master daemon:
Caught exception: 'NoneType' object has no attribute 'primary_node'

Reviewed-by: imsnah