This adds a simple diagram for the cluster architecture and also adds a
dependency on graphviz (both doc and in configure).

Reviewed-by: imsnah

This patch combines all the design documents for 2.0 except the
security one into a single document, in order to ease reading and reduce
duplication of information.

Future patches will start removing wrong pointers to old document names
and some better integration between the sections.

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: iustinp

Reviewed-by: iustinp

If a logging function is called before the logging module is setup
using utils.SetupLogging, it calls logging.basicConfig, which adds
a StreamHandler, on its own. If we leave it in, all log output will
be printed twice on stderr. This patch makes sure to remove all
handlers before adding our own.

Reviewed-by: iustinp

"from ganeti.rpc import RpcRunner" does not conform to the style guide.

Reviewed-by: iustinp

"from ganeti.rpc import RpcRunner" does not conform to the style guide.

Reviewed-by: iustinp

This is one of the last places where the simplejson module is
used directly.

Reviewed-by: iustinp

Currently, HttpClientManager is instantiated for every RPC call. This
will be changed with another patch, as will the use of SSL. The “Run”
method is no longer needed.

Reviewed-by: iustinp

Currently the locking tests are using timeouts to ensure an event will
'never happen'. However, this is suboptimal.

The patch converts all of these to instead: not wait, but sequence the
operations logically and expect that they execute as such. In case of
not so, they will timeout with a big (60s) timeout.

The 'never happen' is tested by multiple repetitions of the same test -
this is not perfect, but again 'X will never happen' is not actually
testable.

This patch reduces the runtime of the tests from ~5.4 to ~0.8 seconds
(with 8 repetitions of each test thas has 'never happen' situations).

Reviewed-by: imsnah

Rev 1978 introduced a breakage on the SIGKILL finall signal to the
process, due to mistyped variable.

Reviewed-by: francis.perron

Implementing SSL for the HTTP client required more work than I expected.
For correct error handling, quite a lot of code is needed. To avoid
code duplication, I moved a lot of the socket handling code into a
single function named _SocketOperation. It takes care of the polling
and error handling for OpenSSL. At the same time, the functions calling
it become a lot smaller.

This patch brings a few more changes:
- Error handling for socket.connect_ex (e.g. name resolution failures)
- Rename "ResponseError" exception class to "_HttpClientError" --
  it's an internal class anyway

Reviewed-by: iustinp

This is a fallout from my work on the HTTP client class. Repeating
timers are prepared, but not yet implemented.

Reviewed-by: iustinp

We change two functions to use RunCmd without shell, and the other
(which needs a ssh command line) is changed to the '>... 2>&1' syntax.

Reviewed-by: imsnah

Currently the dir creation in ganeti-noded is in the main function. This
is not nice: we move it into a separate function and also add creation
of the OS_LOG_DIR (with different permissions, but in the same way).
This will permit cleanup of the creation of the OS_LOG_DIR from the
backend module (it's done multiple places currently).

Reviewed-by: imsnah

Based on the discussion on the mailing list, we remove the special
casing of zombie processes.

Note we don't use kill -0 since that has a different meaning than "check
process is alive", so this function is still linux-specific.

Reviewed-by: imsnah

In case the process we want to signal is our own process, and the signal
we send is a deadly one, we should also cleanup after the process.

This patch adds a new parameter waitpid to this function that does this,
and changes its unittest to pass this new parameter.

Reviewed-by: imsnah

As can be seen from the patch, we remove from the locking manager
inst.name. However, we do this right after cfg.RenameInstance, which
changes inst.name to the new name. So we need to remove old_name not
the new name.

The real question is why does the glm allow me to remove a not-existing
instance name...

The symptom before the patch was: run rename, then:
$ gnt-instance list
Unhandled protocol error while talking to the master daemon:
Caught exception: 'NoneType' object has no attribute 'primary_node'

Reviewed-by: imsnah

Currently the makefile always recreates the ganeti symlink. Make it so
that it does only when it doesn't exist or it's not a symlink.

Reviewed-by: imsnah

Commit 1881 changed a rpc call but didn't fix all its users. This should
fix it (but I can't test as HTTP is broken).

Reviewed-by: imsnah

In order to replace usage of the shell just for 'cd %s; ...' we need to
be able to specify the working directory for childs.

The patch also changes the default working directory (when not
specified) to "/", as opposed to the current dir; since the daemos run
with cwd=/ anyway, this should not change things.

Reviewed-by: imsnah

The same code will be used by the HTTP client.

Reviewed-by: iustinp

While working on the HTTP client class, I wanted to use Mainloop
before deciding to use threads instead.

- Add docstrings
- Rename "quit" to "running", move it to a local variable
- Support adding and removing I/O listeners while running
  (e.g. to add/remove listeners from OnIO(…) calls)
- Allow user of the class to disable signal handlers
- Flag to exit once all listeners are gone

Reviewed-by: iustinp

This is an expected error message and will be used in two places.

Reviewed-by: iustinp

Reviewed-by: imsnah

It is based on the WorkerPool class which is already used for the job
queue and master daemon. Each request must be encapsulated in an
instance of HttpClientRequest, which will then be passed to
HttpClientManager for processing. Upon completion, the request object
contains the response or error message.

While not yet enabled, the HTTP client implementation has been designed
with HTTP/1.1 and keep-alive in mind. For now it only uses HTTP/1.0,
though.

HttpClientManager will likely need more changes when integrating with
the RPC layer. SSL will also be integrated in a second step.

Reviewed-by: iustinp, ultrotter

Reviewed-by: iustinp

The _WaitForSync call changed in r1794 but the GrowDisk call wasn't updated
then. This makes GrowDisk work in trunk.

Reviewed-by: iustinp

As a preparation for the new HTTP client class, add more constants
to ganeti.http.

Reviewed-by: iustinp

Reviewed-by: imsnah

Reviewed-by: imsnah

The newer_than variable can be either None or an int, and we normalize
it to an integer previously and save it in the 'serial' variable, which
should be used instead.

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Currently gnt-backup list exits with the same error code even if can't
contact som nodes for backup information.

The patch changes it to return 1 instead in this case.

Reviewed-by: imsnah

The patch also adds a 'return 0' to a function that was missing it.

Reviewed-by: imsnah