Reviewed-by: iustinp

This patch changes the way documentation is built in order to distribute
the generated output in the 'dist' archive, and thus no longer
requiring the presence of the docbook/rst toolchains during build time.
This will lower the requirements for installation and also makes the
build time insignificant.

First, we remove the docbook2pdf rules and variables, since we no longer
build this kind of docs. Furthermore, the rst source files are not
(today) processed via replace_vars_sed, so the whole .in rules for doc/
go away.

Next, we change the ".sgml|.rst -> replace_vars_sed -> .in -> processor
-> final file" processing to ".sgml|.rst -> generator -> .in ->
replace_vars_sed -> final file"; this means we first process the file
using the formatter, with the @VARIABLE@ entries in it, and save the
output as .in; this output we distribute, and on the user side, the
replace_vars_sed will use the new configure flags to transform the
(almost final .in form) to the final form, without needing the
toolchain.

In configure.ac we also change from ERROR to WARN for the documentation
generators, and extra tests in Makefile.am check that the programs have
been found.

This was tested with distcheck and works as expected.

Reviewed-by: ultrotter

This patch changes the RAPI document, and the RAPI resources
autogenerated-documentation to restructured text. This meant changing
the autogen tool.

The new fragment can be included via RST directives, and doesn't need
passing through replace-sed-vars. This was also the last sgml document
in doc/, so we remove old makefile rules about it.

Reviewed-by: imsnah

This switches back to the hardcoding of the version number, as we don't
yet have a wrapper for rst files that passes them through
replace-sed-vars.

Reviewed-by: imsnah

I've somehow left these two out. Sorry!

Reviewed-by: imsnah

This is a not-complete bash completion file for ganeti commands (gnt-*)
and the burnin tool. It is based on previous work by Minghua Ye
<yeminghua@google.com> for Ganeti 1.1, which wasn't used because the
lack of ssconf keys (which allow easy inspection by the shell of the
existing nodes and instances) made it too slow.

The file works as expected, however I realized that our custom (like
comma-separated, or a=b:c,e:f) options are not very nice for
auto-completion. There are a few FIXMEs in the source for that.

The file is not installed at make install time, but it should be put in
the correct place by packages.

Reviewed-by: imsnah

This also updates the hooks document to 2.0.

Reviewed-by: ultrotter

This is a no-contents change, this doc will need update to conform to
2.0 message contents (and also the code will need increase to version 2
of the iallocator protocol).

Reviewed-by: imsnah

The RST format holds a little bit less information, as all the <file
class="directory"> and <userinput> tags are gone, however we're not
really losing important context here. And it's way easier to read and
update.

Reviewed-by: imsnah

This would help in generating online-viewable docs, which could link to
the man pages.

Reviewed-by: imsnah

This patch updates the gnt-* scripts to show the new 2.0 syntax. It's
not guaranteed to be 80% complete.

Reviewed-by: ultrotter

This patch adds new man pages for the master and RAPI daemons, and
updates the node daemon and watcher man pages.

Reviewed-by: ultrotter

LOCALSTATEDIR is added twice to the sed variables.

Reviewed-by: imsnah

The resources we still need moved to rlib2.

Reviewed-by: iustinp

The tcp port used for migrating KVM instances is selectable at
./configure time. We use a single port as nodes are locked anyway during
a migration, so no two migrations can happen at the same time to the
same node.

Reviewed-by: iustinp

As per RFC2617.

Reviewed-by: amishchenko

Call socat with a full path specified at configure time, rather than
just by its name, and check for the binary to exist at hypervisor
verify.

Reviewed-by: iustinp

This adds a docstring to the _autoconf.py file detailing how it's
generated (the other comment it's not visible in pydoc/epydoc).

Reviewed-by: amishchenko

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

This includes a large rewrite of the HTTP server code. The handling of
OpenSSL errors had some problems that were hard to fix with its
structure. When preparing all of this, I realized that actually HTTP
is a message protocol and that the same code can be used on both the
server and client side to parse requests/responses, with only a few
differences. There are still a few TODOs in the code, but none should
be a show stopper. Many pylint warnings have been fixed, too.

The old code will be removed once all users have been migrated.

Reviewed-by: amishchenko

This is a preparation step for splitting the HTTP client and server code
into two separate modules.

Reviewed-by: amishchenko

Currently the temporary directory used for modules is under the current
directory; this patch changes it so a temp directory is used; so we
could build the apidocs with a R/O filesystem.

Reviewed-by: ultrotter

This patch adds a _FieldSet class that can be used for the new variable
parameter sets: e.g. the sda_size will change to disk/0.size (or
similar) and we need to both check validity and extract the index of the
parameter easily.

The patch also sorts the unittest list in Makefile.am.

Reviewed-by: ultrotter

Adding a new filetype/make rule requires more work than the previous
patch. Sorry! Now pases distcheck.

Reviewed-by: imsnah

This adds a simple diagram for the cluster architecture and also adds a
dependency on graphviz (both doc and in configure).

Reviewed-by: imsnah

This patch combines all the design documents for 2.0 except the
security one into a single document, in order to ease reading and reduce
duplication of information.

Future patches will start removing wrong pointers to old document names
and some better integration between the sections.

Reviewed-by: imsnah

Currently the makefile always recreates the ganeti symlink. Make it so
that it does only when it doesn't exist or it's not a symlink.

Reviewed-by: imsnah

Since now we use only one function from the logger module
(SetupLogging), we move it to utils.py (which is already imported by all
users of this function), and we remove the module.

Reviewed-by: imsnah

This was a leftover from the early days of the remote API in Ganeti 1.2.

Reviewed-by: ultrotter

This mainloop can be used in daemons like ganeti-noded.

Reviewed-by: iustinp

Reviewed-by: imsnah

Reviewed-by: imsnah

This function will be used for auto-archiving jobs via the command line.
The function is pretty simple, we only support up to weeks since months
and higher are not 'precise' entities, and dealing with them would
require us to start using calendar functions.

Reviewed-by: imsnah

Reviewed-by: iustinp

docbook-wrapper had the names for the docbook2* programs hardcoded. This
patch changes Makefile.am and the wrapper script to pass them via
another argument.

Another issue where rapi.in was built before rapi-resources.sgml is
also fixed.

Reviewed-by: iustinp

This patch also adds the design documents to Makefile.am.

Reviewed-by: iustinp

Reviewed-by: imsnah

I forgot to remove these when converting the QA configuration from YAML
to JSON.

Reviewed-by: ultrotter

The cluster no longer keeps individual host's SSH key, but rather
aliases all of them to the cluster name.

Reviewed-by: ultrotter

To my knowledge they're used nowhere and it's at least slightly
confusing to people adding new QA checks.

Reviewed-by: ultrotter