We'll start with a simple ping query, which returns ok if it gets no
query argument, and an error if it gets any. Also non-implemented query
are handled by the base ConfdQuery class, so we don't have to
special-case them in ConfdProcessor anymore.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

doc/examples/hooks/ethers has been added without being shipped in the
released tarball. Putting a stop to this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-confd is a simple asynchronous daemon, which listens on a UDP
port, passes each packet to a processor, and sends back to the client
the result.

It also listens on an inotify socket, in order to reload its
configuration when the ganeti config file changes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A ConfdProcessor is the job processor for a confd query.
It's responsible for all the processing, from unpacking it, verifying
it, to calculating the result.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It will be used for the confd server code, and for a basic client
library.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

AsyncNotifier is a special asyncore class that delivers inotify events
asynchronously.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The method is changed to a normal loop, to avoid calling getattr()
twice. Also __getstate__ is changed to just use ToDict() by default.

This should also make __getstate__ work for objects which have to
override the ToDict function because they contain other objects.

__setstate__ is probably still broken in this case, but so it was
before, and it's not used inside our code, so I'll pretend not to have
noticed, as there is no "nice" way to fix it, without overriding it all
over the place :(

Some unittests are added as a bonus, to make sure we behave well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of multiple uploads to each node, this script copies everything
as needed to the temporary directory, exactly as to be installed in the
destination machine, then runs only one rsync per host.

This is more dangerous (we can break /etc now), but for development
machines is fine.

The patch then also uploads the bash completions and the current name
for the cron job (I think that ganeti-master-cron is a deprecated name,
not that someone actually intends to upload a file named like that). A
flag --no-cron is added to skip uploading the cron file if desired.

The patch also changes rsync to propagate the file permissions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This helps emacs users ☺️



Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a configure-time customizable parameter that will be
used to enable stripped LVs. The default of the parameter is 3.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds a new hypervisor implementation: a chroot manager. This
hypervisor type can be used to manage (in combination with special OS
definitions) the start and stop of chroot areas, and if used with drbd
instances, it allows (via failover) the migration of chroots between
nodes.

This is a work in progress, and the way chroots should work is not very
clear and does not fit very well in the OS definition framework.
However, the hypervisor works and (if the sshd in the chroot is well
configured) it allows login to the instance both via ssh and console as
for a normal instance.

TODOs:
  - implement instance IP add/remove to/from the bridge, if the instance
    has a defined IP
  - investigate improvements to the OS API so that the create script has
    more information available, e.g. about the hypervisor type
  - mount extra disks in the chroot or alternatively refuse to start
    with more than one disk

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we have an insignificant glossary at the end of the design-2.0
document. This patch moves it to a separate file with the goal that it
will grow and all files can refer to it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This big patch converts the documentation build system to sphinx
(http://sphinx.pocoo.org/

). Since that uses reStructuredText sources
too, there is no change (yet) in the documents themselves, just in the
build system.

As before, the docs are pre built by the maintainer, and the end-user
doesn't need sphinx or other rst tools to build the docs. Note that we
are not distributing PDFs, so building that will require the tools.

The docs will be stored under doc/html and the build system also need an
extra directory doc/build. These are considered (by automake)
maintainer-related objects and are removed at maintainer-clean time.

The patch also fixes some small issues: add a docpng variable, add
doc/api (also generated by maintainer) in maintainer-clean-local, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch removes the autogeneration of the RAPI docs from the code
(based on docstrings) and moves the current autogenerated output to
the rapi.rst file.

The reasons behind this are multiple:
  - the build system becomes a little more simple (this could have been
	achieved also by distributing the built documentation, though)
  - it's hard to actually write documentation in docstrings; you have to
	fit restructured text inside the docstrings, and this results in
	not really nice output
  - even by being close to the code, the documentation manages to get
	out of sync (not paying attention to docstrings)

This will also help with the move to sphinx.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These were forgotten in commit 01e2ce3a,
and caused “make distcheck” to fail.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The Python interpreter used to run the test cases is hard-coded to be
/usr/bin/python. If we use the first one from $PATH instead, it is
much easier to test ganeti with other Python versions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Reviewed-by: iustinp

This patch changes the way documentation is built in order to distribute
the generated output in the 'dist' archive, and thus no longer
requiring the presence of the docbook/rst toolchains during build time.
This will lower the requirements for installation and also makes the
build time insignificant.

First, we remove the docbook2pdf rules and variables, since we no longer
build this kind of docs. Furthermore, the rst source files are not
(today) processed via replace_vars_sed, so the whole .in rules for doc/
go away.

Next, we change the ".sgml|.rst -> replace_vars_sed -> .in -> processor
-> final file" processing to ".sgml|.rst -> generator -> .in ->
replace_vars_sed -> final file"; this means we first process the file
using the formatter, with the @VARIABLE@ entries in it, and save the
output as .in; this output we distribute, and on the user side, the
replace_vars_sed will use the new configure flags to transform the
(almost final .in form) to the final form, without needing the
toolchain.

In configure.ac we also change from ERROR to WARN for the documentation
generators, and extra tests in Makefile.am check that the programs have
been found.

This was tested with distcheck and works as expected.

Reviewed-by: ultrotter

This patch changes the RAPI document, and the RAPI resources
autogenerated-documentation to restructured text. This meant changing
the autogen tool.

The new fragment can be included via RST directives, and doesn't need
passing through replace-sed-vars. This was also the last sgml document
in doc/, so we remove old makefile rules about it.

Reviewed-by: imsnah

This switches back to the hardcoding of the version number, as we don't
yet have a wrapper for rst files that passes them through
replace-sed-vars.

Reviewed-by: imsnah

I've somehow left these two out. Sorry!

Reviewed-by: imsnah

This is a not-complete bash completion file for ganeti commands (gnt-*)
and the burnin tool. It is based on previous work by Minghua Ye
<yeminghua@google.com> for Ganeti 1.1, which wasn't used because the
lack of ssconf keys (which allow easy inspection by the shell of the
existing nodes and instances) made it too slow.

The file works as expected, however I realized that our custom (like
comma-separated, or a=b:c,e:f) options are not very nice for
auto-completion. There are a few FIXMEs in the source for that.

The file is not installed at make install time, but it should be put in
the correct place by packages.

Reviewed-by: imsnah

This also updates the hooks document to 2.0.

Reviewed-by: ultrotter

This is a no-contents change, this doc will need update to conform to
2.0 message contents (and also the code will need increase to version 2
of the iallocator protocol).

Reviewed-by: imsnah

The RST format holds a little bit less information, as all the <file
class="directory"> and <userinput> tags are gone, however we're not
really losing important context here. And it's way easier to read and
update.

Reviewed-by: imsnah

This would help in generating online-viewable docs, which could link to
the man pages.

Reviewed-by: imsnah

This patch updates the gnt-* scripts to show the new 2.0 syntax. It's
not guaranteed to be 80% complete.

Reviewed-by: ultrotter

This patch adds new man pages for the master and RAPI daemons, and
updates the node daemon and watcher man pages.

Reviewed-by: ultrotter

LOCALSTATEDIR is added twice to the sed variables.

Reviewed-by: imsnah

The resources we still need moved to rlib2.

Reviewed-by: iustinp

The tcp port used for migrating KVM instances is selectable at
./configure time. We use a single port as nodes are locked anyway during
a migration, so no two migrations can happen at the same time to the
same node.

Reviewed-by: iustinp

As per RFC2617.

Reviewed-by: amishchenko

Call socat with a full path specified at configure time, rather than
just by its name, and check for the binary to exist at hypervisor
verify.

Reviewed-by: iustinp

This adds a docstring to the _autoconf.py file detailing how it's
generated (the other comment it's not visible in pydoc/epydoc).

Reviewed-by: amishchenko

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

This includes a large rewrite of the HTTP server code. The handling of
OpenSSL errors had some problems that were hard to fix with its
structure. When preparing all of this, I realized that actually HTTP
is a message protocol and that the same code can be used on both the
server and client side to parse requests/responses, with only a few
differences. There are still a few TODOs in the code, but none should
be a show stopper. Many pylint warnings have been fixed, too.

The old code will be removed once all users have been migrated.

Reviewed-by: amishchenko

This is a preparation step for splitting the HTTP client and server code
into two separate modules.

Reviewed-by: amishchenko