Some directories don't exist in the repository, but are required at build time
(e.g. doc/html). Until now some were created explicitly, some through the
target “stamp-directories” and other target simply relied on a previous target
to create the directory.

This patch tries to clean this up by getting rid of “stamp-directories” and
instead use rules to recreate any missing directory. As described in a comment
in the code, a file inside each directory is necessary, named “.dir”.

Order-only dependencies are used for directory creation to avoid rebuilding
where only the “.dir” file is missing (see “info make”, section “4.3 Types of
Prerequisites”).

The target for building the documentation is also changed to use “…/index.html”
instead of a hidden file. Some style changes are also made.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- run-in-tempdir should depend on what it copies to the temporary dir
- Add PYTHON_BOOTSTRAP to BUILT_SOURCES
- Don't use “mkdir -p” directly
- Create directory if necessary for writing bootstrap script

In summary, this should make “make distcheck” in pristine
checkout work again.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Eventually this will help ensuring that clients and servers are of the
same version, as long as they're imported from the same path. Currently
it's relatively easy for gnt-* and ganeti-* to be from a different
version.

Scripts will be at ganeti.client.gnt_* and a small bootstrap script
calls a “Main” function from the module.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 6e991d0e.

Conflicts:

	lib/constants.py (this got already removed, so no changes in here)

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The way we currently call pylint, the exact order it inspect modules in
lib/http/ depends on the filesystem order. This is not good, and if
lib/http/server.py is loaded before lib/http/__init__.py, it will throw
a "R0921:763:HttpMessageReader: Abstract class not referenced" (as that
class is used in server.py).

For the short-term fix, we just add server.py after "ganeti", so that
it gets parsed (again?) and pylint sees the usage of the class.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is for cleanup, and for later reuse in other parts of the code
(outside of LUs).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is used to expand the users/group names just once at
initial call.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

etags has cannot properly identify the scripts/gnt-* files as Python, so
we statically force the language to Python.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

“find .” requires that “-path” arguments start with a dot, otherwise
they are not matches. Additionally, we also include the QA files in the
tags, for easier search while modifying the QA suite.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This prepares the remote node to be joined into a cluster

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a new vcs-version file that is generated via git (and
can be adapted if VCS is changed) and then embebbded as VCS_VERSION in
the constants module.

This means two things:
- local modifications without committing to git (or when using a tar.gz
  archive + mods) will not be reflected
- version is fixed at the time of the last make regen-vcs-version (dist time,
  or devel/upload which calls this)

Thus this is more geared at developers rather than end users.

The patch:

- adds rules for generating the vcs-version file
- adds a dist-hook for re-generating the file (if possible) and copying
  the updated version to the distdir
- modifies devel/upload to re-generate the file before upload

The output of --version will look like:
gnt-cluster (ganeti v2.2.0beta0-184-gebca7e6) 2.2.0~beta0

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instead of using our custom HTTP client, using PycURL's multi
interface allows us to get rid of the HTTP client threadpool.
The majority of the code is still in the ganeti.http.client
module.

A simple per-thread HTTP client pool gives cURL a chance to
cache and retain as much information as possible (e.g. SSL certs).
Unused HTTP clients (e.g. due to removed nodes) are deleted after
25 requests going through the pool.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch converts hv_kvm to use utils.ShellWriter for writing
the network script. It also adds a few unittests (the first
for any hypervisor modules).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

By splitting the _WaitForJobChangesHelper class into multiple smaller
classes, we gain in several places:

- Simpler code, less interaction between functions and variables
- Easy to unittest (close to 100% coverage)
- Waiting for job changes has no direct knowledge of queue anymore (it
  doesn't references queue functions anymore, especially not private ones)
- Activate inotify only if there was no change at the beginning (and
  checking again right away to avoid race conditions)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… or on distributions which simply have other implementations of man,
that do not support '--warnings'.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

This patch moves network utility functions to a dedicated module.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The big shell fragment is just noise, for the common case where it
doesn't fail.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This needs lots of work, but it can successfully launch an LXC-based
instance.  See the docstring for the limitations/work to be done.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Support IPv6 configuration for 'drbdsetup show' parser and add unittests
concerning that case. Renames some data files to use consistent names
clarifying their usage.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should catch cases where we update the configuration version, but
forget to adjust cfgupgrade accordingly.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

With the recent addition of a check for directories listed in Makefile
local custom directories are always reported as unlisted. This patch
adds support for a “Makefile.local” file, which can adjust settings in
Makefile. Example: “DIRCHECK_EXCLUDE += xyz .mydata doc/manhtml”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It's easy to forget to add a new directory to DIRS. This check should
report such inconsistencies.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These files are being used to test the job queue performance with
various changes and conditions. Adding them here for posterity.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a workaround until we fully switched to user separation and fixes the
owners of directories/log files so ganeti-rapi will start flawlessly. This is
right now run for every daemon but as it operates on a relatively small subset
its impact is small.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>