This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

As mentioned in commit d975f482, the current way of creating an
instance via RAPI is not very flexible. With this patch, a new
instance creation request data format is introduced and documented.
Support can be detected by checking the list of features returned
by the /2/features resource for the value "instance-create-reqv1".

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add Mlockall() utility to lock current process' virtual adress space into RAM.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The RAPI client module shouldn't depend on any Ganeti module, yet it's
useful to have some Ganeti-specific code, like a PollJob function for
RAPI.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

utils.py, where they were before, is already huge.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

distcheck, lint and apidoc. Take three pay only one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 5fa16422 removed the checks at sed time, but only for %.7. We
need to do it for %.8 too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

*.py[co] were not removed from lib/confd. Removing *.py[co]
from all directories listed in $DIRS can't hurt.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

2.1's cfgupgrade doesn't support upgrading from 1.2, only from 2.0.
However, it's trivial to forward-port the 2.0 cfgupgrade to work with
2.1, thus providing an upgrade path for 1.2 users directly to 2.1,
without the intermediate step of installing 2.0.

The new cfgupgrade12 file has just a few trivial changes from 2.0, and
skipping the load of the config file.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This can be used for two purposes:

- safety copy of the config file, with just the secrets changed
- cleanup of the config file (full randomization), so that (e.g.) users
  could send a broken config file to the devel-list

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is to keep same naming across all tests (modules separate with dot,
followed by _unittest.py).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: David Knowles <dknowles@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
(modified slightly the unittest to account for
 missing httplib2 library)

First, we don't need to check man pages at sed time, because this means
everyone building the package will do - we only need to check at docbook
time, which is mostly at developer time.

Second, don't force LC_ALL to C, as this breaks newer man-db. I've
verified and removing LC_ALL works fine across etch, hardy, lenny aid
squeeze/sid.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This test doesn't cover everything, but it's better than nothing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.
pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and
“notAfter” as strings. Parsing them turned out to be a challenge since
they can be in a variety of formats (YYYYMMDDhhmmssZ, YYYYMMDDhhmmss+hhmm
or YYYMMDDhhmmss-hhmm).

This will be used to verify the validity of cluster certificates in
LUVerifyCluster.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Rationale: the file-based storage backend can add/remove files under a
certain directory. However, the master node is also controlling the
setting of the file-based root directory, so basically it means we can't
prevent arbitrary modifications by the master of the node's filesystem.

In order to mitigate this for setups where the file-based storage is not
used, we introduce a new setting at ./configure time, that controls the
enable/disable of file-based storage. Since this is not modifiable by
the master (over RPC), it is now possible in this case to prevent
unintended modifications of the node's filesystem from the master.

The new setting is used in bdev.py to not expose the file-based storage
at all, and in cmdlib.py to prevent attempts at creation of such
instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds ability to customize ssh config directory with --with-ssh-config-dir
(instead of hardcoded /etc/ssh value). This is useful in Linux distributions with
custom ssh config directories (/etc/openssh in ALTLinux, for example).

Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is the implementation of docs/design-cluster-merger.rst. It allows
the automatic merging of one or more clusters into the invoking cluster.

While this version is tested and working it still needs some tweaking
here and there for error handling and user experience.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

According to “coverage”, this covers 99% of the code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

On fork, the tempfile module's pseudo random generator is
not reset. If several processes (e.g. two children or parent
and child) try to create a temporary file, they'll conflict.
This function can be used to reset the name generator which
contains the pseudo random generator.

A unittest is included. It is in a separate script because
it changes a variable in the tempfile module to speed up the
test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit 83d9f436.

man is still unable to wrap some long lines, so we simply revert this patch
(and filter out the specific message in autotools/check-man).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

A long PREFIX variable (to configure) will result in very long
LOCALSTATEDIR, which when concatenated with lib/ganeti/ (and even more
items under it) will go over the 80 char line length we enforce in the
man checker.

To workaround this, we change two things:

- use a specific REPLACE_VARS_MAN which adds breaking points after each
  slash in paths
- replace some <filename> entries with <literallayout> so that docbook
  generates a non-fill block around them (only a few cases need this
  after the breaking points are added

Note that with normal prefixes (e.g. / or /usr/local) this won't happen.

The patch also fixes a wording in the watcher man page.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

One fix is necessary in gnt-cluster.sgml. Also adding “DELETE_ON_ERROR”
target to remove output file if an error occurred while building it (in
this case the manpage).

This was reported by Iustin Pop in issue 87 and proposed check method
taken from Lintian.
http://code.google.com/p/ganeti/issues/detail?id=87



Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a configure-time parameter that will set the defaults
used by all programs, and command-line parameters in the daemons that
allow overriding it.

Syslog 'yes' enables syslog in addition to file-based logging, 'only'
enables syslog and disables file-based syslog.

The log entries will be of the form:
Jan 27 08:45:04 node2 ganeti-noded[14504]: INFO 172.24.227.5:50850 PUT
/jobqueue_update HTTP/1.0 200
Jan 27 08:45:05 node2 ganeti-noded[14505]: INFO 172.24.227.5:50853 PUT
/lv_list HTTP/1.0 200

and (for a multi-threaded program):
Jan 27 08:51:48 node1 ganeti-masterd[15491]: (MainThread) INFO
ganeti-masterd daemon startup
Jan 27 08:51:49 node1 ganeti-masterd[15491]: (MainThread) INFO
Inspecting job queue

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Ideally we want to/will have per-device DRBD controls of disk/metadata
flushes. In the meantime, we want at least a disable of the barrier
functionality for cases where one has battery-backed caches.

Background: DRBD has four mechanism of handling ordered disk-writes.
From the drbdsetup man-page, these are: barrier, flush, drain and none.
DRBD prior to 8.2 only has drain and none. This patch makes all 8.x
versions of DRBD disable all methods, and revert to none, in case one
fully trusts batteries (either UPS for the whole system or battery for
NVRAM).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Look it up at configure time instead.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This partially reverts commit e4e7c7df,
which made newer pylint versions fail.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

That is, all Python code except unittests. In particular,
autotool/build-bash-completion was missing from the
previous list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows automated builders to override the format, for example.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This runs pylint over all the python files. Yes, it takes a long while,
but it's the only way to properly analyze the source codes as only in
this way pylint can see all uses of the various modules/classes/etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This adds targeted pylint disables, but since _autoconf.py is
autogenerated we need to do this in Makefile.am (hence the separate
patch).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Patch 4352bf6d changed RAPI docs to RST from the previous SGML format
(the last file to be converted to RST).

However, it didn't add rapi.rst to the docrst Makefile variable, and as
such rapi.html was neither build nor shipped in the archive. We fix this
and we also distribute doc/rapi-resources.gen since otherwise “make
distcheck” has issues (since this file is built and thus lives in _built
whereas rapi.rst is shipped and lives in topsrcdir).

Note: yes, this means that no 2.0 release until now had a rapi.html file
(in the .tar.gz archive).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Comparing a number with None is not a good idea:

  >>> (0 < None, 0 > None)
  (False, True)

This patch also adds build-bash-completion to the list
of checked Python scripts and wraps one line of more
than 80 characters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>