After moving the documentation from the .py files to .rst, we had some
cleanups to do.

This fixes the formatting of the comments, improves them a little, and
removes deprecated info (DOC_URI) from the python source.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the hypervisor parameters are checked using hard-coded snippets in
each hypervisor. However, most parameter checks fall into three cases:
  - file check
  - directory check
  - string value in a set

And the remaining ones are checked using simple functions.

This patch moves to a declarative-style for these parameter checks; in
hv_base we add the necessary infrastructure for these checks, and the
above common cases.

This translates into complete removal of the Check/Verify functions for
the Xen hypervisors, and a drastic reduction for the KVM one (which has
inter-parameter dependencies and thus can't use a simple table).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* commit 'origin/next': (25 commits)
  Move more hypervisor strings into constants
  Add -H/-B startup parameters to gnt-instance
  call_instance_start: add optional hv/be parameters
  Fix gnt-job list argument handling
  Instance reinstall: don't mix up errors
  Don't check memory at startup if instance is up
  gnt-cluster modify: fix --no-lvm-storage
  LUSetClusterParams: improve volume group removal
  gnt-cluster info: show more cluster parameters
  LUQueryClusterInfo: return a few more fields
  Add the new DRBD test files to the Makefile
  Remove some superfluous imports
  Make Python interpreter selectable for test scripts
  Pass optional arguments to the daemons
  ganeti.initd: include defaults file, if present
  Fix ;; indentation in the main initd loop
  Avoid DeprecationWarning on Python >= 2.6
  ganeti-noded: add bind address option
  Fix compatibility with DRBD 8.3
  Fix compatibility with DRBD 8.2
  ...

This patch adds constants for the mouse and boot order strings; while
there are still some issues remaining, we're trying to cleanup hardcoded
strings from the hypervisors.

Since the formatting of frozensets is currently wrong, we also add an
utility function for this and change all the error messages to use it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Bugs in either our code or in associated libraries can bring the master daemon
down, and this (due to the 2.0 architecture) stops all work on the cluster.

Since the watcher already does periodic checks on the cluster, we modify
it to try to start the master automatically in case of failures to
connect. This will be tried only once per cycle.

Also, in this case, we modify the code so that the watcher status file
is not updated - its timestamp will reflect thus the time of last
successful connection to the master.

Side note: the except errors.ConfigurationError part could be cleaned
up, since in 2.0 we don't usually get that directly, and if we do it's
an error and we shouldn't touch the file anyway; but that is not a rc5
change.

Signed-off-by: Iustin Pop <iustin@google.com>

This patch adds for current instance a ‘disk_space_total’ key, similar
to the key for the new instance in case of new allocations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch modifies the start instance script, opcode and logical unit
to support temporary startup parameters.

Different from 1.2, where only the kernel arguments were supporting
changes (and thus xen-pvm specific), this version supports changing all
hypervisor and backend parameters (with appropriate checks).

This is much more flexible, and allows for example:
  - start with different, temporary kernel
  - start with different memory size

Note: in later versions, this should be extended to cover disk
parameters as well (e.g. start with drbd without flushes, start with
drbd in async mode, etc.).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch modifies the rpc.call_instance_start - the master side - to
take optional hv/be parameters. The noded side is unchanged and
oblivious to the change.

This will allow implementation of single-user capability and such on
startup (temporary, as opposed to permanent).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently QueryJob returns "None" when a wrong job ID is passed.
Handle this in gnt-job list, by printing an error for each wrong job,
and still giving output for all the jobs which actually do exist.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If the remote info rpc call fails we can't assume that the instance is
up.

Signed-off-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Initial design for the OS parameter changes proposed for 2.1.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This moves yet another hardcoded value to a hypervisor parameter. I
removed the 64/32 difference as it doesn't seem valid to me - it's more
of a local site config rather than arch config.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

For the xen-hvm hypervisor, the KERNEL_PATH parameter is needed but
today is hardcoded to a constants in the xen hypervisor library (argh!).

This patch moves this to a hypervisor constant with the default value
being the current hardcoded path. This will allow cluster/instance
customisation based on the installed xen version.

This should fix Debian bug #528618.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch proposes a mini-design to improve redistribute-config and
integrate it better with other logical units.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently doing a gnt-cluster-modify --no-lvm-storage is silently
ignored, as it passes a None value in vg_name, which is the same as not
modifying that parameter. Explicitely set the passed value to '', so the
non-true not-None value can be evaluate to actually remove a volume
group.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently LUSetClusterParams will remove the volume group if the vg_name
field passed in is not true, but not None. Setting the target volume
group to False or the empty string, though, is a bad idea because it's
not a boolean value, and at cluster init we set it to None if
--no-lvm-storage is passed. With this fix we handle '' (or any other
non-None false value) as the "unset" value, but actually store None in
the config.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Even if we cannot modify all of them, they are useful information about
the current cluster.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some fields can be set at cluster init, and perhaps even modifed with
SetClusterParams but there's no way to know them. With this patch we
export them in the cluster info query.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If a change modifies the way all/most LUs work it should also be
considered core.

Signed-off-by: Guido Trotter <ultrotter@google.com>

This removes some code which was duplicated in shutdown and migrate.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We change some formatting to sphinx-specific, to show how the
documentation can be improved.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we have an insignificant glossary at the end of the design-2.0
document. This patch moves it to a separate file with the goal that it
will grow and all files can refer to it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the KVM hypervisor returns strings for the memory and cpu
values, while the xen hypervisor returns integers. Making this uniform
converting the values to integers in KVM as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUSetInstanceParam currently assumes that the 'memory' value of a
call_instance_info result is an integer, while the rest of the code
explicitely converts it to int(). Converting it to int works around a
bug which prevents changing the memory allocation of a live instance if
the remote call returns the memory in string format.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This big patch converts the documentation build system to sphinx
(http://sphinx.pocoo.org/

). Since that uses reStructuredText sources
too, there is no change (yet) in the documents themselves, just in the
build system.

As before, the docs are pre built by the maintainer, and the end-user
doesn't need sphinx or other rst tools to build the docs. Note that we
are not distributing PDFs, so building that will require the tools.

The docs will be stored under doc/html and the build system also need an
extra directory doc/build. These are considered (by automake)
maintainer-related objects and are removed at maintainer-clean time.

The patch also fixes some small issues: add a docpng variable, add
doc/api (also generated by maintainer) in maintainer-clean-local, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch removes the autogeneration of the RAPI docs from the code
(based on docstrings) and moves the current autogenerated output to
the rapi.rst file.

The reasons behind this are multiple:
  - the build system becomes a little more simple (this could have been
	achieved also by distributing the built documentation, though)
  - it's hard to actually write documentation in docstrings; you have to
	fit restructured text inside the docstrings, and this results in
	not really nice output
  - even by being close to the code, the documentation manages to get
	out of sync (not paying attention to docstrings)

This will also help with the move to sphinx.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These were forgotten in commit 01e2ce3a,
and caused “make distcheck” to fail.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In Ganeti 1.2, “none” was used to signify no initrd. In 2.0 we have
changed to “no_” as a prefix (i.e. “-H no_initrd_path”) and thus we
document in the manpage this.

The QA suite is changed accordingly.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The _TransformPath function is not used anymore in 2.0, let's remove it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Patch for adding network_port to the instance attributes exported by the
RAPI.

[iustin@google.com: slightly changed the formatting]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Minor patch to clarify the URL necessary for accessing the RAPI.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The version is 2.0, and we don't build PDFs by default, only HTML
files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is for Python 2.6 compatibility.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The Python interpreter used to run the test cases is hard-coded to be
/usr/bin/python. If we use the first one from $PATH instead, it is
much easier to test ganeti with other Python versions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sometimes reinstalls are slightly different than new installs. For
example certain partitions may need to be preserved accross reinstalls.
In order to do that on a per-os basis we pass in the INSTANCE_REINSTALL
variable to inform the create script about when a reinstall is
happening.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This document contains a skeleton for the 2.1 design process.
For now it just has introductory paragraphs and a structure for the
various areas' design, but some sections still don't have a text, as
we're still in the early design phases.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>