If the disks= parameter is passed, we can assemble/wait for
sync/shutdown only some disks belonging to an instance, rather than all.

This is useful to only activate/sync/shutdown the affected disk when
growing it.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Backported from master, 72087dcd

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Same result, but what happens is clearer.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This bug was found in the asyncore master patch series, but actually
applies to 2.1 for AsyncUDPSocket as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a new boolean hypervisor parameter to the KVM hypervisor,
named 'use_chroot'.
If it's turned on for an instance, than KVM is started in "chroot mode":
Ganeti creates an empty directory for the instance and passes the path
of this dir to KVM via the -chroot flag.
KVM changes its root to this directory after starting up.

It also adds a "quarantine" feature for moving any unexpected files to
a separate directory for later analysis.

This has been backported from master,
commit 84c08e4e

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The convert= option of compat.tryToRoman is used to do optional
conversion without duplicating formatting code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The new TryToRoman function provides optional easy to use roman
conversion. Nunc cum demonstrationi unitati.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since the current start_timestamp opcode attribute refers to the inital
start time, before locks are acquired, it's not useful to determine the
actual execution order of two opcodes/jobs competing for the same lock.

This patch adds a new field, exec_timestamp, that is updated when the
opcode moves from OP_STATUS_WAITLOCK to OP_STATUS_RUNNING, thus allowing
a clear view of the execution history. The new field is visible in the
job output via the 'opexec' field.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Some confusion arose handling EINTR on this function: in python 2.6
socket.error is an IOError, and thus:
  - It's an EnvironmentError
  - It has an .errno member

In 2.4 and 2.5 it's not, and so its errno variable must be extracted
from the args tuple. This patch fixes both the function, and the
unittests.

This is a cherry-pick of master commit
965d0e5b with the portions not relevant
to 2.1 removed (changes to the RetryOnSignals function).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Tom Limoncelli <tlim@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Introduce 2 new hypervisor options, migration_bandwidth and migration_downtime
and implement KVM migration bandwidth and downtime control.

migration_bandwidth controls KVM's maximal bandwidth during migration, in
MiB/s. Default value is 32 MiB/s, same as KVM's internal default. This option
is a global hypervisor option.

migration_downtime sets the amount of time (in ms) a KVM instance is allowed to
freeze while copying memory pages. This is useful when migrating busy guests,
as KVM's internal default of 30ms is too low for the page-copying algorithm to
converge. This is a per-instance option, with a default of 30ms, same as KVM's
internal default.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

EnsureDirs() should create directories with the exact mode requested
in the arguments, but it currently applies the umask.
This patch makes it independent from the umask.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If the instance crashes between backend.InstanceReboot checks the list
of running instances and the execution of hv_xen.RebootInstance,
ini_info will be None. And if the instance doesn't reboot fast enough,
new_info will be None. Both cases lead to “TypeError: unsubscriptable
object”. Too bad pylint doesn't detect such cases.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since now we use the SingleFileEventHandler together with an error
handling asyncore dispatcher, we don't need the internal try/except
anymore.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This mixes AsyncNotifier with GanetiBaseAsyncoreDispatcher to provide an
AsyncNotifier which will log errors, rather than bail out.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Abstract a few common functionalities between all ganeti asyncore
dispatchers:
  - Handle errors by logging them, and then continue
  - By default check sockets only for readability

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is needed if daemons are in the foreground, and get ctrl+c-ed by
the user. Also add unittests to make sure the correct signals terminate
the mainloop.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The comments in the SingleFileEventHandler are still confd-specific.
Update them to make them generic for any single-file monitoring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Cut&Paste, plus the following changes:
  - The class is renamed to SingleFileEventHandler
  - The monitored filename must be passed in and doesn't default to the
    ganeti cluster config file
  - A small docstring is added to the class
  - Pylint disables for except: and method names are added

This makes it possible to write unittests for this class.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since we contributed AsyncNotifier back to the upstream pyinotify
project, we'll be able, one day, to remove the ganeti version of that
code. For now we still need it to support older distributions, but
having a note about when we'll be able to remove it is nice.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This exception is caught, but never thrown. It became useless when we
moved confd from on/off to enabled/disabled, but always running on all
nodes. Removing its definition and the code catching it can do no harm.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently the EOM terminator is hardcoded on the server side, and is
customizable in the Transport object (with the default being the same as
the value found in the server), but not in the luxi client.

With this patch we move the value to constants, and remove the "fake"
customizability, which would just break client/server communication. If
we ever need to have a luxi transport with a different terminator it's
easy enough to add it back.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will only work on patched or newer (>= 2.6.34) kernels and with a
patched version of qemu-kvm.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows users of the RAPI client to catch GanetiApiError for all HTTP
errors.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently they return a 500 Server Error, not really useful
for detecting nonexistent items.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Inter-cluster instance moves need the disk template. As they run
LUQueryInstanceData to get an instance's details, the disk template
must be returned.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

GetInstanceInfo should return the resource /2/instances/$name/info,
but so far it returns /2/instances/$name. The same applies to
GetNodeInfo, which returns /2/nodes/$name. Both names are stripped
of their “Info” suffix and a new method, GetInstanceInfo, is added
for /2/instances/$name/info. There's no equivalent for nodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Remove duplicate code between a couple of asyncore related function by
having a function in charge of handling EINTR errors. Unittests included.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

By overriding the default asyncore handle_error (which closes the
socket) with our own version, which logs what happened but tries to
proceed, we can get rid of a couple of try/except blocks. The resulting
churn is deindentation of the internal code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Disable warnings for:
- except Exception,
- use of __errno_location,
- redeclaration of handleError()

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix this pylint warning:
[W6501, Mlockall] Specify string format arguments as logging function parameters

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix typo spotted by pylint:
E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently this only supports the new instance creation request data
format version 1, but support for the old version can be easily
implemented.

Most arguments are optional and documented in the RAPI documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As mentioned in commit d975f482, the current way of creating an
instance via RAPI is not very flexible. With this patch, a new
instance creation request data format is introduced and documented.
Support can be detected by checking the list of features returned
by the /2/features resource for the value "instance-create-reqv1".

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>