The import/export daemon code is already large. Moving some code
to a separate module will make it smaller and easier to test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Create a new _MyErrorLoggingAsyncNotifier class which registers
  error counts, rather than logging them
- Add an additional ERR notifier to test with
- Check that no error was returned, for tests that weren't supposed to
- Add a new test case for a callback that's supposed to raise an
  exception

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This mixes AsyncNotifier with GanetiBaseAsyncoreDispatcher to provide an
AsyncNotifier which will log errors, rather than bail out.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Rather than hardcode that we have two notifiers, and notifier 0 is the
terminating one, we abstract this with class level constants. This makes
it easier to add more, with different features.

The only real change is that now the callback class takes as input the
whole test object, rather than just the notified array, to have access
to those constants.

The rest is just replacing of hardcoded 0s and 1s with
self.NOTIFIER_TERM and self.NOTIFIER_NORM, and of notifier_count with
len(self.NOTIFIERS).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

This is needed if daemons are in the foreground, and get ctrl+c-ed by
the user. Also add unittests to make sure the correct signals terminate
the mainloop.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For example, exports on the same node shouldn't be compressed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

By using enterabs we can schedule events at the same time, which will
then be sorted by priority.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, in testDeferredCancel if the self._CancelEvent are entered
more than 0.3 seconds after the _SendSig have been entered, the test
could fail. This is unlikely but may happen. To avoid it we use
enterabs, to use absolute times.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently we never do that, but it's nice to know we can restart the
mainloop, after termination.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Two new resources are added:
- /2/instances/$name/prepare-export
- /2/instances/$name/export

The documentation for the existing resource for creating instances is updated
for remote imports. The RAPI client is extended for the new resources.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

To prepare a remote export, the X509 key and certificate need to be generated.
A handshake value is also returned for an easier check whether both clusters
share the same cluster domain secret.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Importing/exporting an instance to a remote machine creates X509
certificates which expire after some time. They need to be removed from
the nodes as they become useless.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

GetInstanceInfo should return the resource /2/instances/$name/info,
but so far it returns /2/instances/$name. The same applies to
GetNodeInfo, which returns /2/nodes/$name. Both names are stripped
of their “Info” suffix and a new method, GetInstanceInfo, is added
for /2/instances/$name/info. There's no equivalent for nodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove duplicate code between a couple of asyncore related function by
having a function in charge of handling EINTR errors. Unittests included.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently this only supports the new instance creation request data
format version 1, but support for the old version can be easily
implemented.

Most arguments are optional and documented in the RAPI documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As mentioned in commit d975f482, the current way of creating an
instance via RAPI is not very flexible. With this patch, a new
instance creation request data format is introduced and documented.
Support can be detected by checking the list of features returned
by the /2/features resource for the value "instance-create-reqv1".

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The /2/features RAPI resource can be used to detect optional
features implemented by the RAPI server. This will be used
to recognize servers implementing a new request format for
instance creation requests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add Mlockall() utility to lock current process' virtual adress space into RAM.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Read the first non-empty file line. When strict is set, abort if more
than one line is non-empty. Some unittests inspired by the reverted ones
from commit b774bb10, and some new ones.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This partially reverts commit b774bb10.
Unittests unrelated to that particular functionality but introduced in
that commit are left untouched. Since the temporary directory is now
only used to check for a non-existing file, it is removed as well, and
a convenient very-very-unlikely-to-exist file is used instead.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Only the server knows which node roles can be set via RAPI.
Constants are provided for convenience.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- The RAPI client shouldn't check the replacement mode as only
  the server knows which ones are valid (constants are still
  provided for convenience)
- Disks shouldn't be a list of names, but of indexes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

When modifying a storage unit, the “allocatable” flag should default
to “no modification”. This replicates the behaviour of the command
line interface.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Boolean values must be passed as 0 or 1. None should be an empty
value ("").

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Only the RAPI server (actually masterd) knows which storage types
are valid. The exception can no longer be raised as the type is
only checked in the job.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Having the HTTP error code allows users of the class to act differently based
on the error code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

This allows testing error conditions (e.g. if a job wasn't found).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Jobs can't be deleted, but cancelled (even though the HTTP method
“DELETE” is used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

- Replace hardcoded values with constants
- Code formatting
- Exception messages without periods and fixed string formatting

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

Even removing tags returns a job ID.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>

- It's possible to implement all functionality in ganeti.rapi.client
  using Python's standard modules httplib and urllib2
- By doing so, proper SSL certificate verification is implemented
- Adjust some of the code to Ganeti's code style (this is not yet
  finished)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Reading only one line is useful for reading passwords from files. This
also adds unittests for utils.ReadFile.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>