Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per issue 90, current cluster verify is very very brittle. It's one of
the oldest pieces of code, with only additions without cleanups over the
last years.

Among its problems:

- data initialization interspersed with verification of RPC results,
  leading to non-initialized data for some branches
- due to the above, we order strictly some checks and we have the case
  where a bad node time result will skip checking of node volumes
- many many local variables, with each new check adding a new dict,
  leading to a spaghetti of dicts in the main Exec function
- monolithic code, both Exec() and _NodeVerify() do a lot of
  independent checks

This patch does an imperfect rewrite, but at least we gain:

- a clear infrastructure for adding more checks (the new NodeImage
  class, with it's clear and documented fields), and removal of most
  per-node dicts from the Exec() function
- the new NodeImage object should allow better type safety, e.g. by
  allowing pylint to check the actual object attributes rather than
  strings as dict keys
- a-priori initialization of data fields, eliminating the need to
  introduce dependencies between checks
- per-result-key status field, allowing elimination of duplicate error
  messages (where we want)
- split of most independent checks into separate functions, for greater
  clarity

The new code, being new will probably introduce for the short term more
bugs than it removes. However, it should offer a much better way for
extending cluster verify in the future.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This option type enforces its value to either True or False, relieving
the scripts from manually parsing the values in each function.

We also update the bash completion code to use the option type if
possible.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In case LVM is broken, backend.GetVolumeList will raise an RPC exception
(as expected since it's a function exposed over RPC). Therefore we must
be prepared to catch any such exceptions, so that we don't fail the
whole verify call in this case. cmdlib is already prepared to handle
string results for this response key.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also fixed a typo I noticed.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Rather than checking that the file doesn't exist, and then creating it,
we create it with O_CREAT | O_EXCL, making sure the checking/creation is
atomic.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently if we find a live process with the pid we saved we assume kvm
is alive. What could happen, though, is that the pidfile has been
reused.

In order to avoid that we change the check to make sure, everywhere,
that the process we see is our actual kvm process. In order to do so we
open its cmdline, and check that it contains the correct instance name
in the -name argument passed to kvm.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit bd561702.

Turns out our and python's any/all are not compatible.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

All non-oneliner functions, after this patch, have their docstring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Just a few queries are checked, but this should give us confidence that
at least the basic confd framework is working properly.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function allows receiving socket data synchronously.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Rename WaitForSocketCondition to SingleWaitForFdCondition
  - Avoid potentially infinite loop, if we continue to get interrupted
  - Handle eintr correctly
  - Avoid the poller try/finally, as the poller object gets destroyed
    anyway
- Provide a new WaitForFdCondition
  - Using retry, guarantee to continue checking until the timeout
    expires
  - Needs an extra helper class, as it uses retry in a very custom way
    (no sleep happens, because the poller sleeps by itself)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This new confd callback counts received replies for the registered
queries.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

By sending requests with async=False, and receiving replies with
ReceiveReply we can more easily use confd from a synchronous client.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

@keyword was used inappropriately.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This basically implements read handling, without catching all
exceptions. When using the socket in synchronous mode, it's useful to
avoid losing exception data (which, in an async daemon, can only be
logged)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If any or all are already defined (because we're using a new version of
python) just link them inside "utils" rather than redefining them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Pass keyword parameter as such
- Replace “not x == y” with “x != y”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch introduces the option “--use-replication-network” for the
cluster copyfile functionality, which is useful if the primary and
secondary network are significantly different (see issue 32).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch allows GetOnlineNodes to return the secondary IPs instead of
the node names, and to provide filtering of the master node (required to
be done in this function in case we return the secondary IPs).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is a simple patch that adds the no-install mode for instance
creation, allowing import from foreign source of the actual OS (instead
of requiring the preparation of data in a form expected by the import
scripts).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch modifies LUSetInstanceParms to allow OS name changes, without
reinstallation, in case an OS gets renamed on-disk.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch moves the node-has-os checks to a separate function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Report exception text immediately instead of just logging it
- Remove leftover assertion from when it still used “gnt-cluster
  modify”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The current check on whether we require auto_promote or not is wrong, as
we check whether we will have exactly the correct number of master
candidates left. But it is fine if we have more (e.g. when CPS=10 and
mc_remaning=19) than the current number, and in that case we shouldn't
require auto promotion.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” or
simply “HMAC key” everywhere. With the implementation of inter-cluster
instance moves, another HMAC key will be introduced for signing critical
data. They can not be the same, so this patch clarifies the purpose of the
“cluster HMAC key” by renaming it. The actual file name is not changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

To be consistent with RAPI_CERT_FILE, the rather generic named
“SSL_CERT_FILE” constant is renamed to “NODED_CERT_FILE”. The actual file
name is not changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As described in issue 93, just saying ':' is not a valid char can be
confusing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The new test depends on the drbd type tests being enabled, and test
conversion to plain and back to drbd.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is much simpler than the opposite, with fewer possibilities of
failures.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a new mode to instance modify, the changing of the disk
template. For now only plain to drbd conversion is supported, and the
new secondary node must be specified manually (no iallocator support).

The procedure for conversion works as follows:

- a completely new disk template is created, matching the count, size
  and mode of the instance's current disks
- we create manually (not via _CreateDisks) all the missing volumes
- we rename on the primary the LVs to the new name
- we create manually the DRBD devices

Failures during the creation of volumes will leave orphan volumes.
Failure during the rename might leave some disks renamed and some not,
leading to an inconsistent instance.

Once the disks are renamed, we update the instance information and wait
for resync. Any failures of the DRBD sync must be manually handled (like
a normal failure, e.g. by running replace-disks, etc.).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Multiple LUs require that an instance is not running while they operate
on the instance (reinstall, rename, modify, recreate disks, deactivate
disks). The code to do this check is duplicate many times, and not very
consistent (some use call_instance_list, some call_instance_info).

The patch moves this check into a separate function that is then reused.
The only drawback is that _SafeShutdowInstanceDisks now raises an
OpPrereqError (even though it is run during Exec()), but this use case
is fine (there are no other modifications in that Exec).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Both create instance and grow disk check the free disk space on nodes
using the same, duplicate code. Since we'll need this in other places in
the future, we abstract the check into a new function.

The patch adjusts the error message to be more in-line with the one for
memory checking, and fixes the exception raised for RPC errors.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>