GitLab

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This can be useful for debugging locking problems.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There are two major arguments for this:
- There will be more callbacks (e.g. for lock debugging) and extending the
  parameter list is a lot of work.
- In the jqueue module this allows us to keep per-job or per-opcode variables in
  a separate class. Instead of having to clean up the worker class after
  processing one job, these references will automatically go out of scope.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Small fix for a mistake done by bad editor settings.
Signed-off-by: Luca Bigliardi <shammash@google.com>

In case of complete failure results is empty, return immediately
(tnx unittests).
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Extend RunPhase so it will log hooks results in POST phase.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Run post phase of node-remove on the removed node as well.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksAbort is raised, but not documented.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Allow the caller of HooksMaster.RunPhase() to specify an alternative list of
nodes.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a basic version of LUMoveInstance. It doesn't yet
support iallocator-mode and it's implemented in old-style (non-TL) mode.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The mcpu.py:HooksMaster class needs to have a proc attribute/argument to
init in ordet to call its LogWarning method. However, this is available
from the 'lu' attribute, so we can remove this dependency.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This can be used for a 'plain' type instance when the underlying storage
went away, to recreate the storage (and reinstall) instead of removing
the instance and readding it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add an 'empty' logical unit to run hooks after cluster initialization.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a new opcode and lu for checking disk sizes. Currently
it does only top-level disk verification, and also doesn't check
primary/secondary node size mismatches (these two are added as TODOs in
the Exec() function of the LU).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It migrates all primary instances from the node to their secondaries.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a new (global) opcode flag 'dry_run' which, when True,
causes early exit from the LU workflow, returning a special value from
the LU object (initialized in the parent LogicalUnit class, and which if
not overriden from child LUs will be None).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

There were multiple issues:
  - copy-paste resulted in wrong indentation
  - wrong function name
  - missing spaces around assignment
  - overriding built-in names (type, dir) or already defines ones
    (errors, hypervisor)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This also converts (and fixes) unittests and mock objects to deal with
this change, and the custom hook verifier in cmdlib.LUClusterVerify.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This (somewhat big) patch adds support for remotely rebooting the nodes
via whatever support the hypervisor has for such a concept.

For KVM/fake (and containers in the future) this just uses sysrq plus a
‘reboot’ call if the sysrq method failed. For Xen, it first tries the
above, and then Xen-hypervisor reboot (we first try sysrq since that
just requires opening a file handle, whereas xen reboot means launching
an external utility).

The user interface is:

    # gnt-node powercycle node5
    Are you sure you want to hard powercycle node node5?
    y/[n]/?: y
    Reboot scheduled in 5 seconds

The node reboots hopefully after sending the reply. In case the clock is
broken, “time.sleep(5)” might take ages (but then I suspect SSL
negotiation wouldn't work).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch unifies the hardcoded re-encoding attempts into a single
function in utils.py. This function is used to take either an unicode or
str object and convert it to a ASCII-only str object which can be safely
displayed and transmitted.

We replace then the current manual re-encodings with this function. In
mcpu we stop re-encoding the hooks output and instead we do it right at
the hook generation in backend.py.

This passes on my 'custom' lvs output with non-ASCII chars. But there
are probably other places we will need to fix.

Reviewed-by: ultrotter

This is forward port via copy (and not individual patches cherry-pick)
of the latest code on the 1.2 branch related to the migration.

The changes compared to 1.2 are the fact that we don't need the
IdentifyDisks step anymore (the drbd rpc calls are independent now), and
the rpc module improvements.

Reviewed-by: ultrotter

This LU can be used to force a push of the config in case it's needed,
for example after an upgrade to update the ssconf_release_version file.

Reviewed-by: imsnah

This patch changes cluster verify to not alert on offline nodes, but
instead just show a note at the end with the number of such nodes.

It also removes warnings in verify-disks and hooks about failures to
make rpc calls to such nodes.

Reviewed-by: ultrotter

For a long time we had the problem that both RPC-layer errors and
results from the remote node share the same "valuespace". This is
because we shouldn't raise an exception when only one node failed
(and lose the results from the other nodes).

This patch attempts to address this problem by returning a special
object from RPC calls, which separates the rpc-layer status and the
remote results into different attributes.

All the users of rpc (mainly cmdlib, but also bootstrap and the
HooksMaster in mcpu) have been converted to this new model. The code has
changed from, e.g. for boolean return types:

  if not self.rpc.call_...

to

  result = self.rpc.call_
  if result.failed or not result.data:
     ^ rpc-layer error    |
                          - result payload

While this is slightly more complicated, it will allow cleaner checks in
the future; right now the code is just a plain port, without
optimizations.

There's also a "result.Raise()" which raises an OpExecError if the
rpc-layer had errors.

One side-effect of the patch is that now all return types from the
rpc.call_ functions are of either RpcResult (single-node) or dicts of
(node name, RpcResult); previously, some functions were returning
different object types based on error status.

The code passes burnin (after many retries :).

Reviewed-by: imsnah

This patch adds the OpCode, LogicalUnit and gnt-node command for
modifying node parameters, more specifically the master candidate flag
for a node.

Reviewed-by: imsnah

This is the only change needed to make mcpu epydoc-compliant.

Reviewed-by: ultrotter

As discussed previously, many of the routinges in cmdlib.py are using
logging functions as a carry-over from 1.2 (when these also showed the
message on stderr/to the user), instead of actually warning the user.

This patch extends the syntax for Processor.LogInfo/LogWarning in order
to be easier to use them.

Reviewed-by: imsnah

Reviewed-by: imsnah

This big patch changes the call model used in internode-rpc from
standalong function calls in the rpc module to via a RpcRunner class,
that holds all the methods. This can be used in the future to enable
smarter processing in the RPC layer itself (some quick examples are not
setting the DiskID from cmdlib code, but only once in each rpc call,
etc.).

There are a few RPC calls that are made outside of the LU code, and
these calls are left as staticmethods, so they can be used without a
class instance (which requires a ConfigWriter instance).

Reviewed-by: imsnah

Background: when we have multiple jobs in the queue (more than just a
few), many of the jobs (up to the number of threads) will be in state
'running', although many of them could be actually blocked, waiting for
some locks. This is not good, as one cannot easily see what is
happening.

The patch extends the opcode/job possible statuses with another one,
waiting, which shows that the LU is in the acquire locks phase. The
mechanism for doing so is simple, we initialize (in the job queue) the
opcode with OP_STATUS_WAITLOCK, and when the processor is ready to give
control to the LU's Exec, it will call a notifier back into the
_JobQueueWorker that sets the opcode status to OP_STATUS_RUNNING (with
the proper queue locking). Because this mechanism does not save the job,
all opcodes on disk will be in status WAITLOCK and not RUNNING anymore,
so we also change the load sequence to consider WAITLOCK as RUNNING.

With the patch applied, creating in parallel (via burnin) five instances
on a five node cluster shows that only two are executing, while three
are waiting for locks.

Reviewed-by: imsnah

sstore is no longer used in LUs.

Reviewed-by: iustinp

Replacing ssconf with configuration.

Reviewed-by: iustinp

This can be used to retrieve certain cluster config values from
within clients.

OpDumpClusterConfig was not used anywhere, hence I'm just reusing
it. The way ConfigWriter.DumpConfig returned the configuration
was not thread-safe, anyway (no deepcopy).

Reviewed-by: iustinp

With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be
removed again, unless some action is taken.

Reviewed-by: imsnah