GitLab

Update cluster-verify to check the integrity of the default master IP
address setup script and the presence and executability of the external
one (if currently in use by the cluster).
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- add a command line flag to gnt-cluster init and modify to change the
  value of the cluster parameter use_external_mip_script;
- add two constants representing the paths of the default script and of
  the external script;
- add documentation for the new option in the gnt-cluster man page;
- add use_external_mip_script to gnt-cluster info output.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add the use_external_mip_script cluster parameter, that represents
whether the master IP address turnup/turndown procedures must use a
script provided by the user (True) or the one provided by Ganeti
(False).
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Until now LUNodeSetParams would lock all instances if a node's
secondary IP address was to be changed and would then release
all instances it didn't actually need. With this patch the LU
optimistically locks instances and, once it got the locks,
checks whether they're still correct. This is similar to how
node group locking is done.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- check if the master IP with the old netmask is up before attempting to
  change the netmask (to avoid a failed change netmask resulting in an
  undesired activation of the master IP);
- improve error messages of the backend function;
- in case of error, report the problem but otherwise change the cluster
  master_netmask parameter;
- remove duplicate error feedback.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is needed to lock node resources.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Pass instances of objects.MasterNetworkParameters when calling RPCs for
activation and deactivation of master IP.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of manually unpacking the return values of
cfg.GetMasterNetworkParameters, let it return an instance of
objects.MasterNetworkParameters and pass its attributes.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add the master IP family parameter to the master IP deactivation RPCs,
so that the activation and deactivation interfaces are uniform.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Make the master explicitly pass the parameters to the
change_master_netmask RPC, and change all the call flow to use the new
interface.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Make the master explicitly pass the parameters to the
deactivate_master_ip RPC, and change all the call flow to use the new
interface.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

To remove the usage of ssconf in the backend, the master needs to push
the parameters of activate_master_ip to the backend.

This patch changes the entire call path of activate_master_ip to use the
new interface.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- remove any dependence on Logical Units from the HooksMaster;
- add a new function parameter to the constructor, a function that is
  expected to convert the results of the hooks execution in a format
  understood by the HooksMaster;
- add a factory method that builds a HooksMaster from a LU, keeping the
  interface of the old constructor of HooksMaster;
- remove usage of Processor.hmclass from external classes, introducing
  the Processor.BuildHooksMaster method;
- update unit tests.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When a node is marked as online (“gnt-node modify -O no …”), an RPC is
made to the node to check whether the node daemon is running. My recent
RPC changes led to offline nodes being ignored before the actual call is
made, thereby causing this check to fail. This is a workaround and needs
to be cleaned up later.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of filling the parameters in the RPC layer, that is now done
before the wrapper is called, thereby simplifying the wrapper.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Interface changes were necessary as these took more parameters than were
actually passed over the wire. Those parameters were just passed to the
instance serialization function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

… instead of a hardcoded value.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

All other RPC wrappers take the node name(s) as the first parameter.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Rather than just allowing files for all nodes to be optional, we allow
optional files to be per-category. The way this works is that they must
be included in both lists (the new code checks for this).

The code also removes a duplicate assert (present both in verify and
compute ancillary files) and checks the new functionality in unittests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These lists will be used to declare some of the files not necessarily
needed on all nodes. The files selected are files without which the
various hypervisors can still work, but that when they are present
should be synchronized across the cluster (or node group).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If an instance had actually a missing disk, the type check would fail.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We currently use 'filter' as the OpCode, QueryRequest and RAPI field
name for representing a query filter. However, since 'filter' is a
built-in function, we actually have to use filter_ throughout the code
in order to not override the built-in function.

This patch simply goes and does a global sed over the code. Due to the
fact that the RAPI interface already exposed this field, we add
compatibility code for now which handles both forms.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

lib/constants.py
* add to each CV_E* tuple the documentation of the error code
* add the DOCUMENTED_CONSTANTS constant for the doc preprocessor

autotools/docpp
* add a new directive class CONSTANTS_<kind>, that gets data from
  constants.DOCUMENTED_CONSTANTS

lib/cmdlib.py
* modify the code that unpacked the CV_E* tuples to ignore the
  documentation parameter
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Treat the gnt-cluster verify errors identified by the error codes in
--ignore-errors as warnings; just print a warning message for the user.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

lib/cli.py
- add IGNORE_ERROR_OPT;

client/gnt_cluster.py
- pass the ignore_errors parameter to the opcodes

lib/opcode.py
- update OpClusterVerifyConfig, OpClusterVerify and OpClusterVerifyGroup
  to accept the ignore_errors parameter

lib/cmdlib.py
- pass the ignore_errors parameter to the opcodes that need it
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- move the cluster verify error codes from cmdlib._VerifyErrors to
  constants;
- add to each of them the CV (Cluster Verify) prefix;
- add the CV_ALL_ECODES and CV_ALL_ECODES_STRINGS constants;
- wrap the lines that exceed 80 characters after changing the error
  code names to the new ones.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add the master_netmask cluster parameter, that represents the netmask of
the master IP, encoded as a CIDR suffix.

This parameter can be set via the --master-netmask of gnt-cluster init
and gnt-cluster modify. The default behaviour is to be consistent with
the old default (/32 for IPv4 and /128 for IPv6).
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If a cluster has any non-master-candidate nodes, those don't contain all
files (e.g. config.data). With commit aef59ae7

 (March 31st, 2011)
the logic was changed and subsequently verifying a cluster with non-mc
nodes would complain.

This patch fixes this issue by changing the algorithm. It also adds an
additional check for files which shouldn't exist on a machine. A newly
added unittest is included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 64c7b383

 changed the RPC call for verifying SSH connections.
Unfortunately this case in adding nodes was missed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When verifying a group the code would always check SSH to all nodes in
the same group, as well as the first node for every other group. On big
clusters this can cause issues since many nodes will try to connect to
the first node of another group at the same time. This patch changes the
algorithm to choose a different node every time.

A unittest for the selection algorithm is included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

lib/client/gnt_cluster.py:
* Add activate-master-ip and deactivate-master-ip commands

man/gnt-cluster.rst:
* Document the new commands

lib/opcodes.py lib/cmdlib.py
* Add two opcodes and the LU that call the relevant RPCs

test/docs_unittest.py
* Silence an error about RAPI not implemented for the two new opcodes
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit fb926117)

Conflicts:

	test/docs_unittest.py
	  - kept devel-2.5 version, without the RAPI opcode checks

lib/backend.py
* split StartMaster() in ActivateMasterIp() and StartMasterDaemons()
* split StopMaster() in DeactivateMasterIp() and StopMasterDaemons()

lib/server/noded.py, lib/rpc.py
* adapt the call chains to the new functions, define new RPCs

lib/bootstrap.py, lib/cmdlib.py, lib/server/masterd.py
* use the new RPCs
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit fb460cf7)

lib/client/gnt_cluster.py:
* Add activate-master-ip and deactivate-master-ip commands

man/gnt-cluster.rst:
* Document the new commands

lib/opcodes.py lib/cmdlib.py
* Add two opcodes and the LU that call the relevant RPCs

test/docs_unittest.py
* Silence an error about RAPI not implemented for the two new opcodes
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

lib/backend.py
* split StartMaster() in ActivateMasterIp() and StartMasterDaemons()
* split StopMaster() in DeactivateMasterIp() and StopMasterDaemons()

lib/server/noded.py, lib/rpc.py
* adapt the call chains to the new functions, define new RPCs

lib/bootstrap.py, lib/cmdlib.py, lib/server/masterd.py
* use the new RPCs
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* hypervisor/hv_kvm.py
  - parse the memory transfer status

* cmdlib.py
  - represent memory transfer info, if available
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

To add status reporting for the KVM migration, the instance_migrate RPC
must be non-blocking. Moreover, there must be a way to represent the
migration status and a way to fetch it.

* constants.py:
  - add constants representing the migration statuses

* objects.py:
  - add the MigrationStatus object

* hypervisor/hv_base.py
  - change the FinalizeMigration method name to FinalizeMigrationDst
  - add the FinalizeMigrationSource method
  - add the GetMigrationStatus method

* hypervisor/hv_kvm.py
  - change the implementation of MigrateInstance to be non-blocking
    (i.e. do not poll the status of the migration)
  - implement the new methods defined in BaseHypervisor

* backend.py, server/noded.py, rpc.py
  - add methods to call the new hypervisor methods
  - fix documentation of the existing methods to reflect the changes

* cmdlib.py
  - adapt the logic of TLMigrateInstance._ExecMigration to reflect
    the changes
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

* hv_kvm.py, hv_xen.py
  - return the hypervisor version (if available) from GetNodeInfo

* cmdlib.py
  - if hypervisor version is available during the migration, and the
    versions differ, warn the user
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The change to enforce boolean results for cluster verify group opcode
missed the HooksCallBack, which uses a very ugly 1/0
logic. Furthermore, the logic is wrong, since it unconditionally
resets the verify result to true.

The patch is changed to simply treat hook failures as failures, and do
nothing for offline/nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>