GitLab

Before, calls to `gnt-instance list -o console` with an instance on a
node with a custom SSH port failed because of missing group
configuration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The test is turned on by setting `"group-custom-ssh-port": true` in
the config.

Unlike `"ssh-port": ...` in "options", this is a single test that
doesn't affect the rest of the tests.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This is needed for testing SSH ports so that we can remove a node from
a group, add it to a group with a custom SSH port, and then undo the
change.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

For acquiring nodes use `with AcquireManyNodes(num): ...`. The nodes
will be released automatically.
For creating a new group use `with NewGroupCtx() as group: ...`. The
group will be removed automatically.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

If SSH ports other than 22 are configured, abort the config downgrade,
as the operation would certainly fail. Otherwise just remove
_ndparams/ssh_port_ from the configuration.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Added to the other node parameters in ganeti.rst.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This patch adds the `ssh-port` option. If set to a non-standard port,
the QA script sets up the default node group with this port, and before
running tests it adds `iptable` rules to all nodes so that the nodes
see each other's SSH servers as running on this port.
Their SSH configuration is _not_ changed and other machines see the
nodes on 22 as before.

The `iptable` rules are reset on each QA run, trying to preserve any
existing rules (not created by the script) that might be present.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In particular _copyfile_ and _renew-crypto_.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This is accomplished by passing the corresponding node group to
hv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The unit tests now include the configurations of SSH ports for node
groups/nodes.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The parameter is added to Haskell sources, from which the corresponding
Python code is generated.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Also add a method for the retrieval of "ndparams" (node parameters)
of groups, not just nodes.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In an effort to get rid of the python queries soon, this
patch fixes a FIXME of the group queries regarding the
missing testing of niceSort in this context. Due to the
lack of actually weirdly named hostnames, this patch
adds an extra unit test for hostnames instead.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add QA tests for instance shutdown detection with Xen PVM and HVM
hypervisors.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Several methods in KVM hypervisor behave like 'classmethods', however,
they did not have the '@classmethod' annotation.  This patch adds
'@classmethod' annotations to several of these methods with the goal
of making '_StopInstance' available to the QA.  This step is necessary
for the QA test for instance shutdown detection on KVM.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update hypervisor unit tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update watcher to destroy shutdown domains when it detects that a
domain is in the state 'USER_down'.  This requires adding logic
similar to that of restart instance, that is tracking the number of
cleanup attempts in the watcher's state and give up if a given number
of retries has been reached.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update instance queries on the Haskell codebase to report 'USER_down',
similarly to the Python instance queries.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update instance queries on the Python codebase to report 'USER_down'
in the situation where the user has shutdown a Xen domain but Ganeti
still marks this domain as 'ADMIN_up' in its configuration file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Prevent starting a given instance when that instance has been shutdown
by the user, given that the instance must be first properly shutdown.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update Xen backend to properly recognize when a domain has been
shutdown by the user and to properly cleanup a shutdown domain when
Ganeti requests Xen to stop this domain.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add instance state 'USER_down' which is a state used in reporting only
and it represents the situation in which the user has shutdown the
instance but Ganeti's configuration still has this instance marked as
'ADMIN_up'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Change Xen domain configuration to preserve the domain state after
shutdown, as opposed to automatic reboot, which is the default
behaviour.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'InstanceState' datatype which is the Haskell counterpart of the
Python type 'HvInstanceState'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'HvInstanceState' which represents the state of an instance
(either 'running' or 'shutdown') and fix the remaining hypervisor
backends to return a value of this type.  Before this patch, each
backend was returning their own value, some were returning strings,
others integers.  However, the value that represents the state of an
instance was never actually used in Ganeti, until instance shutdown
detection became necessary.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Despite the fact that the '--network' parameter of
'gnt-network add' is mandatory, it is listed in the man
page in parenthesis, which suggests that it is optional.
Fixing this to avoid user confusion.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Disks and nics can not only be addressed by indices, but also
by name. Hence add a constructor for this case as well, to be
faithful to the python world.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

A previous patch [229da00f] added an annotation for ignoring the warning,
but to the middle of a function, which doesn't compile. This patch moves
the annotation to the end of the function to correct the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This warning appears only in newer versions of hlint (mine was v1.8.43),
and in this case it's reported incorrectly. The arguments to "showJSON"
have different types, therefore it's not possible to move "showJSON" in
front of "if".
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

With the change to luxid handling job queue requests, we
also inherited the parsing semantics of luxid, i.e., a
request is only considered syntactically correct, if all
parts are. Document that we decided to consider this change
of semantics desirable.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Historically, Ganeti would partially handle syntactically incorrect
luxi requests, if the outer structure was that of SubmitManyJobs
request. Moreover, this behavior was verified by a test. We changed to
completely reject syntactically incorrect luxi requests and decided
that this semantics was more desirable. Hence adapt the test
accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Now that mandatory opcode fields are mandatory also on disk,
document that, once we use soft drains during upgrades, the disk
representation of the job queue needs to be updated by cfgupgrade
as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In order to guarantee all mandatory slots to be present, add
a custom constructor to BaseOpCode adding those fields with
default value, instead of inheriting the constructor from
outils.ValidatedSlots.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

While it's generally better to beg for forgiveness than ask for
permission in Python, it's not a good idea to rely on exceptions to
model control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

DTS_FILEBASED is a constant that exists and this commit makes sure
that it is used whenever sensible, rather than resorting to hardcoding
the pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

As the responsibility for writing the job queue changed
to luxid, make masterd call to luxid when it is necessary
to write a job.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>