GitLab

This patch introduces some infrastructural modifications that will be used by
the following commits to implement the support for specifying the reason for
the last status change of an instance.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add to Confd server a new request type (and its implementation) to ask for
the list of instances in a node.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit fb251c2c

. On the
master branch we want to continue to have them enabled.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Back when this was introduced, I mentioned that it breaks heavily tab
completion (ob<TAB> doesn't work anymore), but at that moment I didn't
have a suggestion what to name it. I think outils is good and short
enough, and doesn't conflict with anything else, so here it goes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The documentation still points to /etc/ganeti/remote-commands,
although the code is already using restricted-command. Update the
documentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

'rbd showmapped' output formatting differs between older and newer versions of
the ceph tools. Try to use json output formatting if available (currently
available only in the ceph master branch). For bobtail, argonaut and older
releases fallback to manually parsing the 'rbd showmapped' output, handling the
differences in the output format for each rbd version correctly.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

All recent-enough versions of linux see the xen paravirtual device as
/dev/xvd*.

This doesn't break old installations, as the default is only used on new
clusters.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returning
None) in case it does not find the requested network. Remove useless and
duplicate code such as:

if net_uuid is None:
  raise...

This is a cherry-pick of commit 1cce2c47

.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Until now if one disconnects a network with --no-conflicts-check
and then remove it, there is a possibility to leave instances with NICs
referencing non-existing networks. This causes network queries,
instance removal and modification to fail.

This patch allows a network to be disconnected from a nodegroup,
only if instances residing in the nodegroup do not have NICs attached
to the network. Otherwise OpPrereqError is raised. The
--no-conflicts-check option is removed from the gnt-network disconnect
command as well.

This is a cherry-pick of commit 6bb939a2

.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch improves the error messages given when a
“errors.AddressPoolError” exception is caught. Includes some small style
fixes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This can be used in hypervisor code as well. For consistency
export *NETWORK_NAME and not *NETWORK throughout the code.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This info is not used by Ganeti and therefore is removed.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This slot was not used by Ganeti so the same info can be
provided via tags. In order not to break configuration data
we add a FromDict() method in Network config object that
removes the deprecated network_type (if found) and then invoke
the parent's method (which does the validation).
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Cluster-verify used to check that the value of exclusive_storage is uniform
within node groups. Now, it's impossible to change the flag for a single
node, so that check has been removed and an equivalent one has been added
in ConfigWriter.VerifyConfig(). Unit test provided. The unit test covers
only the new check, not the whole VerifyConfig method.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There's never been support for a configuration where nodes in the same node
group have different values of the exclusive_storage flag. This patch
disables the possibility to change the flag for individual nodes.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Before, only some upgrades were written back to the configuration file. A
little refactoring of _UpgradeConfig() has been done to write unit tests.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since now confd also serves a Luxi endpoint, the current message in
cli.FormatError is misleading when actually failing to connect to
it. The patch adds a somewhat hackish way to show the right daemon
name.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

1) Move the hooks environment dict generator inside the object. This
also adds missing values such as network family and uuid.
2) Use the same generator both for the os environment and for the
instance hooks.
3) Update manpage and hooks documentation.

Also update a forgotten docstring about a tuple's content.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Although this does nothing for now, running it is safe, and consistent
with how other objects behave.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When absolutely no OSes are present on the cluster, the result of
OpOsDiagnose is an empty list. This is currently handled in gnt-os as
an error condition, probably due to how OpOsDiagnose used to return
errors in the past.

However, with the query backend, an empty list is a reasonable answer,
so I see no reason why gnt-os should treat that as an error,
especially as it doesn't give any hint of what is "wrong". Simply
removing that code block allows it to proceed normally, and list no
OSes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The method names were wrong due to copy & paste.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The QA code will also have to generate virtual paths and this function
comes in handy.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

- Capitalize acronyms such as “QMP” or “SPICE”
- Remove punctuation from error messages
- Improve error reporting when sending a monitor command failed
- Put “or” at end of line
- Fix indentation for a concatenated string
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Monitor calls should be replaced with QMP once KVM >= 0.14 is required.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 3329f4de

 changed the Haskell log file from constants to
functions, but introduced a bug: it uses now the daemon name instead
of the correct log file, which means "ganeti-confd.log" instead of
"conf-daemon.log".

In order to fix this, we need to abstract the log file base (in
constants.py) into a separate set of constants, so that we can reuse
it in the Haskell code.

This fixes issue 343.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

“objects.ConfigObject” contains two useful functions for working with
containers of serialized objects, “_ContainerToDicts” and
“_ContainerFromDicts”. This patch separates those functions and moves
them into “objectutils” as they'll be useful for converting parts of the
QA configuration to objects. Unittests are added to cover the parts not
already tested through other code paths.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

“utils.RunCmd” is re-exported from “utils.process.RunCmd”. Epydoc
doesn't fully understand this, so we have to refer to the original.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Handle de-serialization correctly when pool is not defined
- Serialize to empty list when the attribute is None (this should never
  happen in reality as the attribute is always set when de-serializing)
- Add tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This uses the new _FormatVerifyResults helper function to return
multiple errors.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since LXC now relies on cgroup memory limits to enforce memory, let's
make hypervisor verification (and thus cluster-verify) return errors
when the cgroup filesystem is not mounted.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will allow easier multi-error results from hypervisors; right
now, we only report the first error, which is not nice.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The latest lxc patches included a few whitespace style errors, that make
lint fail. This patch fixes those.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add support for the memory resource controller, useful to actually place
memory limits on instances.

Support is still optional, in case the kernel doesn't have it compiled
in, or -in the case of Debian- has it dependent on a kernel command-line
switch (cgroup_enable=memory specifically).
Signed-off-by: Faidon Liambotis <paravoid@debian.org>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently hv_lxc depends on the behavior and output of older LXC tools,
which have since changed, making it unable to function in current
distributions (e.g. Debian wheezy).

Adapt the tools and expectation for the output and make it into a
working state again, at least for LXC 0.8.0.

LXC tools seem to be split into two categories: the actual, binary lxc
tools, plus a set of shell scripts to provide a higher-level interface.
Ganeti doesn't need the latter and the shell scripts make some
assumptions (like /var/lib/lxc) so remove the only reference to such a
shell script (lxc-ls).

While at it, remove a few comments about "current (Spring 2010)
distributions" that do not apply anymore.

This is a backwards-incompatible change, meaning it won't work on
systems with older LXC userspace. However, hv_lxc is experimental anyway
and there's virtually no documentation about it in the tree.
Signed-off-by: Faidon Liambotis <paravoid@debian.org>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes: “ganeti/hypervisor/hv_kvm.py:93: [W1201, _GetTunFeatures]
Specify string format arguments as logging function parameters”, and
also updates the copyright years.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Currently, the node listing RPC is very slow due to missing
parallelisation. For the 2.7 release, we reset these back to masterd,
hoping to revert them by the time 2.8 is ready.

There are a number of queries that I've left pointing to confd, as
they are non-RPC:

- node fields query
- node static fields query (and added a note in one case where it
  could become a problem)

All the other converted queries (job listing, group queries, tags,
cluster config) remain pointing to confd, as they don't use RPC.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also, fix @rtype and @return elements of utils.TestDelay(), which now
returns a tuple but this wasn't being indicated.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returning
None) in case it does not find the requested network. Remove useless and
duplicate code such as:

if net_uuid is None:
  raise...
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

Until now if one disconnects a network with --no-conflicts-check
and then remove it, there is a possibility to leave instances with NICs
referencing non-existing networks. This causes network queries,
instance removal and modification to fail.

This patch allows a network to be disconnected from a nodegroup,
only if instances residing in the nodegroup do not have NICs attached
to the network. Otherwise OpPrereqError is raised. The
--no-conflicts-check option is removed from the gnt-network disconnect
command as well.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>