GitLab

Add an option to hroller, to plan for full evacuation of the
nodes to be rebooted, i.e., also plan for replacement secondary
nodes for all instances on the node after migrating out instances
with this node as primary.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Separate the partitionNonRedundant function in hroller into a
general functional that partitions a list of nodes according
to some clearing strategy and the specialization of moving
non-redundant instances out. In this way, we don't have to
duplicate code when partitioning due to capacity of finding
new secondary nodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The method clearNodes in hroller greedily clears nodes of
non-redundant instances by moving them to a different node. This patch
separates the greedy clearing algorithm from the specialization to
non-redundant instances; in this way, we don't have to duplicate code
when supporting clearing nodes of secondaries.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

When planing on where to evacuate the non-redundant instances
of the nodes to be rebooted, it doesn't make sense to consider
offline nodes. So add this restriction to hroller.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

hroller schedules moves of instances to have rebooted nodes
free of instances with this node as primary. Update the comments
to reflect that this move planning is for non-redundant instances
only.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

When given the option --print-moves, hroller prints a list of
instance relocations after each reboot group. This sequential
presentation can lead to the wrong impression, that these are
all the moves to be carried out. Therefore, explicitly mention
the assumption that after each reboot of a group, all instances
are moved to their initial position.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Originally, hroller started as a tool for offline maintenance only.
There it made sense to warn about instances still running. By now,
default planning is to migrate instance off the nodes to be rebooted,
with options for other behavior (like pretending that all instances
will be shut down before the actual reboots start). In that scenario
instances running is the normal situation, not something to be warned
about.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As different users have different needs about the chroot, allow
customization in its build script. This is achieved as follows.
- The environment variable EXTRA_DEBIAN_PACKAGES is honored,
  and the packages named there are additionally installed in
  the chroot. This should cover the most common cases.
- More complicated customizations can be achieved by setting
  the environment variable CHROOT_FINAL_HOOK. This is the last
  command executed in the chroot; typically, it would be
  a call to a personal script.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In the last change to the chroot build script, the
apt-get command 'install' was moved to the APT_INSTALL
macro; however, it was forgotten to remove it outside
the macro, causing an error, as there is no package
with name 'install'.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In this way, it is easier to change all the apt calls in the chroot.
In particular, this patch fixes the missing -y option for the installation
of the tools needed to build debian packages.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

With exclusive storage, PVs are supposed to be used only up to the
standard PV size.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Disk growth is now supported also with exclusive storage, so we let the
operation continue.

Enable disk growth test in qa also with exclusive storage.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The exclusive storage flag is passed along in the backend functions for
disk growth. LVM volumes handles the flag by reusing the same PVs, as disks
are constrained by the number of their spindles.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The flag is passed to the RPC, but it's still ignored by noded.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch includes the design document for openvswitch support
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In order to be able to use it as vif-script one has to
create a symbolic link inside /etc/xen/scripts pointing
to /usr/lib/ganeti/vif-ganeti and then add is a hypervisor
parameter.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add new script vif-ganeti that could be used instead of
default vif-bridge in order NIC customization can be more
flexible.

Extract common code from kvm-ifup and insert it in a new
file net-common that is being sourced by kvm-ifup and
vif-ganeti and located under package lib dir (/usr/lib/ganeti/).

Install vif-ganeti properly with make under /usr/lib/ganeti too.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

For every NIC write down a file that contains NIC's
info (MAC, mode, link, network details), that can
be sourced by any vif script.  The file location is:
/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>.

This file is created upon cfg file creation and before
starting the instance and can be sourced by an external
script. Before file creation ensure that all parent dirs
exist.

Cleanup NIC dir when removing config files. Upon Xen
configuration file removal, remove NIC dir as well.

Add new hv param vif_script that allows execution of
user defined networking script and overrides the one
xend is configured with in xend-config.sxp
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The requirement to be able to plan full node evacuation came up after
the initial design. So add a section describing the algorithm, if
full evacuation is requested.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

So far, the iallocator makes a node info call, converts
the result into a legacy node info structure and then
retrieves various values from it. This patch removes
the call to MakeLegacyNodeInfo completely, by letting the
consumers of the hypervisor info part of the result
retrieve that part from the result of the node info call
directly.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The iallocator adjusts the free memory by a calculation
of how much memory is actually currently used by instances
on the node. This calculation is rather complicated and
deserves its own function to increase testability. No
functional changes in this patch otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

So far, the iallocator made a node info RPC call, converted
the result into a legacy node info format and then extracted
the storage information from that. This patch removes the
middle step of this procedure, skipping the legacy node info
usage.

In later patches, we will completely eliminate the need for
the legacy node info in iallocator.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

In order to extend the Haskell version of the NodeInfo
query, we need some utility functions to deal with
disk templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch changes the semantics of dealing with the result
of the RPC call "node_info" when converting them into a
LegacyNodeInfo. A legacy node info, can only hold storage
information about one storage type, which so far was always
LVM storage. With the recent changes in the RCP call
'node_info', a node info call returns storage info of several
storage types and not necessarily with lvm being the first
in the list. In order to be backward-compatible with tools
that assume that LVM storage is the only storage type, we
make sure, that MakeLegacyNode info can be called
requesting LVM info specifically. This is just an interim
solution to not break everything at once. The long-term
goal is to get rid of MakeLegacyNodeInfo completely.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch extracts an inline 'get_attr' method to extract
data from the result of the RPC call 'node_info'. It was so
far an inline function in the middle of a bigger method, which made
the code hard to read.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch does a small refactoring in the allocator. It extracts
the preparation and actual call of the 'node_info' call into a
separate function. This is done for better testability. No functional
changes otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If there are no LVM based disk templates enabled in QA, reset the LV
name after the corresponding gnt-cluster modify tests. Otherwise
gnt-cluster verify would complain about the missing volume group.

Also make sure that no disk templates requiring LVM are ever enabled
without actually providing a LV name.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Changing the drbd usermode helper in gnt-cluster modify fails if DRBD is
not installed on the nodes. Thus skip this test if DRBD is not in the
set of enabled disk templates.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This route is required on the Fedora test machine, as otherwise fping
would not work and hang forever.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The tempfile command only exists in Debian. In order to run the VCluster
QA also on Fedora, replace its use by mktemp.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.8:
  gnt-cluster info (py): add enabled disk templates
  Version bump to 2.8.0~beta1
  Change version numbers in documentation
  Fix issue with python coverage tests
  Merge branch 'stable-2.7' into stable-2.8
  Disable python test if required libraries are missing
  Better specify what packages to install
  Improve install guide
  Fix typo in the documentation index
  Fix typos in the documentation index
  Update security document wrt confd access to SSL cert
  Add tools for building deb packages to build_chroot
  Improve the final message of build_chroot
  Make build_chroot self-contained

Conflicts:
	configure.ac (trivial)
	lib/cmdlib/instance_storage.py (trivial)

The version numbers changed in 'Change version numbers in documentation'
and 'Version bump to 2.8.0~beta1' were adapted to 2.9.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.7:
  Provide the right fix for the kvm_extra parameter
  NEWS and version updates for 2.7.0~rc3
  kvm: don't pass whole extra as a single argument
  For node queries allow short forms of host names
  Provide a special filter for host names

Conflicts:
	NEWS: trivial
	configure.ac: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

With the previous adjustment the arguments were boxed into one extra
array. This actually makes the parameter work (with the same caveats).

The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Time for a new release.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If extra is longer than a single word passing it the way it was passed
won't work. Of course this solution is also only partial, as won't take
into account any form of quoting. :/
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>