GitLab

While the general policy for Ganeti is to just accept the situation
it finds after being upgraded from an older version, in some cases
additional actions might be necessary. So support a hook for doing
so.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Our design states, that the intent-to-upgrade file contains "the current
version of ganeti, the version to change to, and the process ID". Make the
implementation fit with that design.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Mention inherited changes as well as the (very few) changes made to
stable-2.10 since beta1.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

pylint 0.26.0 on Python 2.7 generates a warning on the string '\ ',
recommending to use the r prefix. This patch adds the missing prefix.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The following changes are made:
 * Add M4 as required dependency
 * Change reference to qemu-img package to qemu-utils
 * Never use `sudo` for easy_install (not used for apt-get neither)
 * Add libpcre3-dev as required package for Wheezy (otherwise linking
   fails)
 * Install sphinx from packages (easy_install would install an
   incompatible version on Wheezy)
 * Fix spelling error
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* stable-2.9
  Bump revision for 2.9.2
  Update NEWS for 2.9.2 release
  Pass hvparams to GetInstanceInfo
  Adapt parameters that moved to instance variables
  Avoid lines longer than 80 chars
  SingleNotifyPipeCondition: don't share pollers
  KVM: use custom KVM path if set for version checking
* stable-2.8
  Version bump for 2.8.3
  Update NEWS for 2.8.3 release
  Support reseting arbitrary params of ext disks
  Allow modification of arbitrary params for ext
  Do not clear disk.params in UpgradeConfig()
  SetDiskID() before accepting an instance
  Lock group(s) when creating instances
  Fix job error message after unclean master shutdown
  Add default file_driver if missing
  Update tests
  Xen handle domain shutdown
  Fix evacuation out of drained node
  Refactor reading live data in htools
  master-up-setup: Ping multiple times with a shorter interval
  Add a packet number limit to "fping" in master-ip-setup
  Fix a bug in InstanceSetParams concerning names
 ...

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Besides a few local fixes, the main improvement are the changes
inherited from stable 2.8.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The format of the /etc/ganeti/file-storage-paths file was not documented
in the man page. This patch adds a short note about the format there.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

...so that the xen command to be called can be determined. This
fixes another semantical conflict of the last merge.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Due to a change in the code organization in stable-2.9, some
method variables became instance variables, causing a semantic
merge conflict. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

...as they're a lint error.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.8
  Version bump for 2.8.3
  Update NEWS for 2.8.3 release
  Support reseting arbitrary params of ext disks
  Allow modification of arbitrary params for ext
  Do not clear disk.params in UpgradeConfig()
  SetDiskID() before accepting an instance
  Lock group(s) when creating instances
  Fix job error message after unclean master shutdown
  Add default file_driver if missing
  Update tests
  Xen handle domain shutdown
  Fix evacuation out of drained node
  Refactor reading live data in htools
  master-up-setup: Ping multiple times with a shorter interval
  Add a packet number limit to "fping" in master-ip-setup
  Fix a bug in InstanceSetParams concerning names
  build_chroot: hard-code the version of blaze-builder
  Fix error printing
  Allow link local IPv6 gateways
  Fix NODE/NODE_RES locking in LUInstanceCreate
  eta-reduce isIpV6
  Ganeti.Rpc: use brackets for ipv6 addresses
  Update NEWS file with socket permission fix info
  Fix socket permissions after master-failover

Conflicts:
	NEWS
	configure.ac
	lib/cmdlib/instance.py
	lib/cmdlib/instance_migration.py
	lib/hypervisor/hv_xen.py
	lib/masterd/iallocator.py
	lib/objects.py
	src/Ganeti/HTools/Backend/IAlloc.hs
	src/Ganeti/HTools/Backend/Luxi.hs
	src/Ganeti/HTools/Backend/Rapi.hs
Resolution:
	NEWS: take both additions
	configure.ac: ignore revision bump on stable-2.8
	Rest: manually apply the stable-2.8 changes on stable-2.9 code;
              for lib/hypervisor/hv_xen.py this also includes passing
              the additonal hvparams around, and adapting tests.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

List all the changes that happened between 2.8.2 and 2.8.3.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

If param=default and the param already exists then we remove
it from params dict. This is stolen by GetUpdatedParams() which
is used for hvparams modification/inheritance.

This means that 'default' value is not accepted for an arbitrary
param of an ext disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Disks of ext template are allowed to have arbitrary parameters
stored in the Disk object's params slot. Those parameters can be
passed during creation of a new disk, either in LUInstanceCreate()
or in LUInsanceSetParams(). Still those parameters can not be
changed afterwards. With this patch we override this limitation.

Currently, for the other disk templates we allow modifying only
'name' and 'mode'. Therefore, we introduce new constants
MODIFIABLE_IDISK_PARAM* to include those params. If any other
parameter is passed, _VerifyDiskModification() will raise an
exception.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Commits 5dbee5ea and cce46164

 fix disk upgrades concerning params
slot. Since 2.7 params slot should be empty and gets filled
any time needed.

Still ext template allows passing arbitrary params per disk.
These params should be saved in config file for future use.
For instance if we have the shared-filer provider and we
specify shared_dir param during instance create, this param
is needed when we want to attach the disk e.g., during
retrieving instance info. If it gets overridden during a daemon
restart or a config reload we fail to get the instance's info.

To avoid such a failure, we set params slot to an empty dict
only if params not found in the first place.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

SetDiskID() fills physical_id slot of a Disk object.

LUInstanceSetParams() does not invoke SetDiskID() upon creation of a
new disk. As a result the physical_id slot of the Disk object in
config data is missing.

In case of ext disk template, in AcceptInstance() we invoke
_GatherAndLinkBlockDevices(). This takes `instance` as an argument
which includes current disks info. So, after adding a disk,
migration of ext instances will fail because FindDevice() expects
the physical_id slot.

With this patch we invoke SetDiskID() for every disk of the instance
before accept_instance() RPC.

Fixes Issue 633.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This is required to prevent race conditions such as removing a network
from a group and adding an instance at the same time. (See issue 621#2.)
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

According to commit 599ee321

, any job-related error messages should
be encoded within a Ganeti-specific error and not passed on as a
string, to allow for easier parsing.

For jobs suffering from an undesirable status after an unclean master
daemon shutdown, the message was not encoded, as reported in issue 618.
This patch fixes the problem.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

If the file driver of an instance with file based storage is not specified, the
default one is automatically added by the UpgradeConfig function.

Fixes Issue 571.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Update hypervisor unit tests.

Partial cherry-pick from d2e4e099

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Update Xen backend to properly recognize when a domain has been
shutdown by the user and to properly cleanup a shutdown domain when
Ganeti requests Xen to stop this domain.

Partial cherry-pick from 9d22cc90

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* fix node daemon not to skip data, such as, memory and disk size,
  when building the node list to send to HBal, given that these data
  are important for HBal to determine whether an evacuation is
  possible
* fix iallocator to properly load drained nodes from the list passed
  by the node daemon, instead of zeroing all the data, such as, the
  memory and disk size
* this fixes issue 615
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

This simplifies different handling of individual items.

Cherry-picked from 8c72f711

.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

In the case of network problems, one ping packet can possibly get lost.
Sending multiple packets is safer. The interval between packets is set
to 200ms so that the check finishes faster.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This fixes issue #630. Apparently there is a bug in fping 3.5 where it
loops forever without "-c" given an unreachable host, even though
"-c 1" should be the default according to the man page.

The "-c" flag works on Squeeze. Checking the man pages on the Internet,
fping supported "-c" at least since 2007. So there should be no backward
compatibility problems.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

This method is invoked during _AnalizeSerializedRuntime() and is
meant to modify runtime files in the way cfgupgrade does for
config.data.  This could remove deprecated fields, change the
format of the file, add/remove sections, etc.

There is a possibility data included inside runtime
files to cause migration failures after a Ganeti upgrade.
Use this method to avoid such cases.

Commit cc6fd3da

 exports NICs' UUIDs to configuration scripts.
It assumes UUIDs as an non optional slot in NIC object.
This is true for latest Ganeti versions. Still there might
be instances created back in Ganeti 2.7 missing that slot.

Ganeti 2.10 supports hot-plugging of devices. A prerequisite
for this feature was to change the format of runtime files and
add another entry containing block devices. This was done in
_AnalizeSerializedRuntime(). Move this logic in the new method.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

In case no name is passed in disk modifications we should
keep the old one. If name=none then set disk name to None.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

As widely known Ganeti uses a better[1] lock condition notification
library based on operating system pipes.

Inside this library we were using a shared poller for all threads
waiting for a condition. While poller is not thread safe, since (1)
we're holding the condition lock while calling poll and while parsing
results, and (2) we don't reuse the poller between different conditions,
or with newer fds our usage *is* actually safe. Unfortunately newer
versions of python take a hand-holding approach and don't trust us to do
the right thing. As such we are forced to create a new poller each time
we call wait.

This costs one system call more per wait, but practical measurements
have shown no significant impact on Ganeti. This is also a temporary
measures as newer versions will get away with the threading altogether
and move job schedulers to luxid.

The patch is losely based on a patch submitted by Daniel Néri, but has
been modified to reduce even further the scope of the poller variable to
just the waiter class.

[1] because I say so.
[1 bis] also because it produces fairer results, avoids possible
starvation and eliminates busy-wait polling.
[2] http://bugs.python.org/issue8865

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The newest version does not build on Debian squeeze, so avoid
it being pulled in as a dependency.

This is the same issue that has been fixed in [1e078ef3

] on master.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fixes issue 616.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Commit 0e3f65d2

 erroneously installed pyinotify on the host system.
This commit fixes that and adds a comment on why we have to install
this dependency separately in the first place.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Each host using IPv6 always has a link local address in fe80::/10. It is
common to use fe80::1 as default gateway to ease client configuration.
Ganeti prevented this usage, because it made sure that the IPv6 gateway
is in the IPv6 network the instance is connected to.

This patch also allows to specify a IPv6 gateway in the link local
network in addition to the network the instance is connected to.

This fixes issue 624.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Both NODE and NODE_RES locks were acquired opportunistically if so
requested by the user. LUInstanceCreate requires, however, that the
actually locked elements on NODE and NODE_RES level are the same.

This patch changes the locking of NODE_RES such that those locks are not
acquired opportunistically any more. Instead, the mandatory locks are
set to the acquired NODE locks once they are actually granted.

This fixes issue 622.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This is not only better style, but also fixes a lint error.
Also use the infix form of `elem` to increase readability.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

We detect an IPv6 vs V4 address based on columns, rather than passing
the family from the cluster object to be more future proof (in case
we'll ever support mixed clusters).

Unfortunately quite a bit more code is required to test this: we need an
arbitrary node that does the right thing w.r.t. ip addresses and also
test-only exports. As such we'll do this out of the stable branch.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This commit fixes two TODOs from 2008 about using the hardcoded
"default" path for KVM where a custom one could've been set through
`gnt-cluster modify`.

As a result, `gnt-cluster verify` will no longer fail if a custom
path was set in such a manner.
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>