GitLab

This patch, extends the 'ganeti.utils.storage_unittest.py' unit test
with a new test for the Gluster storage type. Also, the current unit
tests are updated accordingly to correspond to the latest changes.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 42fdf9d0

Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch, extends the file-based storage type verification checks to
include the new Gluster storage type. Those modifications include a
missing check for the gluster storage directory at cluster
initialization time, and extends the 'gnt-cluster verify' command to
also verify the gluster file-storage path.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 604c8be2

Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch, adds the 'gluster' storage type to the set of storage types
for which full node storage reporting is available. This set is used by
the 'LUNodeQueryStorage' logical unit for getting information on storage
units on node(s).
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 7ec47185

Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Currently, the 'gluster' and 'sharedfile' disk templates are both mapped
to the Shared File storage type. This compromise causes the Gluster block
devices to be interpreted in the config as 'sharedfile' devices, instead
of 'gluster', and subsequently to be wrongly handled as Shared File
disks. This behavior, currently makes the 'gluster' disk template not
functional.

This patch, fixes this issue by introducing the Gluster storage type,
which will be exclusively used by the 'gluster' disk template.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from ba550291

Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.11
  (no changes)

* stable-2.10
  Check for gnt-cluster before running gnt-cluster upgrade
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stbale-2.10
  Check for gnt-cluster before running gnt-cluster upgrade
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

With Ganeti imposing no requirement on Xen versions, the configuration
file that should be present varies greatly on the choice of toolstack
and Xen version. xend-config.sxp was considered obligatory, but in
higher versions and with xl, it is superseded by xl.conf and considered
deprecated. This patch makes it have the same status as xl.conf - Ganeti
will distribute it but not require its presence.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When ganeti is removed (not purged) `/etc/cron.d/ganeti` is not deleted,
thus after a reboot cron tries to execute gnt-cluster upgrade and fails.

The same pattern is used on all other cron entries.
Signed-off-by: Christos Trochalakis <christos@skroutz.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

..  with the latest bugfixes and known issues
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

After cherry-picking the changes needed for GHC7.8, the former isn't
used any more.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

to ensure we are not reporting stray volumes on other VGs as orphans.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Prior to patch eeda5882, Ganeti checked all the LVs on nodes under
its control, and dutifuly declared any stray volumes as orphans in
warnings emitted during cluster-verify. After the patch, the nodes
returned information related only to the LV set as *the* Ganeti VG,
bypassing this problem.

Unfortunately, this led Ganeti to report the disks of instances created
on a different VG as non-existent. This patch fixes things by reverting
patch eeda5882

 and making Ganeti warn about orphan volumes only if
they belong to the Ganeti-administered VG.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When opening domain sockets for communication, the Haskell daemons did
not set any permissions for the sockets, defaulting to 0700. This was
fine when all of them ran as root, but was bound to cause trouble in a
split-user setup. The first issue is RAPI access after master-failover,
where RAPI could not send make any inquiries until the watcher restored
the desired permissions of the socket.

This patch modifies Luxid to use a g+rw socket, and leaves other servers
to their default of 0600.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

It turns out that GHC 7.8's -dynamic-too is forbidden not only for
profiling builds object files, but also for those using HPC coverage.

This commit accordingly renames HFLAGS_NOPROF to HFLAGS_DYNAMIC
(since it now is not conditional on profiling any more),
and makes sure that it is not used for profiling or HPC.

This way we achieve that, for profiling+coverage builds
  - .dyn_o files are available for use in TH in the following 3 cases
  - .o files are for the normal binaries
  - .hpc_o files are for coverage-enabled test binaries
  - .prof_o files are for profiling binaries

We make .hpc_o files depend on the .o files because the creation of
the .o files will also create the .so files needed for TH.
This was already in place for .prof_o files.

This requires that HFLAGS now also contains `-itest/hs` since
the rule for .o files is now also responsible for building .o/.so files
in test/hs.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked-from: 1ad14f38

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This fixes
  ./configure --enable-haskell-tests && make hs-tests
builds on GHC 7.8.

It changes `-osuf .ext` to `-osuf ext`; the old form was allowed in
GHC <= 7.6, but GHC 7.8 seems to have problems with it:
  https://ghc.haskell.org/trac/ghc/ticket/9760



The form without dot works in all GHC versions, and we don't have
dots in other -osuf arguments in the same file either, so this
was probably an accident.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked-from: 9664aff9

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

GHC 7.8 switched to dynamic linking being used for ghci, which requires
that .so file are being built for modules that are loaded for TH.

For this reason, GHC >= 7.8 has a -dynamic-too flag, which we now use.

However, -dynamic-too must not be enabled for profiling builds,
which is why this commit introduces a HFLAGS_NOPROF variable
(currently only containing -dynamic-too) that must be passed to
all GHC invocations that are not creating profiling output.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked-from: 083776b9

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This works around https://ghc.haskell.org/trac/ghc/ticket/9749

:

GHC 7.8 (undocumentedly) changed the way in which ghc -M generated
object file dependencies, lacking the underscore that older versions
added automatically to the file names.
It also requires a -dep-suffix for the plain object file (.o).

This commit detects GHC 7.6 and older (7.7 is development only, has no
release and is treated equal to 7.8), and adjusts the -M invocation
appropriately for newer GHC versions.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked-from: b78a2c30

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Whenever we set a configuration, we also upgrade it.
In particular, there is no need for an unconditional upgrade;
so avoid upgrading the configuration over and over again
if it did not change.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

We only acquired a config lock if it was not shared in order
to have lock-free reads. Hence, only release the config lock
if we actually acquired it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>

Cherry-picked-from: 35056743

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add a mention into the install file, and provide a link to the official
DRBD documentation.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...as we still support python versions older than the introduction
of assertIn.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Make sure that there is an environment variable for each parameter
with the correct value.
Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

and unobscure them
Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

...even if the initial hand shake succeeded.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...so that we do not accumulate open file descriptors.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

.. with the latest bugfixes and known issues.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Old versions of Ganeti (in particular, 2.5 and earlier) did not
have UUIDs assigned to objects. If we happen to find such an old
configuration, we assign UUIDs now, during the upgrade. However,
we must do this recursively, as disks might have children. Note
that before Ganeti 2.12 this didn't matter, as the UUIDs of the
children where never used or enforced. With the strict type checking
introduced by the switch to haskell, we have to care about those
little details.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In 2.11, children disks have UUIDs, too.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This was omitted after the refactoring of SSConf to Haskell, now being
added. Fixes #1073.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. and a function for converting hypervisor types to the keys.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. instead of String. This imposes a type-level restriction that the
keys of the map are just hypervisor names.

Note about 'Arbitrary GroupDiskParams': Since GroupDiskParams and
ClusterHvParams were both synonyms for 'Container (Container JSValue)',
the Arbitrary instance worked for both. After fixing the type of
ClousterHvParams, its instance become different from GroupDiskParams,
hence the latter needs the addition.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

...as it was accidentally readded in the merge.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.11
  Update configure file to version 2.11.7
  Update NEWS file for 2.11.7 release
  Add logging to RenewCrypto
  Fix format string for gnt-network info
  Replace textwrapper.wrap by a custom version for networks
  Add SSL improvements to NEWS file

* stable-2.10
  Update tag limitations
  Fix typos in doc/design-storagetypes.rst
  Make getFQDN prefer cluster protocol family
  Add version of getFQDN accepting preferences
  Make getFQDN honor vcluster

Conflicts:
	NEWS: take all release entries
	configure.ac: ignore revision bump
	lib/cmdlib/cluster.py: manually apply 2.11 changes to 2.12
	src/Ganeti/Daemon.hs: trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.10
  Update tag limitations
  Fix typos in doc/design-storagetypes.rst
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

and use one-element tuplesfor string formatting
Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

After refactoring the code for network addresses to Haskell, a
regression appeared that added 30 to all IP addresses in the generated
list. Since the code for generating the list used just 'show' and look
for 0s and 1s in the string (and treating anything else as 0), it turned
out that after adding a separate type for address pools, the strings
become

    "AddressPool {apReservations = ...}"

and the initial part of the string was exactly the mysterious 30
characters.

This fix replaces this by switching to correct types and iterating over
the bits in BitArray.

Fixes #1068.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>