GitLab

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Describing the changes since the last release candidate. Also, while there,
remove the incorrect "known issue" about 2.14 not being able to be built with
GHC 7.8; that always worked from stable-2.13 onwards. Same for gluster support.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.13
  Increase waiting time to avoid races
  Avoid watcher interaction in TestFilterRateLimit
  Avoid races in TestAdHocReasonRateLimit
  Avoid races in TestFilterRateLimit

* stable-2.12
  Add a unit test for the Gluster storage type
  Add missing Gluster storage verification checks
  Add Gluster type to the node storage reporting set
  Introduce the Gluster storage type
  Make xend-config.sxp optional
  Revision bump to 2.12.4
  Update NEWS for 2.12.4
  Ignore *.dyn* files generated by GHC 7.8
  Replace HFLAGS_NOPROF with HFLAGS_DYNAMIC in the Makefile

* stable-2.11
  (no changes)

* stable-2.10
  Check for gnt-cluster before running gnt-cluster upgrade

Conflicts:
	Makefile.am: take stable-2.14 version
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.12
  Add a unit test for the Gluster storage type
  Add missing Gluster storage verification checks
  Add Gluster type to the node storage reporting set
  Introduce the Gluster storage type
  Revision bump to 2.12.4
  Update NEWS for 2.12.4
  Ignore *.dyn* files generated by GHC 7.8
  Replace HFLAGS_NOPROF with HFLAGS_DYNAMIC in the Makefile

* stable-2.11
  (no changes)

* stable-2.10
  Check for gnt-cluster before running gnt-cluster upgrade

Conflicts:
	NEWS: take all additions
	configure.ac: ignore revision bump
	lib/cmdlib/cluster.py: trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

In tests, give Ganeti enough time to actually start up jobs before
asserting that they succeed. While normally forking and executing
a job is finished in less than a second in some circumstances it
can take longer; so give enough time to avoid flaky tests.

While there, also pause the watcher, so that it doesn't submit
jobs inbetween that cause our 0.01 second delay jobs to take
longer while waiting for locks taken by the watcher's jobs.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This test verifies a global limit on the number of jobs running. This
requires knowledge of all jobs submitted between the addition of the
filter and the submission of our last test job. While we send these
commands directly one after the other, this still takes a second or
two, thus giving the watcher slightly less than a 1% change of
interfering. Avoid this by pausing the watcher during this test.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

The test submits 3 jobs in an environment rate-limited to two, and verifies
that the first 2 jobs are running whereas the third one remains queued. However,
it takes for Ganeti some time to get a submitted job to actually being confirmed
waiting (in some circumstances over a second). So wait a bit longer to avoid
losing the race. While there, increase the time of the delay jobs to avoid
them finishing during that test in an extremely slowly running QA. (Note that
the jobs are killed forcefully at the end of the test, so that time is never
really waited for.)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This patch, extends the 'ganeti.utils.storage_unittest.py' unit test
with a new test for the Gluster storage type. Also, the current unit
tests are updated accordingly to correspond to the latest changes.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 42fdf9d0Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch, extends the file-based storage type verification checks to
include the new Gluster storage type. Those modifications include a
missing check for the gluster storage directory at cluster
initialization time, and extends the 'gnt-cluster verify' command to
also verify the gluster file-storage path.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 604c8be2Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch, adds the 'gluster' storage type to the set of storage types
for which full node storage reporting is available. This set is used by
the 'LUNodeQueryStorage' logical unit for getting information on storage
units on node(s).
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from 7ec47185Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Currently, the 'gluster' and 'sharedfile' disk templates are both mapped
to the Shared File storage type. This compromise causes the Gluster block
devices to be interpreted in the config as 'sharedfile' devices, instead
of 'gluster', and subsequently to be wrongly handled as Shared File
disks. This behavior, currently makes the 'gluster' disk template not
functional.

This patch, fixes this issue by introducing the Gluster storage type,
which will be exclusively used by the 'gluster' disk template.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked from ba550291Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.11
  (no changes)

* stable-2.10
  Check for gnt-cluster before running gnt-cluster upgrade
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

The test submits 3 jobs in an environment rate-limited to two, and verifies
that the first 2 jobs are running whereas the third one remains queued. However,
it takes for Ganeti some time to get a submitted job to actually being confirmed
waiting (in some circumstances over a second). So wait a bit longer to avoid
losing the race. While there, increase the time of the delay jobs to avoid
them finishing during that test in an extremely slowly running QA. (Note that
the jobs are killed forcefully at the end of the test, so that time is never
really waited for.)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stbale-2.10
  Check for gnt-cluster before running gnt-cluster upgrade
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

With Ganeti imposing no requirement on Xen versions, the configuration
file that should be present varies greatly on the choice of toolstack
and Xen version. xend-config.sxp was considered obligatory, but in
higher versions and with xl, it is superseded by xl.conf and considered
deprecated. This patch makes it have the same status as xl.conf - Ganeti
will distribute it but not require its presence.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When ganeti is removed (not purged) `/etc/cron.d/ganeti` is not deleted,
thus after a reboot cron tries to execute gnt-cluster upgrade and fails.

The same pattern is used on all other cron entries.
Signed-off-by: Christos Trochalakis <christos@skroutz.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

..  with the latest bugfixes and known issues
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

After cherry-picking the changes needed for GHC7.8, the former isn't
used any more.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.13
  (no changes)

* stable-2.12
  Expand orphan volume test
  Restrict Ganeti's orphan volume checks to the single VG
  Modify UDS server startup to set permissions for sockets
  Add wheezy chroot files to gitignore file
  Makefile.am: Don't use -dynamic-too for .hpc_o files
  Makefile.am: Don't use dots in -osuf
  Fix compiler invocation for GHC >= 7.8
  Makefile.am: Fix wrong -dep-suffix for GHC 7.8
  Only upgrade configs not upgraded
  Only unlock config if we did lock it
  Mention preferred DRBD module settings when using Xen
  Avoid assertIn
  Test presence of public and private parameters
  Put private parameters into the environment
  Always close pipe on job forking
  Clean up pipes early on failed forks

Conflicts:
	.gitignore
	Makefile.am
	lib/config/__init__.py
	src/Ganeti/WConfd/Server.hs
	test/py/cmdlib/cluster_unittest.py

Resolution:
        .gitignore: Join additions
        Makefile.am: Use Niklas' changes
        __init__.py: Take the 2.14 version, incorporating cherry-picked
                     fix already
        Server.hs: Take 2.13 imports
        cluster_unittest.py: Take 2.13 change with changed import
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.12
  Expand orphan volume test
  Restrict Ganeti's orphan volume checks to the single VG
  Modify UDS server startup to set permissions for sockets
  Add wheezy chroot files to gitignore file
  Makefile.am: Don't use -dynamic-too for .hpc_o files
  Makefile.am: Don't use dots in -osuf
  Fix compiler invocation for GHC >= 7.8
  Makefile.am: Fix wrong -dep-suffix for GHC 7.8
  Only upgrade configs not upgraded
  Only unlock config if we did lock it
  Mention preferred DRBD module settings when using Xen
  Avoid assertIn
  Test presence of public and private parameters
  Put private parameters into the environment
  Always close pipe on job forking
  Clean up pipes early on failed forks

Conflicts:
	test/py/ganeti.backend_unittest.py

Resolution:
        Merge all test changes.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

to ensure we are not reporting stray volumes on other VGs as orphans.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Prior to patch eeda5882, Ganeti checked all the LVs on nodes under
its control, and dutifuly declared any stray volumes as orphans in
warnings emitted during cluster-verify. After the patch, the nodes
returned information related only to the LV set as *the* Ganeti VG,
bypassing this problem.

Unfortunately, this led Ganeti to report the disks of instances created
on a different VG as non-existent. This patch fixes things by reverting
patch eeda5882 and making Ganeti warn about orphan volumes only if
they belong to the Ganeti-administered VG.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When opening domain sockets for communication, the Haskell daemons did
not set any permissions for the sockets, defaulting to 0700. This was
fine when all of them ran as root, but was bound to cause trouble in a
split-user setup. The first issue is RAPI access after master-failover,
where RAPI could not send make any inquiries until the watcher restored
the desired permissions of the socket.

This patch modifies Luxid to use a g+rw socket, and leaves other servers
to their default of 0600.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

It turns out that GHC 7.8's -dynamic-too is forbidden not only for
profiling builds object files, but also for those using HPC coverage.

This commit accordingly renames HFLAGS_NOPROF to HFLAGS_DYNAMIC
(since it now is not conditional on profiling any more),
and makes sure that it is not used for profiling or HPC.

This way we achieve that, for profiling+coverage builds
  - .dyn_o files are available for use in TH in the following 3 cases
  - .o files are for the normal binaries
  - .hpc_o files are for coverage-enabled test binaries
  - .prof_o files are for profiling binaries

We make .hpc_o files depend on the .o files because the creation of
the .o files will also create the .so files needed for TH.
This was already in place for .prof_o files.

This requires that HFLAGS now also contains `-itest/hs` since
the rule for .o files is now also responsible for building .o/.so files
in test/hs.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked-from: 1ad14f38Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This fixes
  ./configure --enable-haskell-tests && make hs-tests
builds on GHC 7.8.

It changes `-osuf .ext` to `-osuf ext`; the old form was allowed in
GHC <= 7.6, but GHC 7.8 seems to have problems with it:
  https://ghc.haskell.org/trac/ghc/ticket/9760

The form without dot works in all GHC versions, and we don't have
dots in other -osuf arguments in the same file either, so this
was probably an accident.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked-from: 9664aff9Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

GHC 7.8 switched to dynamic linking being used for ghci, which requires
that .so file are being built for modules that are loaded for TH.

For this reason, GHC >= 7.8 has a -dynamic-too flag, which we now use.

However, -dynamic-too must not be enabled for profiling builds,
which is why this commit introduces a HFLAGS_NOPROF variable
(currently only containing -dynamic-too) that must be passed to
all GHC invocations that are not creating profiling output.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked-from: 083776b9Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This works around https://ghc.haskell.org/trac/ghc/ticket/9749:

GHC 7.8 (undocumentedly) changed the way in which ghc -M generated
object file dependencies, lacking the underscore that older versions
added automatically to the file names.
It also requires a -dep-suffix for the plain object file (.o).

This commit detects GHC 7.6 and older (7.7 is development only, has no
release and is treated equal to 7.8), and adjusts the -M invocation
appropriately for newer GHC versions.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked-from: b78a2c30Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Whenever we set a configuration, we also upgrade it.
In particular, there is no need for an unconditional upgrade;
so avoid upgrading the configuration over and over again
if it did not change.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

We only acquired a config lock if it was not shared in order
to have lock-free reads. Hence, only release the config lock
if we actually acquired it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>

Cherry-picked-from: 35056743Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add a mention into the install file, and provide a link to the official
DRBD documentation.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...as we still support python versions older than the introduction
of assertIn.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Make sure that there is an environment variable for each parameter
with the correct value.
Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

and unobscure them
Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

...even if the initial hand shake succeeded.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...so that we do not accumulate open file descriptors.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>