GitLab

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add section for 2.12.0~beta1, document changes since 2.11 and known
issues.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. as it causes an instance to become unresponsive.

See 885: Network hotplugging on KVM sometimes makes an instance
unresponsive
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

- metad is only optional;
- Debian Jessie provides the rest of the required dependencies.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If somebody tries to modify the meta-data, the daemon must be enabled
during configure time.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In such a case, the UpdateMedatada function will just return.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In particular, skip its compilation, if the required packages aren't
present.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In particular, kvmd, luxid, metad and wconfd aren't dependent on
enabling confd.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

To expose certain attributes, the psutil module performs python magic
that results in an attribute being present in the module, but not in
the __all__ list. As pylint uses the __all__ list to determine whether
property accesses are errors, it is understandably confused about this
state of affairs.

To fix this, we just use the newer cpu_count attribute which is
supposed to replace NUM_CPUS anyway.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

b9e12624

 introduced a `KRInstanceConsoleInfo` constructor, but used it
as `KRConsoleInstanceInfo` (which can't possibly have compiled).
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This requires to compile each object file twice, once without profiling,
for TH splicing, and then once with profiling enabled.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Since we're installing cabal from scratch, we can take the advantage of
it and build everything with profiling enabled (including libraries that
are installed during the installation of cabal).

Also since we're using a newer version of cabal, we can let it install
packages in parallel, speeding up the schroot build.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.11
  (no changes)

* stable-2.10
  Adapt relative path expansion in PATH to new environment
  Exclude archived jobs from configuration backup
  Expose early_release for ReplaceInstanceDisks

* stable-2.9
  Make ganeti-cleaner switch to a save working directory
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.10
  Adapt relative path expansion in PATH to new environment

* stable-2.9
  Make ganeti-cleaner switch to a save working directory
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The test framework we use sets PYTHONPATH for tests so that
the python module under test is used. Unfortunately, the paths
in PYTHONPATH are relative paths. So we need to expand them for
this test. Here we just update our fragile work around, but the
only way the test is called is, well, in testing, so we can build
it for our test environment.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.9
  Make ganeti-cleaner switch to a save working directory
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

For some reason, find does stat the current working directory,
which, when invoked via cron, may or may not be readable by the
process (issue 880). Therefore, first change to a directory that
is definitely accessible to the process; the directory it logs
into is such a safe place.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Currently, when a job fails to starts, no error message is set, leading
to "OpExecError: None". Since the function already receives a message in
a ReasonElem, this patch:

- extends the reason trail message with a description of the error that
  caused the job failure and
- sets the message in the Op code result.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.10
  Exclude archived jobs from configuration backup
  Expose early_release for ReplaceInstanceDisks
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

For a cluster that existed for a long time, the archive of
old jobs might be big. Additionally, this data is neither
necessary for running new jobs, nor is it touched in the
upgrade procedure. Hence it is safe to be excluded from
the backup created on Ganeti version upgrades, thus obtaining
a smaller backup.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Otherwise logs get full of critical errors when the configuration is
being sent from WConfd to ConfigWriter in the middle of a job that
modifies the configuration (just as in the case of _WriteConfig).

This should be fully solved once WConfd itself does all the verification
checks (issue #759).
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In the middle of a LU, the configuration can be rightfully inconsistent
(for example a disk template is removed, another one added afterwards).
Therefore these checks produce a lot of unjustified critical errors in
the logs.

Only verify the configuration when commiting an Update call.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Replace it with FIXME not to flood logs so much.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Otherwise the configuration might not be fully distributed to the master
candidate and be in an inconsistent state, causing errors.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Previously if the standard output was captured, the content wasn't
printed, so it wasn't visible what happened in the case of an error.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The Python style has several restrictions that combine
very unfortunate in the given situation.
 - The value of a keyword argument has to follow
   immediately.
 - We may not have over long lines.
 - Indentation of arguments needs to be consistent.
The only solution that satisfies all these restrictions
I could come up with is to reindent all arguments of the
call to OpInstanceCreate.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from 5c23365c

.

Conflicts:
	lib/cli.py
Resolution:
	Redo the reformating on this branch
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Remove implemented ones from the draft list and add the ones fully or
partially implemented in 2.12.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.11
  Remove user-initiated shutdown design from drafts
  Improve startup/shutdown in the admin guide
  Improve user down documentation
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. as it has been already implemented in 2.11.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Instance creation supports opportunistic locking. So,
also export it on the command line. There is no reason
to have this feature RAPI only.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from e845c100

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Fixes issue 877.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

... namely, the 'user-shutdown' cluster parameter.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Change the way MultiMap is an instance of Arbitrary to
be a random term of the exported functions. In this way,
we really obtain a sample of all values of type MultiMap
that are definable outside the module. In particular, we
verify that all invariants of the implementation are preserved
over all exported functions. This renders the extensionality
test meaningful.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

With the recent changes, opportunistic locking should not
require retries any more on clusters that have enough capacity.
So it is fair to add the --opportunistic-locking option the
parallel instance creation tests. In this way, we
- verify that the absence of the need to retry is indeed true and
- get a realistic performance measurement for truly parallel
  instance creation.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from 37d6f7f5

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Explicitly state, that parallelism enhancing options are to be used
in parallel performance tests, if it is best practice to use them on
the respective versions. A prerequisite for those options is that
using them does not require, even under heavy load but assuming enough
capacity in the cluster, any jobs to be retried.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from a305cb96

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Change lock allocation in mcpu to honor the required minimum
of locks to allocate in opportunistic lock allocation.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from f3f1fc57

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In opportunistic locking, we always ask for as many locks
as possible to later chose what fits best. However, we
may also ask for a minimum as otherwise it cannot work
for principle reasons. For DRBD, this minimum should
be two, rather than the default one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Cherry-picked from ed94242f

.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>