Bullet lists start after a blank line.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

DRBD 8.4.3 does not support specifying volume ids directly while
creating the meta device. It requires a slightly different syntax (using
flex-external as volume id) which is fortunately also supported by (at
least) DRBD 8.4.2.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* stable-2.8: (42 commits)
  Add shelltests for hspace allocation
  hspace: Handle multiple ipolicy specs
  QA: Test multiple instance specs
  QA: Handle multiple instance specs
  Unit test for cli.FormatPolicyInfo()
  Add command-line support for multiple specs in ipolicy
  Add multiple min/max specs in instance policy
  Separate checks for std spec compliance
  QA: Transpose instance specs
  Improve check for  "unreleased" versions in NEWS
  Update documentation for text format
  Add missing fields in htools text-backend documentation
  cfgupgrade: Remove enabled_disk_templates on downgrade
  Reason trail implementation for "start"
  Reason trail implementation for "shutdown"
  QA: More tests for instance policies in groups
  QA: Split function to set and parse instance policies
  QA: Update tests for new ipolicy specs command-line options
  Add unit tests for cfgupgrade with a real configuration
  Split functions in cfupgrade unit tests
  ...

Conflicts:
        lib/backend.py
        lib/bdev.py
        man/gnt-cluster.rst

lib/bdev.py was renamed to lib/block/bdev.py in master, so I've manually
applied the bc3427b7 commit to the new file. The other two are
straightforward conflicts.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

QA cleans up DRBD 8.4 devices via drbdsetup, which currently only worked
for DRBD 8.3. This (rather ugly) command makes sure that DRBD 8.4
devices are cleaned up as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The DRBD84CmdGenerator class, which generates commands suited for DRBD
8.4, is added. A common baseclass for DRBD83CmdGenerator and
DRBD84CmdGenerator is introduced as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Based on the version of DRBD (as reported through /proc/drbd), the DRBD8
class chooses helper objects which adapt for the differences between
DRBD <= 8.3 and DRBD 8.4. This new test verifies that the proper objects
are created.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Python Mock (http://www.voidspace.org.uk/python/mock/

) is a mocking
library for Python. It is included in the standard Python distribution
as of Python 3.3, but has been around for quite a while. Debian unstable
contains a version we require.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As the DRBD code was extracted from bdev.py, the corresponding tests
should as well be extracted into their own file. That's what this patch
performs.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In order to easily support the DRBD 8.4 command syntax, this patch
extracts all command generation logic (i.e. where calls to `drbdsetup`
are assembled) into a new class DRBD83CmdGenerator.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Now the DRBD8 class actually checks which version of DRBD is running on
the system and chooses the right class to parse the `drbdsetup show`
output.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Common functionality between the DRBD 8.3 and DRBD 8.4 parser has been
extracted into BaseShowInfo. A test which verifies the behaviour is
included, but the DRBD84ShowInfo class is not yet used within the DRBD8
class.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Parsing the status (/proc/drbd) and `drbdsetup show` output takes quite
a bit of code, which has to be augmented by even more with the DRBD 8.4
support. So extract all the related classes into their own file.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This class parses the `drbdsetup show` output and represents it in an
easily accessible format. It got extracted so that 1) the DRBD8 class
can focus more on DRBD logic rather than parsing and 2) it's easier to
adapt the parser to new formats.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The current parser already supports the output of DRBD 8.4 /proc/drbd,
so add tests which actually verify this behaviour.
Note that the empty lines in proc_drbd84.txt are actually present in the
output of DRBD 8.4, they always appear when minors are not contiguous.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The previous patch removed a (rather implicit) test for /proc/drbd not
containing a version line. This adds an explicit test for this case.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In the new context of the DRBD8Info class, the name
_JoinProcDataPerMinor doesn't fit any more. _JoinLinesPerMinor reflects
more precisely what's done, and doesn't imply that the data comes from
/proc/drbd.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As the DRBD8 class got bigger due to the previous merge of BaseDRBD, now
parts of it are ripped out into DRBD8Info. This new class parses
/proc/drbd and exposes the information in an easily accessible way. This
allowed to simplify some methods in DRBD8 and do make the tests more
concise.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

BaseDRBD was probably useful when DRBD 0.7 and DRBD 8 were supported.
However, there is only one subclass of BaseDRBD remaining (DRBD8), and
the separation of responsibilities between those two classes was rather
randomly chosen.

The unification into one class also eases the introduction of
DRBD 8.4 support, as the responsibilities of the class(es) will be
distributed anew.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

That's what the method actually does, so there is no reason why we
should use a funny but unreadable name.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Both tiered and standard allocations are tested, with a single and a double
min/max instance specification.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

With tiered allocation, hspace uses all the max specs in turn as the
initial instance spec.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The test of the interaction between policy and instances is extended.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Existing tests are updated to cope with the new instance specs format.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

We try to parse the string obtained by formatting a policy, and see if we
get the same policy.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Command line options accept multiple min/max specifications in instance
policies. The output of show-ispecs-cmd is updated accordingly.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Now instance policies can contain more than one min/max specs.  This is the
main element of the "Constrained instance sizes" section in the
"Partitioned Ganeti" design doc.

This is a big patch, but changing the type of a configuration item requires
to change all the code that handles it.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is needed to be able to validate the std spec against multiple min/max
spec pairs (appearing in next patches).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The format used to store instance specs in QA is changed to better handle
multiple instance specs, a feature contained in the following patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This patch makes the constant DEFAULT_ENABLED_DISK_TEMPLATES
a list instead of a set, because the order of the elements actually matters
as the first one is planned to be the default disk template for instance
creation. Also, this constant is always used like a list and thus providing it
as a set complicates the code that uses it.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This updates the man page of 'gnt-cluster' regarding the
removal of the '--no-lvm-storage'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Was accidentally introduced by commit 912737ba.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch does three things:

1. It deprecates the option '--no-lvm-storage' of 'gnt-cluster modify'
and 'gnt-cluster init'. Technically, it is not fully removed, but kept in order
to warn the user that it is no longer supported and that she should use
--enabled-disk-templates instead.

2. The consistency check between '--no-lvm-storage' and '--vg-name' is replaced
by checks between '--enabled-disk-templates' and '--vg-name'. There are these
cases:
- vg name, lvm disk template enabled = ok
- no vg name, lvm disk template enabled = error
- vg name, no lvm enabled = warning
- no vg name, no lvm enabled = ok
I added quite a lot of tests for all these and the transitions from each case
to another to the QA.

3. The check whether or not the volume group is available on all nodes is now
done only in these cases:
- the volume group name gets set and lvm is already enabled
- lvm is getting enabled and the volume group was set before

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch add a couple of utility functions dealing with
storage types and disk templates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes the problem with the distcheck-release target failing needlessly.

* stable-2.7:
  Improve check for  "unreleased" versions in NEWS

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Currently this is checked only when distcheck-release is called.
Check it in check-news instead, for all versions before the "current"
one (as defined by configure.ac)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In commit 000ca91a the text format was extended to also include
information about the master node. This commit updates the
htools man page to reflect this change.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Conflicts:
	NEWS
	doc/rapi.rst

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>