Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

* Expand the Name column of the table (for c-delay-target)
* Add the c-* DRBDparameters to the table containing the disk parameters
* Add the unit of measurement in square brackets, when needed
* Document the supported DRBD version, warn about the DRBD version
  needed for barriers and for the dynamic resync speed parameters.
* Add links to some documentation about the dynamic resync speed
  parameters

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This makes more explicit the field behaviour - previously an optional
field was detected via a "Maybe" constructor, and an optional one via
a "Just defval" one. With this, field behaviour become more explicit
than auto-deduced.

In THH.hs, I slightly changed the fieldVariable function to use the
field name (if the field is not renamed), so that we have the exact
same output as before.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

These were repeated needlessly; I hope I grouped them correctly.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

Add the disk-barriers and meta-barriers parameters described in the
design doc.

constants.py:
* add the needed LD and DT-level parameters, use the defaults provided
  at ./configure time;
* add constants representing which barriers should be disabled and the
  set of valid options.

lib/bdev.py:
* factor the barriers handling code to a class method, for testing
  purposes;
* implement the more granular version checking logic;
* use the LD level parameters;
* add stricter check on DRBD version (8.0, 8.2 or 8.3), as we do not
  support 8.4 yet.

lib/cmdlib.py:
* translate DT level parameters to LD level ones.

configure.ac, Makefile.am:
* set both disk and meta barriers parameters depending on the value of
  --enable-drbd-barriers.

test/ganeti.bdev_unittest.py:
* unit tests for the code that sets DRBD barrier parameters depending on
  the version.

doc/design-resource-model.rst:
* reword the description of meta-barriers;
* change all disk parameters names to use dashes instead of underscores.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Oops, forgot to check this before initial commit, sorry!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

We should also display the value we can't parse, otherwise debugging
is very hard.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This fixes two problems:

- first, when we deserialise a big object, showing its value is not
  useful, as it will hide the actual error message
- second, we shouldn't deserialise a container at once, because then
  we will lose the detail of which 'key' failed to deserialise; we
  change to manual deserialisation of each key/value pair, so that we
  can keep this information

The last point requires that we import JSON.hs into THH.hs, in order
not to duplicate functionality.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This system based on explicit types instead of ad-hoc rules
(e.g. instead of deducing from "Maybe Int" an optional field, we now
can say explicitly OptionalField ''Int). In the first phase, this will
be used for the equivalent of lib/objects.py, which has slightly
different rules than luxi/opcodes.

We should look at merging the two systems later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This can be used to test live servers; currently there's not direct
way to interact with a confd server, except for burnin's builtin tests
(which were the source of this file).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

While the 2.1 design is old and should be “immutable”, I can't find
documentation about the confd protocol anywhere else, so let's correct
the design doc.

The patch is mostly style changes, plus a clarification on the ‘query’
field of the request, which varies *a lot* per request type.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is useful for working with a node's hypervisor state, where only
the primary hypervisor will be authoritative.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

configure.ac:
* change the documentation of --with-lvm-stripecount parameter to
  reflect the change

doc/design-resource-model.rst:
* change drbd/stripes to drbd/data-stripes and drbd/metastripes to
  drbd/meta-stripes

rest of files:
* add the plain/stripes, drbd/data-stripes and drbd/meta-stripes disk
  parameters

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* add the params attribute to BlockDev, and add the corresponding
  parameter to all the BlockDev classes;
* change the Create, Assemble and FindDevice factory functions interface
  to accept as parameters an objects.Disk instance and a list of
  children block devices; update their callers;
* make the factory functions provide default values for params if
  needed;
* factor out a check in the block device factory functions.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

objects.py:
  * add disk parameters to Disk, Cluster, NodeGroup.

constants.py:
  * add dictionaries that will hold types and default values for disk
    parameters (for now, empty).

test/ganeti.constants_unittest.py:
  * add unit tests for consistency in disk parameters default values.

rest of files:
  * add to gnt-cluster and gnt-group the options to manipulate disk
    parameters.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A quick QA run successfully finished with these changes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

Commit 78519c10 broke everything. Here's the fix.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These fields just return the node attribute's contents. They will be
used by the watcher to detect out of date node states.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- Report memory used by hypervisor (“mem_hv” as per resource model
  design document, “xmem” in htools)
- Also report number of CPUs available to Dom0
- Some other, small changes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This will be stored in the node object and used for calculations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- Data objects
- Serialization/deserialization
- Unittests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Keeping the node state up to date will require information from multiple
VGs and hypervisors. Instead of requiring multiple calls this change
allows a single call to return all needed information. Existing users
are changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

With this patch the “LockSet” and “GanetiLockManager” classes have a new
function to check if a single or a group of locks (at a certain level)
have been acquired in a specific mode. This will be used for additional
assertions. Until now they could only check if a lock has been acquired,
but not in which mode. One use-case will be updating the node state in
various places, where the node lock must be acquired in exclusive mode.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.5:
  ConfigWriter: Fix epydoc error
  ConfigWriter: Fix epydoc error

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.4:
  ConfigWriter: Fix epydoc error

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

* stable-2.5:
  ConfigWriter: Fix epydoc error

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

The parameter is called “mods”, not “modes”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
(cherry picked from commit 1730d4a1)

The parameter is called “mods”, not “modes”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

* devel-2.5:
  LUGroupAssignNodes: Fix node membership corruption
  LUGroupAssignNodes: Fix node membership corruption
  Fix pylint warning on unreachable code
  LUNodeEvacuate: Disallow migrating all instances at once
  Separate OpNodeEvacuate.mode from iallocator
  LUNodeEvacuate: Locking fixes
  Fix error when removing node

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

* devel-2.4:
  LUGroupAssignNodes: Fix node membership corruption

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

* stable-2.5:
  LUGroupAssignNodes: Fix node membership corruption
  Fix pylint warning on unreachable code
  LUNodeEvacuate: Disallow migrating all instances at once
  LUNodeEvacuate: Locking fixes
  Fix error when removing node

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Note: This bug only manifests itself in Ganeti 2.5, but since the
problematic code also exists in 2.4, I decided to fix it there.

If a node was assigned to a new group using “gnt-group assign-nodes” the
node object's group would be changed, but not the duplicate member list
in the group object. The latter is an optimization to require fewer
locks for other operations. The per-group member list is only kept in
memory and not written to disk.

Ganeti 2.5 starts to make use of the data kept in the per-group member
list and consequently fails when it is out of date. The following
commands can be used to reproduce the issue in 2.5 (in 2.4 the issue was
confirmed using additional logging):

  $ gnt-group add foo
  $ gnt-group assign-nodes foo $(gnt-node list --no-header -o name)
  $ gnt-cluster verify  # Fails with KeyError

This patch moves the code modifying node and group objects into
“config.ConfigWriter” to do the complete operation under the config
lock, and also to avoid making use of side-effects of modifying objects
without calling “ConfigWriter.Update”. A unittest is included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 218f4c3d)

Note: This bug only manifests itself in Ganeti 2.5, but since the
problematic code also exists in 2.4, I decided to fix it there.

If a node was assigned to a new group using “gnt-group assign-nodes” the
node object's group would be changed, but not the duplicate member list
in the group object. The latter is an optimization to require fewer
locks for other operations. The per-group member list is only kept in
memory and not written to disk.

Ganeti 2.5 starts to make use of the data kept in the per-group member
list and consequently fails when it is out of date. The following
commands can be used to reproduce the issue in 2.5 (in 2.4 the issue was
confirmed using additional logging):

  $ gnt-group add foo
  $ gnt-group assign-nodes foo $(gnt-node list --no-header -o name)
  $ gnt-cluster verify  # Fails with KeyError

This patch moves the code modifying node and group objects into
“config.ConfigWriter” to do the complete operation under the config
lock, and also to avoid making use of side-effects of modifying objects
without calling “ConfigWriter.Update”. A unittest is included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>