As discussed in cover letter of patch series introducing hcheck, in
machine readable version we should always print values of all variables.
When hbal simulation was not run, FINAL values are equal to INIT values.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As per Iustin Pop's suggestion, we are printing group mappings
from group idx (known only to htools) to group uuid, which identifies
the group uniquely. This is required because group uuid and name
may contain characters such as '-' which are not valid parts of
bash variable name.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Instead of printing group id in human readable version of hcheck,
we now print group name.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As suggested during offline discussion, this patch adds a new bash
variable and description field in hcheck, which informs if the
rebalance was possible. This is to determine if there are split
instances on the cluster, in which case rebalance is impossible.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As suggested by Rene, I added comments to types for printKeys
and printFinal. Also, realigned some lines in Hspace.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This annotates the disks for the blockdev_remove where it is
appropriate. It leaves out 2 cases were we can't reliably annotate disk
parameters due to lack of knowledge what we should annotate. Those cases
affects only lvs used for drbd, so it doesn't affect the bug reported by
Constantinos.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is also related to the bug reported by Constantinos,
as we've only one getmirrorstatus_multi call in whole cmdlib, we just
annotate them while we are building the disk list.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Not annotating them works for DRBD but not for RBD as reported by
Constantinos.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Hinotify was not marked as a dependency for hconfd.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Simple tests for hcheck using shelltestrunner. Among other, we check
that we can run hcheck on multi-group cluster.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Machine readable output for hcheck.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Full hcheck functionality is implemented in this patch. Hcheck runs
per-group checks, provides summary for the entire cluster and simulates
rebalance if required, the printing the modified metrics.
In this patch, only the human-readable output is available.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Part of hcheck's simulation required us to silently run hbal. For that
purpose we expose iterateDepth and allow it to run silently.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As hspace and hcheck both use machine readable options, they require
similar functions - which are now generalized and moved to CLI and
Utils.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Introduce infrastructure required to add Hcheck and build it
successfuly.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Clarify some things, use ```` where appropriate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This updates the NEWS file for the upcoming rc1 release.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This would have caught our previous issue, since QA otherwise doesn't
do add, remove or changes of NICs.

Signed-off-by: Iustin Pop <iustin@google.com>
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

_PrepareNicModification returns the invalid type, which triggers an
assert resulting in a mysterious error:

Failure: command execution error:

Without any explanation. We fix this by removing the return value from
_PrepareNicModification, and instead returning the expected type
(since it differs per create/modification) from the (existing)
wrappers for this function. We don't need to return actual changes
from this function as _ApplyNicMods is the function that
computes/returns the formatted changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also provide some extended unittests to catch those cases.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is basically another case of commit d488adb6.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The function is supposed to return a list, not a string. Without this
patch the error message ends up as “C, a, n, ', t,  , c, h, a, n, g, e,
…”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Thanks to Agata Murawska, a random seed that generates bad test cases
was found ("seed was 1768143307 2118231514, test size 74"). This is
due to the fact that some nodes were generated with tMem=0, fMem=0,
which means that pMem was NaN and thus the node was not comparing
equal to itself.

Worked around by limiting the test case generation to nodes with
positive values; a better fix would be to compare nodes differently
(subset of properties or ignore NaN values).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Commit 8a69b3a8 changed the DRBD barriers disabled by default if
“--disable-drbd-barriers” is used to “bfd”, where “d” stands for
disk-drain. DRBD's upstream updated their documentation[1] to
say disk drains should always be used.

[1] http://git.drbd.org/?p=drbd-8.3.git;a=commit;h=f952dcfed57acf



Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

While reviewing the RAPI docs (for commit 003306f9), I realised that
the punctuation used in RAPI is inconsistent, especially with regards
to definition lists.

This patch changes all such lists to not have punctuation, except for
one case (the ecode descriptions, which have multiple
paragraphs). Furthermore, it fixes the few opcodes which had docs
ending with punctuation to not do so anymore.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Sadly we don't have a way to test that, for example, all
(G|N|I)Q_CONFIG fields are exported in RAPI, so for now we don't test
this; but we should, in the future.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The hypervisors are a list and need to be comma-separated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This prevents from setting for example drbd options on the plain disk
template.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This verifies the user (especially in nested dicts) does not
provide a key which is not seen in the defaults dict for that dict.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

For example, the “initrd_path” parameter is only available with xen-pvm,
whereas “boot_order” is only available with xen-hvm.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some parts of QA should only be run with certain hypervisors (e.g.
testing instance HV parameters). To make that happen we always define
the cluster's hypervisors and provide a function to get the default
hypervisor.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sorry, I broke the cfgupgrade unittests via 904910c4, since that
commit added the requirement for the "instances" dict in the
configuration.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since Ganeti 2.6 does warn about disks with wrong iv_names, it would
be better to fix them when upgrading to 2.6.

Not sure if this is too brittle and whether we want to add more
safeguards on the contents of the config/instance/disks before
touching them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These can be used, for example, to get jobs submitted after a certain
timestamp.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As reported on the devel mailing list by Christos Stavrakakis,
creation of plain instances is broken when the --no-wait-for-sync flag
is passed, because in that case WaitForSync is not called, hence
SetDiskID is not called at all, resulting in a None physical_id being
passed to backend.

We fix that by explicitly calling SetDiskID, which will cover the
pause/resume and os_add RPC calls.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch only touches comments and man pages.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>