In case a secondary node is already N+1 failed, currently the node
selection will accept a node that cannot start (at all) the new instance
as valid. This is wrong, so we add a new simple check to prevent the
case of instance's memory size being higher than the node's free (not
available, which might be lower than 0 for N+1 failures) memory.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will make it clearer than many if…then choices.

Currently we test each instance for relocation in checkMove; however, it
is a little more clear if we pass only the relocatable instances to
checkMove. The patch also slightly rewrites (indendation/style) the
second half of the checkMove function.

Currently in the balancing function we do two thing:

- take the decision where to do a new balancing round or not
- and actually computing the balancing round

This is not nice, as the two parts are conceptually separate, so this
patch splits the decision on whether to descend or not to a new
function.

It seems that set -e does not affect subshell (only simple commands),
and thus we don't actually get failures from make check being run in a
subshell. Rather than trying to handle this better, we remove the
subshell and invoke make with the required subdirectory.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>

This also updates its manpage, some information were really old.

This patch changes from the two separate files to a single file, with
sections separated by a blank line. Currently only the node and instance
data is accepted, later the cluster tags will be read too via this
format.

This makes all the programs accept the new format, but hscan doesn't yet
generate it.

This is in preparation for the text format changes.

We actually use stddev and not the coefficient of variance (as wrongly
noted before), so we update the documentation appropriately.

We also note that the dynamic load values must be pre-normalized, since
we don't do such a normalization in the code.

The live test needs a populated cluster, so let's write this down.

This changes the 'dist' rule to also do a check that the archive can
build all the programs and passes the check test itself, and shows the
sha1sum at the end automatically.

Ganeti/HTools/Version.hs is generated at build time from version (which
is the only one shipped), so it must be removed by the clean rule.

This was found, of all things, via lintian during the Debian packaging…

This increases the priority of fixing N+1 failures compared to balancing
metrics.

* master:
  Use the oper_ram field if available
  rapi, luxi: treat drained nodes as offline

This helps with evacuation/failover of instances on 2-node clusters with
one one offline.

Currently we use the offline instance percentage (with range [0, 1]),
but this is not good, since we want the evacuation of such instances to
have a high priority; therefore we change this to a count of offline
instances, which has higher weight than a metric with range [0, 1].

For the RAPI and LUXI backends, we can get the actual memory usage (if
instances are running) via the oper_ram, whereas backend/memory only
tell what the instance will use at the next boot.

Not using oper_ram means that the node model is flawed and we consider
wrong values for the instance's memory (resulting sometimes in hilarious
values such as x_mem = -700 MB).

Commit e97f211e changed the iallocator backend to handle drained nodes as
offline. This commit completes that change by making the rapi and luxi
backend do the same (the text backend ignores any '?' values which are
returned by ganeti when nodes have problems, so it doesn't need this
change).

This can be used to test that all the existing commands work correctly. It
needs a running cluster with at least one instance to run all the tests.

This really shows the need for actual dist-time full testing (not
unittests).

* next:
  Update documentation for the iextags
  Re-wrap the README
  Configure exclusion tags via the cluster tags
  hail: add '-p' option intended for debugging use
  Read cluster tags in the IAllocator backend
  Read cluster tags in the LUXI backend
  Read cluster tags in the RAPI backend
  Introduce support for reading the cluster tags
  Collapse the statistical functions into one
  Specialize the math functions
  Use conflicting primaries count in cluster score
  Node: add function for conflicting primary count
  Add a new node list field
  Add a command-line option to filter exclusion tags
  Introduce tag-based exclusion of primary instances
  Add a tags attribute to instances
  Small change in some list arguments
  Use either \- or \(hy in manpages

… since we added the fill-column 72 setting.

This patch adds reading of the exclusion tags from the cluster tags: any
tags starting with htools:iextags: will convert their suffix into an
exclusion tags prefix. In other words, "htools:iextags:service" will
cause any "service:X" tag to become an exclusion group.

This prints the initial node list on stderr, since stdout is reserved for the
iallocator protocol (even though ganeti won't pass -p itself).

This also shows them in hbal in verbose mode.

While these are not actually populated from the backends, and all the
programs ignore them, this patch contains the changes in the function
types required.

The tiered allocation output which contains spaces makes the output of
hspace non-sourceable. This patch adds a new function to ensure
non-alphanumeric values are quoted such that the output can be parsed
easily via the shell.

The patch also fixes a bug in the DSK_AVAIL key (found after adding the
quoting) which added an extra space at the end of these keys.

This allows us to get rid of two duplicate list length computations,
with a minor speedup.