The change is similar to the text loader change.

This change, which will be followed by similar changes in the other
loaders, splits the parsing of the data from the actual loading from
disk. Since the parsing doesn't usually involve IO actions, we will be
able to better test the parsing. The loading becomes a smaller part of
the code and thus inability to test it has a smaller impact.

This break compatibility with Ganeti pre-2.3.

* devel-0.2:
  Fix tag exclusion weight

Currently, the tag exclusion metric has a weight of one, which means
there might be cases where we won't move instances around because it
upsets the cluster metrics. However, we do want to make a higher effort
for cleaning up tag collisions, so we increase the weight to an
empirically-determined value of 2.

Pandoc 1.5.x uses the locale information to parse its input files (only
1.5, pre and post version use always UTF-8). Hence we need to enforce a
UTF-8 locale for proper parsing of input files.

This simplifies the maintenance of the man pages, and unifies the rst-to-*
converter to pandoc.

This is a work in progress, will be modified along with the progress
of Ganeti 2.3.

Currently, the key metrics/tiered spec computations show the virtual cpu
count. However, since we do have a maximum ration Vcpu/Pcpu, we can also
show the “normalized” cpu count, i.e. the equivalent physical cpu count
corresponding to the virtual ones.

Currently, hbal will abort immediately when requested (^C, or SIGINT,
etc.). This is not nice, since then the already started jobs need to be
tracked manually.

This patch adds a signal handler for SIGINT and SIGTERM, which will, the
first time, simply record the shutdown request (and hbal will then exit
once all jobs in the current jobset finish), and at the second request,
will cause an immediate exit.

Also adds them in hbal.

Recent hbal seems to run many steps for small improvements (< 1e-3), so
we should stop early in this case.

We add a new option (-g), that will be used for the minimum gain during
balancing. This check will only become active when the cluster score is
below a threshold (--min-gain-limit), so as to not stop rebalances too
early.

This will make the automated builds flag any problems.

These are just variations of the standard debug, but are provided for
simpler code, since lazyness is something causing non-computation of
debug statements.

The addition of a new secondary on a node is doing two memory tests:
- in strict mode, reject if we get into N+1 failure
- reject if the new instance memory is greater than the free memory (not
  available memory) on the node

The last check is designed to ensure that, irrespective of the other
secondary instances on this node, we are able to failover/migrate the
newly-added instance.

However, we should allow this, if the instances comes from an offline
node, which doesn't offer anything (not even disk replication).
Therefore this patch makes this check conditional on the strict mode.

Otherwise the saved cluster state and the in-memory one are wrong.

This also uncovered a few issues with the allocation model (instances
not being marked up, etc.).

Compared to hbal, hspace will generate either one or two files (for both
the standard and the tiered allocation mode), depending on the input
parameters.

The Cluster.iterateAlloc and tieredAlloc functions are changed to also
return the updated instance list, since it is needed to have a “full”
cluster view.

Also move the LUXI execution (-X) to the end, after all the output
messages are printed. No good in waiting for the messages for a long
while, especially as they are not up-to-date stats after the job
execution, just an estimation of what the state will be.

This is currently hardcoded in an internal function in hscan.hs, and we
move it to Text.hs for later use.

This option will in the future be used to serialize the cluster state in
hbal and hspace after the rebalance/allocation steps.

This checks that the Node text serialization and deserialization
operations are idempotent when combined other.

Currently, the hostnames are almost fully arbitrary chars, which breaks
the assumption that nodes/instances will be normal DNS hostnames.

This patch adds some custom generators for these hostnames, that will
allow better testing of text loader serialization/deserialization.

Currently these are in hscan, and cannot be reused easily.

Again, thanks to lintian.

Currently we show the instance index, but this makes no sense outside
the current running program. Instead, we show the instance name.

This looks better for text-only viewing…

If some clusters failed during RAPI collection, exit with exit code 2 so
that tests can detect this failure.

The (recently-enabled) live test coverage stats found a few low-hanging
fruits in the tests we do…

… which fixes the issue noted in the previous commit (almost a brown
paper bag change).