This is in preparation for the text format changes.

This was found, of all things, via lintian during the Debian packaging…

This increases the priority of fixing N+1 failures compared to balancing
metrics.

This helps with evacuation/failover of instances on 2-node clusters with
one one offline.

Currently we use the offline instance percentage (with range [0, 1]),
but this is not good, since we want the evacuation of such instances to
have a high priority; therefore we change this to a count of offline
instances, which has higher weight than a metric with range [0, 1].

For the RAPI and LUXI backends, we can get the actual memory usage (if
instances are running) via the oper_ram, whereas backend/memory only
tell what the instance will use at the next boot.

Not using oper_ram means that the node model is flawed and we consider
wrong values for the instance's memory (resulting sometimes in hilarious
values such as x_mem = -700 MB).

Commit e97f211e changed the iallocator backend to handle drained nodes as
offline. This commit completes that change by making the rapi and luxi
backend do the same (the text backend ignores any '?' values which are
returned by ganeti when nodes have problems, so it doesn't need this
change).

This really shows the need for actual dist-time full testing (not
unittests).

This patch adds reading of the exclusion tags from the cluster tags: any
tags starting with htools:iextags: will convert their suffix into an
exclusion tags prefix. In other words, "htools:iextags:service" will
cause any "service:X" tag to become an exclusion group.

This also shows them in hbal in verbose mode.

While these are not actually populated from the backends, and all the
programs ignore them, this patch contains the changes in the function
types required.

This allows us to get rid of two duplicate list length computations,
with a minor speedup.

The statistics functions are currently defined as polymorphic with a
Floating constraint. Changing this to monomorphic on Double type makes
them stricter and much more performant (~70% speedup). This is a cheap
way to recoup some of the loses incurred by the recent proliferation of
metrics.

This small patch adds the number of conflicting primaries in the cluster
score. This is different from the other non-CV metrics where we usually
compute the percentage of failing instances (for that metric); but for a
somewhat big cluster, 1-2% failing instances will be a too small value
to cause the relocation of conflicting instances (future patches will
also switch other non-CV metrics to this method).

This patch adds a new node list field (ptags), showing the primary
instance tags.

Since we don't want all instance tags to be used for exclusion, we add a
command line option to filter on these. Since the iallocator protocol
cannot accept command line options, currently it's not possible to
specify these for hail, and thus it will never use any exclusion tags.

This patch introduces exclusion of primary instances based on tags. This
is incomplete as currently all tags are being excluded, and we don't
optimise towards relocation of instances sharing tags on the same node.

… and read it in all the loaders. hscan is modified to save it to the
files it generates.

The attribute is not yet used in any place.

This is simpler than the concat operator.

The print nodes option can now accept an optional field list to
customise the output. This is ugly, since the field names do not match
the header names, but it is at least barely customisable (at runtime).

This will prepare for the runtime-selectable field list.

This fixed the unbalanced secondary instances on partially empty
clusters, and helps in general for the cases where real utilisation data
is not available.

Currently the JSON conversion in Rapi and Luxi are giving something
like:
  Error: failed to load data. Details:
  Unable to read Double

This doesn't tell one where the error is (in a node specification? and
which node? etc.). This patch annotates such messages with the owner
node:
  Error: failed to load data. Details:
  Node 'node1' key 'mtotal': Unable to read Double

For errors during parsing of the node/instance name (unlikely, but
still), the output is:
  Error: failed to load data. Details:
  Parsing new node key 'name': Unable to read String

Currently the fromObj function takes a JSON object which is then
converted into a list of (String, JSValue) in which we make a lookup.
However, most of the callers of this function call it repeatedly on the
same object, which means we do the object→list conversion repeatedly.

This patch converts it to take directly the list, and converts its
callers to do the conversion themselves (and only once).

While this is not in the hot-path today, it would be if we ever were to
process much data over Luxi (or RAPI), and is a good cleanup in any
case.

This imports PeerMap as P and reindents some lines.

This is a first attempt to get a readable output of tiered allocation
stats in hspace's output. Not very nice, but it should be somewhat
parseable.

Both the simulate and the tiered allocation mode take a machine spec on
input via a comma-separated list. This patch makes this a little bit
more consistent (always use disk,ram,cpu in this order).

This patch adds a function that, for some given failure modes, shrinks a
given instance in the hope that allocation will succeed when retried
with the new spec.

This patch reworks the internal handling of the instance spec CLI
option, and adds a tiered spec option that will be used in hspace to
enable the (auxiliary) tiered-spec allocation mode.

It also introduces a new data type for holding the instance
specification.

This allows us to do verification of option arguments in the assignment
functions themselves.

This doesn't need to be a monadic function, let's make it a simpler one.

Since the utilisation file might be generated at a different time from
the hbal run, and instances could dissapear in the meantime, it's better
to simply ignore unknown instances rather than abort.

This adds run status, resource parameters and load parameters for
instances.

This patch changes the signature and implementation of the function;
returning the item makes more sense (saves a lookup later again in the
container, and applying idx is cheap), and the previous implementation
was ugly.