While trying to understand why some code was not being tested, I
realised that we have a bad data type in Hcheck.hs.

We have "data Level = GroupLvl | ClusterLvl", but then we need to pass
the group name/index as well, so we have functions that look like the
following:

  f :: Level -> Maybe String -> …
  f GroupLvl (Just gname) = …
  f GroupLvl Nothing = … -- never used
  f ClusterLvl _ = … -- wrong, since a (Just x) could be passed

It's clear that we just need to move the group name in the "GroupLvl"
constructor, and simplify the call chain.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in
the future).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The modules are moved from the ‘top’ namespace to ‘Ganeti.HTools’, in
compliance with standard practices.

This patch moves the version string creation into a function in Utils
which shows some more information.

This patch adds a -V, --version command line option that shows the
program version and also updates the hn1 usage string (similar to hbal).

This just reorders some functions for a more logical ordering.

This patch changes the solution list to include a step counter so that
it's more clear these are successive steps (in a definite order), and
not just an unordered list of changes.

This patch changes the gnt-instance failover to migrate, and fixes a bug
in the formatting of commands.

This patch added a verbose output and changed the output so that by
default it is less verbose and more clear.

This patch adds a new instance move, FailoverAndReplace, which promotes
the old secondary to primary and then uses a new secondary node.

This is the last move that we can do within the limitations of one node
changed per move.

This patch moves the formatting of the command list to Cluster.hs and
enhances it with separator messages between the steps.

This patch adds a new replace secondary and failover move (equals to
“r:x f”), which can improve the solution (since we are testing more
options at each step).

Aligned the comments in Instance.hs

Currently hbal does multiple rounds, stopping when a rounds doesn't
bring improvements. With the recent changes to not remove instances from
the candidate list, this is obsolete as the first round will always run
to the end of the improvements.

This patch changes this so that the Cluster.checkMove function doesn't
recurse, but just computes the next best move (as its docstring says).
This means we can actually incrementally compute and print the solution,
and this is needed as otherwise an instance could move twice and the
second time it needs the current placement to compute the exact command
line and operation needed for the move.

This abstracts the individual placement solution so that it can be used
independently.

The current code restricts each instance to one move per round. This is
bad, as an computation restarted in the middle of the solution will have
a different set of instances to work and will thus lead to a different
end-solution.

Once this is applied, further rounds are not possible since the first
round will have tried all instances at its end. As such, the removal of
the rounds feature will be next.

The code adds a hard-coded 100 moves limit, which for big clusters is
actually small.

This prints the total memory/disk and also adds a header.

This is to keep in style with the other memory variables.

This is useful and not easy to compute otherwise.

This patch attempts to embed the VCS id in binaries, based on the way
other projects seem to do this.

This removes obsolete entries from the clean rule and adds the hbal
binary.

Since for a very many N+1 failures in a cluster, we could actually
degrade the N1 CV by making a node N+1 compliant, we need to make sure
this value only decreases when fixing non-compliant nodes.

The easiest way is to compute the N+1 score as a percentage of failed
nodes, with the caveat that the domain of values might not be fully
compatible with the other scores. It is still [0, 1] but does not vary
like the others.

This patch adds two new variables to the cluster score:
  - variance of the failN1 attribute
  - variance of the reserved memory percentage

The variance of the failN1 helps make the cluster N+1 happy, whereas the
reserved memory percentage helps balance the unused memory for
redundancy on the nodes.

This patch adds the node attribute “reserved memory percentage” that is
derived from the maximum reserved memory for a node and its total
memory.

This will be useful for enhancing the balancing algorithm.

This patch adds a score variable to the placement type, so we can record
the changes in the cluster CV for later display.

This gives visibility in the decrease of the parameters and can show
which are the most important steps to perform (out of the full move
list).

This patch adds printing the initial and final cluster coefficients in
hn1 too, to better understand the found solution.

This patch removes the primary/secondary instance lists from the node
status and also removes the tabbed formatting with explicit width
formatting.

This patch makes the tabular solution list nicer, by changing from tabs
to explicit widths.

Learned how multi-line string literals work in Haskell :)

This patch automatically removes the longest common (domain, i.e.
starting with a dot) suffix from the node and instance names. This gives
a much clearer display, and this format is compatible with the way
Ganeti accepts shortened names.

Based on the node changes, we remove the N+1 check and only show a
warning instead.

Currently, we fail a new instance placement if the new node status is
not N+1 compliant. This means that an allocation on an already N+1
failed node still fails, even though (conceptually) we're not worse than
before.

This patch changes this model to fail the allocation *only* if the node
was N+1 compliant before. This allows balancing to work on non-N+1 happy
clusters, with the caveat that they probably won't be N+1 happy at the
end.

Since we skip N+1 check in some cases, we add a new “failHealth” check
that verifies the node still has strict positive free memory and disk
space.

This patch adds a '*' character to nodes which are not N+1 compliant to
the output, to help with understanding pre- and post-changes cluster
status.

Since we only use the totals for computations, and we always convert
them via fromIntegral, let's just store them directly as Doubles.

This just makes indendation nicer in many expressions.

This patch changes the computation of p_mem / p_dsk from on-demand
(whenever the cluster stats are computed) to after-modify (after a node
is modified, we update its stats). This brings a god speed-up as only
one node or two are usually changed between cluster-wide stats are
computed.

This patch flattens the two folds into one, by simply building the whole
list of moves instead  of the double recursion (nodes and the each
node's moves). This has no functional change, but it's much cleaner.