This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.

These are just a few opcodes we need for executing instance moves.

This will allow a more descriptive output later in the solution list, as
opposed to trying to reconstruct the move from the node indices.

The patch also documents the Placement members.

The Luxi loader implements both a generic Ganeti Luxi client and the
loader; it is better if these two are separated. The patch adds a
Ganeti/Luxi.hs (not under HTools!) since that is generic for Ganeti, and
not related necessarily to htools.

Since moving two instances between different node-quadruples (inst X: A,
B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, it
makes sense to split the operation list into jobsets whose execution
must be serialised, but whose individual jobs can be parallelised.

Of course, this doesn't help, because today a single logical job is for
some cases split in multiple gnt-instance invocations, which (if run
using ‘--submit’) might be wrongly parallelised by Ganeti. So while for
now this is mostly a cosmetic hint to the user, and it just a step for
future improvements.

Due to the Control.Exception changes between 6.8 and 6.10, using it
portably is difficult. Since we're only interested in handling I/O
errors, we can use prelude's catch and not have to deal with
Control.Exception at all.

The use in Luxi.hs where we just use bracket is fine across the two
versions, so we keep that.

Haddock doesn't like pre-processed files (at least not in all versions).
Thus we need to remove the ExtLoader module from the haddock-procesed
file list.

Test.QuickCheck.Batch.runTests doesn't return any error statistics,
which makes the test suite just display errorrs and always exit with
exit code 0. This is not good, since one cannot then actually batch run
tests.

This patch adds a wrapper over Batch.run which also modifies a passed
IORef Int to keep track of how many test failures or test aborts we had.
This makes it easy then to exit with an appropriate exit code.

The Makefile was intented to be -Wall and not simply -W, but I missed
that. This enables more warnings and also enables -Werror (except for
the tests).

During testing I used the test inversely to see it triggers correctly,
and committed by mistake the inverted test. Fixing it.

Since curl is not always needed (e.g. when only using luxi or less
likely file backends only) and is also not always available, it is
useful for building without it. This of course disabled the RAPI
backend.

This patch changes ExtLoader to build with the ‘-cpp’ option which makes
ghc run it through cpp first; and based on whether ‘NO_CURL’ is defined
or not, this toggles RAPI/curl inclusion. The patch also removes the
import of Rapi in QC.hs since it's not actually used in tests.

Invoking make as ‘make HEXTRA=-DNO_CURL’ is enough to trigger the new
build mode.

Currently the external data loader is in CLI.hs, which makes all
programs that need cli functionality (options, etc.) link against the
network modules (most importantly curl). This patch splits this
functionality into a new module such that (for example) hail which only
deals with file I/O doesn't link against these libraries.

This patch fixes a logic bug in luxi that breaks receive of messages
bigger than 4096 bytes. The send message is not impacted as it uses a
different algorithm.

This include instance text load tests.

Currently the computation, recursing part and the IO part (progress
updates) of the balancing main function (iterateDepth) are all in the
same function, which makes it hard to test. This patch moves the
decision/computation part (whether to proceed one more round, whether we
got a good result, etc.) into Cluster.hs, and leaves only the iteration
and screen update in hbal.hs.

This patch adds support for cheap (failover/migrate) operations only in
the balancing algorithm and in the hbal command line options.

This allows a very quick balancing (compared to allowing replace-disks)
which can be useful as a scheduled operation.

This fixes one warning from hlint.

While we can't guarantee that the instance will be in the same state by
the time the migrate/failover command will be run, we can at least try
to do the right thing assuming no other changes to the cluster state.

Of course, the right fix would be for Ganeti to accept a
migrate-or-failover opcode that atomically does the move…

Instead of using ioError . userError, we format the error ourselves.
This is nicer - no ‘)’ at the end of the output.

This is useful especially for hspace, where we might want to simulate a
hypothetical cluster to check allocation beforehand.

Signed-off-by: Guido Trotter <ultrotter@google.com>

This patch adds an error handler for any exceptions that are raised
during the external data load phase. This can be improved further, but
it's a good start.

This patch moves all the command line options and their internal
representation into CLI.hs. This means that duplicated options between
any two binaries are no longer declared twice, and that we no longer
need the two *Option classes.

This patch beautifies the man pages for hbal and hspace.

This patch makes CLI abort if more than one backend is selected.

This patch changes the backend selection method in CLI to prefer, in order:
  - a RAPI specification
  - a Luxi specification
  - and finally the node/instance files

It also modifies hspace and hbal to provide a ‘-L’ command line option
for enabling Luxi.

This patch adds a luxi backend that allows direct query of the master
daemon on the local node. This patch doesn't enable the backend to be
used.

There are a couple of things still missing in the implementation:
  - we don't have a master timeout in reads and writes, only a
	per-recv/send-call one
  - the failure reporting is not as good as it could be

The patch adds two constants in Types.hs for connect and query timeout,
then modifies Rapi.hs to use them as the connect and general curl
timeout.

Rapi could be improved more, as currently we wait double the total
timeout due to not aborting early in case the node queries failed.

This is basically a hspace release, so the changelog is small.

Currently hspace doesn't handle failures from tryAlloc correctly; this
patch changes the iterateDepth function in hspace to return a Result (…)
so that errors can be propagated correctly.

The patch also changes one output key to be more clear and a typo in
Cluster.hs

Currently, the tryAlloc and tryReloc function return a list with all the
results, both failures and successes. This is fine for hail, which does
one round of allocations, but is not so good for hspace, which does
iterative rounds; since at each (successful) step we only take the best
solution, it means that we're using lots of heap space to compute and
store node lists which are thrown away at the end of the step.

This patch changes these two functions and their callers in hail/hspace
to only return the best solution, and error/success counters. This
allows hspace to run in a much smaller space, and reduces GC cost
greatly.

Overall, it is a cleanup, as hail/hspace did a lot of work to chose this
best solution, whereas now it's automatically promoted within
Cluster.concatAllocs.