GitLab

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

First, we update the recommended hlint version to what I used to get a
clean output (1.8.15). Most of the changes are:

- remove unneeded parentheses
- some simplifications (intercalate " " → unwords, maybe … id →
  fromMaybe, etc.)
- removal of some duplicate code (in previous patches)

There are still some warnings which I didn't clean out but plain
ignored:

- 'Eta reduce' in some specific files, because the type inference
  specialises the function on the first call, and annotating the type
  properly would be too verbose
- use of 'first', 'comparing', and 'on', since these don't seem to be
  widely or consistently used (outside ganeti/htools, I mean)
- use of Control.Exception.catch, as we only care about I/O errors; at
  one point yes, we will need to transition to this new API
- 'Reduce duplication', since hlint warns even for 3 duplicate lines,
  and abstracting that away seems overkill to me

After this patch, make hlint is clean and doesn't exit with an error
anymore; we could enable it automatically on 'make lint' if hlint is
detected (future patch).

Note that we explicitly skip the THH.hs file from checking because it
seems that hlint doesn't parse correctly for now the splice notation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, the code in Node.hs is overly strict: once a node's free
memory reaches 0, it will refuse to add any instances (offline or
not). I think this is a safe safeguard (I don't expect nodes to run
without at least 1MB of free memory), so rather than change this
behaviour we need to restrict the Node generation in the unittest to
skip such nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a few niceties to the test suite:

- allows matching test groups case insensitive and emit warnings when
  we give test group names that don't match anything
- add a new operator that is similar to assertEqual in Python: it
  tests for equality and emits the two values in case of error
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This makes error message change from "Test 4 failed …" to "Test
prop_Loader_mergeData failed", which is much more readable. It also
removes the duplication of test suite names in the test.hs file.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This changes the names for some helper functions so that future
patches are touching less unrelated code. The change replaces
shortened prefixes with the full type name.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Originally, this data type was used both by instance allocation (1
result), and by instance relocation (many results, one per
instance). As such, the field 'asSolutions' was a list, and the
various code paths checked whether the length of the list matches the
current mode. This is very ugly, as we can't guarantee this matching
via the type system; hence the FIXME in the code.

However, commit 6804faa0

 removed the instance evacuation code, and thus
we now always use just one allocation solution. Hence we can change
the data type to a simply Maybe type, and get rid of many 'otherwise
barf out' conditions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

The tryEvac/evacuateInstance functions are no longer used in the new
multi-group world order, so we remove them and change the unit-test to
test the actual IAllocator function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… and failover too. Not many changes otherwise except for
serialisation and unittests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds tests for the opToResult and eitherToResult functions from
Types.hs, and changes two other tests for the same module to test JSON
serialisation (which automatically also tests the lower-level to/from
string conversion functions).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Tested only on GHC 7.x, will test on 6.1x too before commit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds the binaries code to the coverage, and thus the coverage
finally shows the real coverage over all logic code (except for the
htools.hs code, which is not logic code related to the algorithms, so
it doesn't matter — plus it's also very small).

Next steps will be to actually add coverage for this code, especially
for hbal and hspace, which are relatively big compared to hail and
hscan (around 800 expressions versus 200-300 expressions).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the tieredAlloc/iterateAlloc functions will not return
until the allocation fails; this means unit-testing their
functionality (e.g. that an instance can be allocated) is slow, since
they will allocate all possible instances.

This patch adds an optional limit that allows allocation to return
early; this makes the cluster unittests twice as fast.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In some cases, we have a [Node.Node] list, instead of a Node.List
structure. Instead of converting [Node.Node] to the latter, let's
rework compCV and add a compCVNodes such that we skip this
(potentially expensive) step.

This already allows a simplification in a unittest, and will simplify
consumers of Node.computeGroups.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the test prop_Node_rMem does some reserved memory checks on
nodes, and for the test to work correctly (can add more instances) we
require that the node free memory is greater than zero via a
prerequisite.

However, if the node free memory is exactly 1, the call to
setInstanceSmallerThanNode will set the instance memory to "free
memory `div` 2", and since this is integer division, the instance
memory will get set to 0. Which means that the node reserved memory
will not increase when we add this instance as secondary.

To work around, we simply require that the node memory is high enough
(we use our 'base' unit of memory).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is similar, but not identical, to Utils.ParseUnit. The biggest
difference is that we don't round up/down; as we only use integral
types, the result will always be rounded down.

Moreover, since (real-world) disk sizes come in SI units, the function
differentiates between SI and binary prefixes, using lower-case for
binary and upper-case for SI (similar to lvm usage). This distinction
should be ported to the Python code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Rather than running all possible tests in one go, split them out in
three cases: "normal", undefined default value, undefined list values
(both of which require a valid response before them).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

if' is just a functional version of the if-then-else construct. Using it
we can manipulate ifs as functions, and also build other functionality.

select allows to choose a result depending on truth value of statements,
with a final default, and can be used to substitute cascaded-nested
if-then-else statements.

Both come with unit tests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This requires that we change the signature of loadInst in order to
properly annotate the error messages, which in turn requires more
unittest changes.

Also, this invalidates yet again saved data files…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently always fixed to DRBD8, as that is what we assume already.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

… to autoBalance, per the recommended style.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

No code changes (except one definition being moved around in QC.hs)
are contained in this patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch fixes a bug in the test specification where we allowed nodes
with zero free memory (hence no instance can be added, at all) and adds
a simple labeling of the way this test can fail.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This unittest had a corner case where it could fail if the same
primary/secondary node names were generated.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Using new functionality in QuickCheck 2 (the suchThat function), we
generate now better test cases, such that (heh) we have no longer
incomplete tests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the way we generate nodes in some cases is by creating a
totally random node, then restricting the test based on whether the node
'size' (as defined by multiples of base unit) satifies some high/low
rules. This results in hard-to-satisfy conditions, so we change this
model to be able to specify node sizes directly in the generation
process, thus no longer needed post-creation filters.

This fixes prop_ClusterAllocBalance which before had at most 1-2
satisfiable tests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The new scaffolding which replaced the batch driver of QuickCheck 1 now
shows how many passes we have for incomplete tests. Some tests show very
low pass counts, so we rework them to have more actually valid test
cases.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Since current distros don't package anymore QuickCheck 1.x, let's move
to 2.x.

This requires also a few changes to the code:

- Test.QuickCheck.Batch doesn't exist anymore, so we need to write some
  scaffolding code to replace it
- the way test sizes are generated has changed, and we need to restrict
  (in some tests) the cluster size, as our code is not yet ready for
  hundreds of thousands of nodes in a cluster and we run out of stack
  (which could be a bug somewhere by itself, needs investigation)
- at least with GHC 7, floating point errors make a perfect cluster
  score even bigger, so we need to bump up the max. rounding error
  allowed
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Removal of duplicate parantheses, removal of extra 'do', conversion from
nested if to guards, use hierarchical imports. All per hlint.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Note that --no-disk-moves and --no-instance-moves are not incompatible,
but if both are used no solution can possibly exist.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>