1. 04 Dec, 2012 37 commits
• Allow ignoring successful commands in "gnt-cluster command" · d5b031dc
Michael Hanselmann authored
```

In some cases it is useful to ignore the output of and avoid mentioning
successful commands. One would be when looking for a certain string in
a file:

\$ gnt-cluster command egrep -q '^testing\$' /etc/...
• Add "proper coloring" unittest check · c94f9990
Guido Trotter authored
```

We have to check that for each edge its vertices have different colors.

This is very easy to do with a vertex-to-color map, but not so easy with
a color-to-vertex one. Since all our coloring algorithms created a
vertex-to-color map behind the scenes and then converted it, we flip
them back to returning it directly, and do the conversion explicitly
where we need it (which for now is everywhere except when testing this
property).
• Fix Dsatur and add Dcolor · 8b50de5c
Guido Trotter authored
```

Our Dsatur implementation was incorrect: while the paper defined the
degree of saturation of a vertex as the number of different colors it is
adjacent to, we were using the number of colors, without considering
uniqueness. This effectively implemented a different algorithm, which is
very similar to the previous one, and while it performs slightly worse
on average it still beats Dsatur on some cases.

So we refactor the implementation to effectively support both algorithms
without code duplication, and then we export both the old algorithms as
"Dcolor" and the new one as "Dsatur". Since these are all fast
algorithms in hroller we will still be able to pick the best result.

Note that the new Dsatur implementation uses an IntSet to calculate the
uniqueness. Results with nub + length on a list were significantly
slower.
• HTools/Node: add mkNodeGraph function · dae1f9cb
Guido Trotter authored
```

This function helps treating node node problems as graph problems.
It can transform a list of nodes plus a list of instances into a graph
which uses the nodes as vertices, and instances as edges connecting them
(as long as they have both a primary and a secondary node)
• Add tests for verticesByDegree{Asc,Desc} · 34a21cc4
Guido Trotter authored
```

This brings our coverage of Graph.hs to 100%
• Ganeti/HTools/Graph Add isColorable · faef859e
Guido Trotter authored
```

Check whether coloring on a given graph makes sense. This is the case
only if there are no loops and the graph is undirected.
• Add Dsatur implementation · 742bd043
Guido Trotter authored
```

Implement the Dsatur algorithm for Graph coloring. This also abstracts
the neighColors function into two subfunctions that this algorithm can
reuse.
Guido Trotter authored
```

This module implements some algorithms on Data.Graph data structures.
At the moment its main functionality is an LF-color implementation
(greedy coloring in descending order of degree). There are also a few
extra functions to calculate the degree order, and convert the node to
color mapping to color to nodes.
• errors: Show error descriptions in API documentation · 402ff455
Michael Hanselmann authored
```

Comments with a colon after the hash sign (“#:”) show up in the
epydoc output.
• Small improvement to rpc-test · 4d829f55
Iustin Pop authored
```

This will show smaller/better error messages: full node dumps are no
longer included.
• Fix locking mistake introduced in commit 5cc1f887 · ef66500f
Michael Hanselmann authored
```

The node resource locks were not set correctly on instance import.
• Switch luxi submit job calls to use MetaOpCode · 7e723913
Iustin Pop authored
```

This patch changes the luxi submit job calls to use wrapped opcodes,
and therefore it changes Hbal to submit actual meta opcodes. For
nicety, hbal also submits a comment now, showing who generated the
job.
• hbal: restore nice job ID display · 2746a72c
Iustin Pop authored
```

This fixes:

Got job IDs JobId {fromJobId = 1052613}

And restores to:

Got job IDs 1052624

Other improvements could be done here, of course.
• Add safety check on job dependency/TIsLength · dd076c21
Iustin Pop authored
```

If TIsLength is applied to a non-container item, it will fail (type
error) due to invalid application of len(). Since this can happen on
user-supplied data, we add an explicit TList/TTuple check (the TTuple
test is a new one).
• Add CommonOpParams and MetaOpCode types · 4a826364
Iustin Pop authored
```

This patch adds the "meta" opcode type and the common op
params. Compatibility tests with Python are changed to pass Meta
opcodes.
• Remove unused THH function 'saveLuxiField' · d5af1f95
Iustin Pop authored
```This was deprecated via commit 88609f00

, “Switch Luxi TH code from
simple to custom fields”.
• Change opcode/luxi showJSON generation in THH · 84c2e6ca
Iustin Pop authored
```

Currently, the opcode and luxi "showJSON" functions generate directly
a JSValue; in contrast, the object (single-constructor) types generate
a 'toDict' function, and then `showJSON = makeObj . toDict`. This is
useful, as the 'dict' form can be manipulated if needed.

This patch changes the opcode and luxi types to behave the same; we
generate a dict, and then (since this differs between opcodes and
luxi) generate showJSON either as `makeObj . toDict` (for opcodes), or
(for luxi) `showJSON . map sn . toDict`.

The change is needed for MetaOpCode implementation.
• Add a type alias for simpler THH signatures · e45be9d4
Iustin Pop authored
```

This is reused in more than just a few places, so adding it makes the
signatures much nicer.
• Add types and parameters for common opcode implementation · b46ba79c
Iustin Pop authored
```

This will go into a separate type; this patch adds the needed
underlying types and parameters.
• Add a negative type · c67b908a
Iustin Pop authored
```

This mirrors the positive one, and will be needed for relative job
IDs.
• Add a 'real' type for JobIds · c48711d5
Iustin Pop authored
```

Currently, the job ID is a simple type alias. This is suboptimal, as
it means we can't use a custom JSON (or Arbitrary) instance for it.

The patch changes it into a newtype, and then a) simplifies some
deserialisation code and b) changes some more fields to this new type
(rather than plain 'Int').

We also move the JobId to types, since it will be needed in opcodes as
well.
• Add type for finalised job statuses · 6903fea0
Iustin Pop authored
```

For now, we don't need a pending job status type as well, so we'll
• Add test for opcode fields equivalence · d1ac695f
Iustin Pop authored
```

As opposed to the existing test, which tests the type/serialisation of
fields, this one simply tests the equivalence of the list of fields
for each opcode.
Iustin Pop authored
```Commit 213076fe

(“Fix locking in networks”) changed Python OpNetworkAdd
• Add a few missing fields from opcodes · 67fc4de7
Iustin Pop authored
```

Due to manual conversion, a few fields were missing from the
conversion, but as they were optional our type equivalence checking
didn't detect this.
• Rework/enhance original htools opcodes · 3d7e87b8
Iustin Pop authored
```

The original htools opcodes were minimalistic and not 1:1 equivalent
with the Python ones. Let's add all missing fields and, since we
changed the order, switch to more readable record syntax for building
the opcodes.
• Add a function for all fields of a given OP_ID · 3929e782
Iustin Pop authored
```

This patch changes THH to export a new function which defines all
fields of a given OP_ID. Not very clean, since for an invalid OP_ID we
return empty list, but since it will only be used in tests it should
be good enough.

The generated code looks as follows:

allOpFields :: String -> [String]
allOpFields "OP_TEST_DELAY"
= ["duration", "on_master", "on_nodes", "repeat"]
allOpFields "OP_INSTANCE_REPLACE_DISKS"
= ["disks", "early_release", "iallocator", "ignore_ipolicy",
"instance_name", "mode", "remote_node"]
…
allOpFields _ = []
• Rework custom fields handling · fa10983e
Iustin Pop authored
```

This patch changes a bit the handling of custom fields. Since in
general we use custom fields to aggregate multiple entries in the JSON
object into a safer data-type, we should also have a way to declare
which extra entries this field covers (so that in the future we can
say what are all the JSON keys for an object).
• Two small haskell-related Makefile improvements · af42ffb6
Iustin Pop authored
```

Just some cleanup:

- don't run hlint over hpc-htools, since it's just a symlink to
htools.hs (so we'd get duplicate warnings)
- build rpc-test in make really-all, via HS_ALL_PROGS variable
• Fix long lines in Ganeti.HTools.Types and add Make check · 9cda4d6a
Iustin Pop authored
```

Since hlint doesn't check style issues but rather code only issues,
let's add a simple Makefile check for too-long lines, and fix the
single bad case we have currently.
• Small Makefile.am cleanups · a2d7ea09
Iustin Pop authored
```

Indentation within rules is fixed to always be 2 spaces (as opposed to
mixed tabs/2 spaces/4 spaces). Additionally, the check-dirs rule,
which was more complex than necessary, has been simplified: moved
auto4mte.cache to DIRCHECK_EXCLUDE, and simplify the find call and the
exit condition (no need for if test …; then exit 0; else exit 1; fi).
• Fix generation of clusters via 'genEmptyCluster' · 5f4fdf93
Iustin Pop authored
```

The current uniqueness algorithm (generate random node names, suffix
them with node index) is actually wrong: a node named "21" at index 5
will end up with the same name as a node named "2" at position 15.

To fix this, we also add a character from a different "set" ("-"), so
that such mixups can't happen again, and also add an explicit check
for it.
• LUClusterSetParams: Use node allocation lock · dbd1264a
Michael Hanselmann authored
```

All resources are acquired and opportunistic instance creations would
• LUInstanceCreate: Acquire node allocation lock · 5cc1f887
Michael Hanselmann authored
```

Opportunistic locks are not yet used. This patch changes
LUInstanceCreate to acquire the node allocation lock to avoid conflicts
with other opcodes acquiring many node locks.
• Acquire node allocation lock for failover/migration · 2cb3181a
Michael Hanselmann authored
```

See code for an explanatory comment. The lock is released as soon as
possible.
• Use GetMultiInstanceInfo in LUNetwork* opcodes · 1dacbb06
Dimitris Aragiorgis authored
```

LUNetworkConnect/Disconnect looks up a nodegroup's instances for
conflicting IPs. To do so, use GetNodeGroupInstances() and
GetMulitInstanceInfo().

Additionally, check if the correct locks were acquired.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
• utils.text: Function to verify MAC address prefix · d242923c
Michael Hanselmann authored
```

The network management code needs to verify a MAC address prefix.
Instead of (ab)using NormalizeAndValidateMac, clean code should be used.
Unit tests for NormalizeAndValidateMac are updated and new ones for
2. 03 Dec, 2012 3 commits
• Improve the Haskell api doc directory creation system · cbcd9144
Michele Tartara authored
```

Instead of having two different lists of directories, now there is only one.
• Factorize code for checking node daemon certificate · 0602cef3
Michael Hanselmann authored
```

This code is going to be used by a new utility for setting up the node

Additionally, the certificate and key stored in “server.pem” are
verified, too.
```