Currently, all IAllocator requests return the same result type: a list
of strings (usually nodes, but can also be instance:nodes). This won't
be true in the future, with the new request types that will return
opcode lists.

Therefore we will need to support multiple result types in the
workflow; the first step is to make the final formatResponse step
result-type-indepedent, so we take out the formatRVal invocation out
of this function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The logic of request processing is not separated into its own
function, and (beside CLI interaction, e.g. verbosity handling) all
logic is now in IAllocator.hs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This will make it easier to unittest the code, and keeps all login in a
single module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a lot of duplication with modeMreloc, which will get sorted
out once we convert mreloc to change-group.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This was potentially hiding issues in the group query, so let's remove
it since we're now bound to the current Ganeti version.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The evac-mode goes into the Types.hs module since it needs to be
available to Cluster.hs too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

No code changes (except one definition being moved around in QC.hs)
are contained in this patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

First, auto-generated files were not processed if HsColour thus the
'source' link in Haddock documentation was dangling.

Second, we document the original Python constant name in the
Ganeti.Constants source, so that it's easily retrievable (as opposed
to undoing the name transformation).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Oops, this shouldn't have gone in in the first place…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Especially for Ganeti.Constants, which has many declarations, it
doesn't make sense to check for coverage.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In order to increase the amount of information returned from the
IAllocator plugin, we enhance the return values from the node-evac and
change-group operations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch allows commands to be run on and files to be copied to all
nodes within a specific group.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patc changes cli.GetOnlineNodes to use query2, which does the
filtering in the master daemon, and adds a new parameter to filter by
node group.

Unittests were added for the old implementation and then adopted to
ensure no functionality was lost.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For some reason these were still in tere.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Explicitely defining “__call__” silences a pylint warning when wrapped
type check functions are used directly. I had no idea pylint is this
intelligent.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.4:
  ht: Add new check for numbers
  Fix off-by-one bug in job serial generation
  Shorten some unbreakable lines in man pages
  Correct some spelling mistakes
  Fix bug in recreate-disks for DRBD instances
  Fix a lint warning
  KVM: configure bridged NICs at migration start
  Fix RAPI documentation regarding master role
  Fix bug in drbd8 replace disks on current nodes

Conflicts:
	lib/cmdlib.py: Trivial
	lib/opcodes.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Places which receive floats can usually also deal with integers, e.g.
OpTestDelay. Tests are added and the new check function is used for the
aforementioned opcode and verifying query results.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 009e73d0 (September 2009) changed the job queue to generate
multiple job serials at once. Ever since it would return one more than
requested.

The “serial” file in the job queue directory is defined to contain the
“last job ID used” (design-2.0). With the change above, the serial file
would always contain the next serial number. The first value returned by
the generating function was the one contained in the file, so during the
switch in 2009 one job may have been overwritten.

This patch changes the code to always return the exact number of
serials, to keep the last used serial on disk and adds an assertion.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commits 030a9cb8 and
ae082df0.

There are two problems:

- Makefile.am breakage, which is trivial to revert
- unittest breakage, which honestly I'm not sure how to fix and how
  serial consoles interact with the unpause helper

After the reset, the startup --paused still works but won't unpause
the instance automatically (if I understood the code correctly).

Furthermore, the code also fixes a style issue in hv_kvm.py (too long
line) introduced by the next commit after the above two.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Creates the instance, but pauses execution before booting. This combined
with 'gnt-instance console' unpausing instances means that the entire
boot process can be viewed and monitored.

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Use constants and an assertion
- Update documentation for node migration

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUNodeEvacStrategy has been replaced with LUNodeEvacuate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The change is not backwards compatible, see the updated NEWS file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

By default it'll now evacuate all instances from the node, not
just secondaries.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This new opcode will replace LUNodeEvacStrategy, which used to return a
list of instances and new secondary nodes. With the new opcode the
iallocator (if available) is tasked to generate the necessary operations
in the form of opcodes. This moves some logic from the client to the
master daemon.

At the same time support is added to evacuate primary instances, which
are also evacuated by default.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Am I the only one to make that mistake 10 times a week?

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Haskell has two main integer types:

- Int, which is a native-type, and is guaranteed to have at least
  [-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range
- Integer, which is a software type (implemented using libgmp), and
  thus unbounded

For performance reasons, the node/instance properties use Int for
their attributes (and Double for some, but that's another story). This
is all fine and doesn't cause problems. However, the CStats type which
holds the overall cluster resources starts to fail when we analyse
clusters with more than around 400 nodes and big memory/disk sizes on
32 bit platforms.

The simple fix would be to restrict cluster sizes, but that's no
nice. I've benchmarked and changing to Integer doesn't show a visible
slowdown on 64-bit platforms (as far as I can read on the internets,
GHC knows to optimise Integer and only use software types when the
values are large enough), and it also fixes the 32-bit problem. So
this patch changes the CStats types to Integer, except for the
instance count (which I don't expect to overflow 2²⁹ anytime soon).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

There were some implicit assertions in the code that all node groups
have nodes, which is not necessarily true.

Additionally, the patch does a wrapping change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Beside the 'dscription' typo, also make the punctuation more
consistent.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In order to make the display right on 80-columns terminals.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

New lintian is even smarter:

- overriden → overridden
- allows to → allows one to

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Just saw this while testing the migration to QuickCheck v2: while
configure.ac detects that QuickCheck-2.x is not available, the test in
Makefile.am was against WANT_HTOOLS (overall htools compilation), not
on a more-specific WANT_HTOOLSTESTS.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch fixes a bug in the test specification where we allowed nodes
with zero free memory (hence no instance can be added, at all) and adds
a simple labeling of the way this test can fail.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This unittest had a corner case where it could fail if the same
primary/secondary node names were generated.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This just adds glue to allow replaying of tests using a given RNG state
and test size (both are needed for exact replayability).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Using new functionality in QuickCheck 2 (the suchThat function), we
generate now better test cases, such that (heh) we have no longer
incomplete tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently the way we generate nodes in some cases is by creating a
totally random node, then restricting the test based on whether the node
'size' (as defined by multiples of base unit) satifies some high/low
rules. This results in hard-to-satisfy conditions, so we change this
model to be able to specify node sizes directly in the generation
process, thus no longer needed post-creation filters.

This fixes prop_ClusterAllocBalance which before had at most 1-2
satisfiable tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>