This adds supports for the (non-validated) cpu parameter to KVM.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fixes: http://code.google.com/p/ganeti/issues/detail?id=251


Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.6: (21 commits)
  Release Ganeti 2.6.0 rc4
  Prepare NEWS for Ganeti 2.6.0rc4
  Add some rudimentary node group ipolicy checks
  Fix setting ipolicy on node groups
  Fix --no-headers for the new list-drbd command
  Add a simple QA test for gnt-node list-drbd
  Add a new gnt-node command list-drbd
  Implement a node to drbd minors query function
  Add a new unused confd query
  Add support for computing instance all/secondary nodes
  Add disk logical ID support in Objects.hs
  Fix a docstring in bdev's DRBD8 class
  Extend the Template Haskell loadFn model
  Change how customFields are built
  Implement lookup of partial names in Config.hs
  Reorganise the lookup functions
  Remove an unused function
  Ensure that disk.params is always defined (and a dict)
  Another small consistency fix with if branches
  Fix inconsistency in the LUXI protocol w.r.t. args
  ...

Conflicts:
        htools/Ganeti/HTools/QC.hs (imports changes, trivial)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

By a strange accident, this file seems to be tabified, instead of our
own 2-space rule. Re-indent it and also break one long line.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Another release candidate is coming.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

On node groups we don't have the std field. However, the InstancePolicy
object always verifies that the std value is within a given range. As we
fill it up with defaults if not set (as it happens to be on node groups)
and the min value is higher than the default std value (taken from
constants.py) we fail.

We overcome this situation by simply let the function know if we want to
verify the std value at all. If we don't want to verify std, we just set
it to a compliant value (min_v) and continue.

We also slightly adapt the error message provided, as we don't have std
values on groups.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Setting a high prefix discourages the bridge from adopting the tap's
MAC. Xen is not affected by this since the MAC is forced to
"fe:ff:ff:ff:ff:ff".

This addresses issue #217.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sorry, I forgot that with GenerateTable this needs to be handled
manually. Fixed now and tested in both ways.

(But to be honest, this should be abstracted in GenerateTable, instead
of the 'if' test in all its callers.)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This will be run only if 'haskell-confd' QA configuration has been
enabled.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This uses confd to query the DRBD minors, which is very special; no
other command currently does so.

Since the backend is only implemented in the Haskell version of confd,
we have checks that 1) confd is enable, and 2) hs confd is also
enabled. If by mistake people do manage to query Python confd, the
error message will be clean:

  Query gave non-ok status '2': not implemented

So nothing breaks in an "ugly" way.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This can be queried remotely since it's a pure configuration query.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is not implemented currently.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This fixes an old FIXME. Since we now how actual DRBD configuration
data, we can finally compute the instance's secondary nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is a complex field, so we have to do a lot of manual work for now.

The complexity arises from the fact that the contents of the field,
and the way to parse it, depends on the disk type field, so we don't
have a single, static way of parsing it. Hence we needed the
extensions to the Template Haskell code.

Since we now can both load and save the disk type, we can remove the
in-memory (duplicate) disk type from the disk objects, relying only on
the logical ID to hold the type information.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

It seems this was not updated since the move to static minors…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, we only allow field-by-field de-serialisation. Since we
have cases where information about how to un-serialise a field is
split across two JSON fields (e.g. disk type and disk logical_id,
hypervisor and hvparams, etc.), we need to pass the entire object to
custom read functions.

Furthermore, since we will have to generate two actual fields from the
single in-memory field, we need to extend the custom save function so
that they can generate additional fields beyond the "main" field value
they currently generate.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Instead of passing an expression (which cannot come from the current
module), we pass a name, which is allowed to reference functions from
the module we're in. Since we currently don't have custom fields, we
don't need to modify any callers.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This uses the recently-moved functions to implement partial lookup of
names on getNode and getInstance, similar to the Python codebase.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, the LookupResult, MatchPriority and related functions are
locate in Loader.hs, since (so far) only hbal needs them in the
selection of instances. However, with the new functionality on confd
side, we need these functions there too, but we don't want to import
Loader.hs (which pulls in lots of balancing-related code). So we move
all these function to BasicTypes.hs, since that module is a leaf one,
with no other dependencies.

Unittests are slightly adjusted (but they are still tested under the
'Loader' group).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is not used, as we need a more complex serialisation, which is
done in the saveObjectField function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Commit cce46164 fixed upgrading from other 2.6 versions, but
accidentally broke upgrading from 2.5 (disk.params was left as None,
which breaks FillDict). The simplest way to handle params is to always
set them to an empty dict (disregarding what they currently contain).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

While looking at the testability of this piece of code, I found another
consistency problem. We have two if branches instead of one, with
elif's.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This inconsistency was found during rebalancing. Hbal failed because,
Ganeti couldn't load the opcode. After digging through the cause, an
inconsistency with the "args" field in the LUXI protocol was triggered
by the TemplateHaskell side where it's done uniformed.

For SubmitJob and SubmitManyJobs we treat args as one argument,
containing the job definition. In every other LUXI call args is actually
a list of arguments. This patch fixes this consistency.

This change is NOT backwards compatible.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This bug was found during disk parameter debugging. While looking at the
config some values seem present on the disk parameters, but that's not
expected behaviour. This patch fixes this, and also fixes the "broken"
configs automatically upon masterd restart.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.6: (72 commits)
  Make Ganeti 2.6.0 rc3 release
  Allow reinstall even when secondaries are offline
  Prepare NEWS file for Ganeti 2.6.0 rc3
  QA updated to test instance removing when sec. is offline
  Ignore offline node errors when removing disks
  Allow instance disc activation with offline secondaries
  RAPI regression beparams/memory fix
  Fix redistribution of files w.r.t. offline nodes
  Fix cluster verify error on master-ip-setup script
  Add a Makefile rule to test unclean git status
  Add test helpers to .gitignore
  Fix a typo in NEWS
  Release Ganeti 2.6.0rc2
  Prepare NEWS file for 2.6.0 rc2
  Update time on Haskell binaries after build
  hcheck: add two simple type aliases for readability
  hcheck: rework output mode
  hcheck: reword and fix typo in instance/offline msg
  Add hlint warning for wrong use of 'length'
  Remove can_rebalance functionality from hcheck
  ...

Conflicts:
        htools/Ganeti/HTools/QC.hs: due to the cleanup of missing type
        signatures, and the fact that in master two test functions had
        a changed argument list, trivial to fix

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Update NEWS regarding rc3 fixes and document changes in behaviour.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Added test checking that it is possible to remove an instance with drbd
disk when non-primary node is offline.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

When we delete DRBD disks from some instance, we do not want to get
errors due to nodes other than that instance's primary being offline.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently, this is not allowed, so one can't run a replace-disks; this
breaks any non-invasive method of recovering the redundancy of the
instance if its disks are already stopped (but it still works if the
disks on the primary are active). So let's fix this inconsistency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Since we're just talking about converting Maybe into another monad, we
can do that via the maybe function, instead of explicit casing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

There are two current issues with the coverage values:

- we don't import all modules, thus leading to incomplete
  coverage results (too optimistic);
- we use hpc in its default mode (intersection), which means that even
  modules which do have coverage results but are not used in all
  binaries we test will be dropped from the results; thanks to Agata,
  passing --union to hpc is enough to have better results (don't
  remember why this wasn't there in the first place…)

After adding more modules to the import list and fixing the combining
mode, we now have a complete list of modules in coverage results, many
with zero coverage, so our overall coverage has dropped to about 60%.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Note however that this won't work correctly with older compilers, due
to ghc issue #4256; however, it's a handy way to build complete a TAGS
file for Emacs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, we build the Haskell unittests with custom GHC flags,
because we (I) were quite lazy when initially writing the
unittests. This is not a problem for the tests themselves, but it
creates problem when (for example) one would want to pass all
$(HS_LIB_SRCS) to the compilers; this is not doable unless we
"degrade" the flags used for all modules, instead of just for QC.hs.

So we do two things to fix this:

- first, we go and add type declaration to all functions that were
  missing them (in QC), and fix the couple of cases of monomorphism
  restrictions; this gets us rid of -fno-warn-missing-signature and
  -fno-warn-monomorphism-restriction
- then, we move the actually important remaining options
  (-fno-warn-orphans and -fno-warn-unused-imports; see the explanation
  for the latter in the newly added FIXME) to a compiler pragma in the
  file, so that when building the unittests only this file is using
  the extra options

And finally we can then drop the other unused options
(-fno-warn-missing-methods anmd -Wwarn), leaving htools/test use
simply -fhpc. This is more in-line with the other files, and thus we
can handle all of HS_LIB_SRCS the same.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

For compatibility with the old Ganeti version, we want to keep the
beparams/memory field around for another release. This patch fixes this
regression.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch aims at improving documentation for installing ganeti - both for users and developers.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since this is not yet used by any targets, we didn't detect yet the
compilation warnings. Just trivial exports/imports cleanup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>