Since managing of different storage units is now done using
disk templates and not storage types, we remove the obsolete
enabled storage types.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtarara@google.com>

Adds setting of the list of enabled disk templates to 'gnt-cluster modify'.
Note that this does not yet include any verification regarding disk templates
currently used by instances.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtarara@google.com>

This parameter will be used to set the new primary node
of an instance, assuming that the disks have been moved
by other means (outside ganeti's control).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The test is aimed mostly at instance policies, which changed recently.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Minimum and maximum instance specs are put together into a single element
of the instance policy. This is in preparation for introducing multiple
min/max specs.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

It's now possible to go back to the previous stable version. Unit tests
provided.

This is mostly useful during development, when going from master to
stable/devel.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Useful for subsequent patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The function produces a human-readable output, which is also a valid YAML
file, from an intermediate data structure. This will be used by "gnt-xxx
info" commands. The output of "gnt-xxx info" was almost YAML-compliant, and
only minor adjustments are needed to make it fully compliant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The "queryargs" parameter of the __init__ function of rlib2 classes should
be a dictionary, as defined in the ResourceBase parent class (in
lib/rapi/baserlib.py).

In the rlib2 and baserlib unit tests, when the queryargs parameter is not
actually needed, it is sometimes erroneously initialized with an empty list or a
"None" value instead of an empty dictionary.

This commit solves the problem and introduces an assertion to prevent it from
happening again in the future.
The use of the assertion is safe, because in production code the __init__
function is only called by the RAPI server, with the queryargs parameter
initialized by the Mapper.getController function (lib/rapi/connector.py) that
always returns a dictionary. It can only affect test code and future code.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch extends the 'gnt-cluster modify' command to manipulate the list
of enabled storage types. Note that this currenlty does no validation
with respect to whether or not there are instances currently using a storage
type that is being removed from the list.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This commit adds the core infrastructure of the monitoring daemon,
and integrates it in the build and test systems.

The actual functionality of the monitoring daemon is still completely
missing.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also, add its logfiles and extra log files.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some daemons will need more than the single logfile that is currently
allowed. This patch introduces the infrastructure to allow this.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds some very simple IPv4 address/network types, and uses
them in the 'Network' config object.

We need these in order to properly compute the reserved IP addresses,
without depending on an external library (which I haven't found, by
the way). Currently the only operation supported is 'get next IP
address', which is enough for us.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This is in preparation for using OverloadedString extensions in this
file, which conflicts with hlint 1.8.28's handling of annotations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

HLint 1.8.28 requires us to always add "ignore CamelCase", which can
be problematic sometimes (e.g. when using OverloadedStrings). Let's
expand TestHelper to also support less-standard 'caseFooBar' names, so
that we can remove the annotations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Currently, both the Python and Haskell code return the internal
instance list unsorted, which means the output can vary depending on
the phase of the moon (well, the Haskell code actually uses internally
a tree, sorted by the instance name, but it's implementation detail).

By adding an explicit sort step, we guarantee both stability of the
output and consistency between the two query paths.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Tests for:
  objects.InstancePolicy.CheckParameterSyntax()
  objects.InstancePolicy.CheckDiskTemplates()
  objects.InstancePolicy.CheckISpecSyntax()

Instance policies with an empty disk-template list now are reported.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

IPOLICY_DEFAULTS is now a legal policy (the disk-templates entry was a set
instead of a list, before).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unknown elements were silently removed on startup. This means that a
software upgrade could result in lost configuration information if
cfgupgrade wasn't run promptly.

Added unit test for Cluster.UpgradeConfig() to cover this case.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Policy violations of back-end parameters that used the cluster default
value were not reported in cluster-verify.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Not 100% coverage, though.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instance disk template is checked against the policy, and diskless
instances aren't checked for the number of disks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch does multiple enhancements to the way we build the Haddock
docs, motivated by the fact that if we don't run Haddock over test
files as well, bad formatting can be submitted and will accumulate
over time (without any checks).

The patch does:

- replace manual built ALL_APIDOC_HS_DIRS with automatically built one
  (from HS_DIRS)
- change Haddock so that it runs from the top directory (instead of
  from src/)
- change HsColour target file to be built via bash parameter
  substitution, rather than sed (I don't know how to do it in one go,
  so I use 2 intermediate variables)
- change 'hs-apidoc' target so that it depends on the real target
  file; in case no source file has been modified, running 'make
  hs-apidoc' twice will not result in two runs
- run HsColour/Haddock under en_US.UTF-8 locale, otherwise they can't
  parse correctly the Unicode chars in the test files

Additionally, wrong formatting (oops) in a test file has been
corrected.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This further improves the comparison for "non-trivial"
numbers. Without this patch, there are still cases where the absolute
error is too big, and we need to switch to relative error.

Concept has been taken from
<http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

>.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the test uses a frequency of 5 string/5 double/1 list for
generating Arbitrary instances of ListConfig. However, the list case
has simply a "choose (1, 20)" `vectorOf` arbitrary, which means it
could recurse forever.

Manually running only this test gives runtime as such:

- ~100-200ms: very often
- ~1-2s: often
- ~5s: rare
- ~20s: very rare (but I hit this when running < 30 times the test,
  so…)

On average, this makes this test one of the slowest ones, which is
annoying.

By changing to a sized generator, we can control the depth of the
recursion, ensuring that we have a consistent runtime: out of 100
runs, one is 229ms, one is 164ms, the other are 80-120ms.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this tests and its helper function 'isAlmostEqual' uses
plain booleans to signify failures, which means you can't really debug
a failed test. The patch changes the call chain to use annotated
properties all through, which results in messages like:

  Failing almost equal check
  Delta 3.725290298461914e-9 not smaller than 1e-12
  expected: 2.147785408767952e7
   but got: 2.1477854087679517e7

which (IMHO) it's much more readable.

The patch also replaces a 'fail' with 'failTest' in another test.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This implements QffHostname and fixes the node listing (as well as
export listing when filtering on node name).

This bug was hidden by the fact that node listing with "gnt-node list
aa" works if you don't have live queries (as it was originally), as
the choosing of wanted nodes out of the config based on short names
works. What didn't work was later post-filtering based on such short
names (kind of duplicate, but that's how the code path is).

By implementing QffHostname, we can have custom equality checks, like
in the Python code. What I don't like is how convoluted the testing on
various left/right combinations is, but I didn't find an easier way.

The included unittest tests the partial filtering behaviour, and fails
if the node name flag is not set to QffHostname.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since we use the primitive string type for group UUIDs, the group
fields have a bug where we pass the group name as filter for node
tests, whereas the nodes themselves use the group UUID. This results
in zero node count, empty node list, and no instances being reported
as assigned to groups.

The patch fixes this and adds a test for the node count. It does some
test generation improvement, and also cleans up whitespace issues in
Test/G/Q/Query.hs (the functions case_queryNode_allfields,
prop_queryGroup_noUnknown and case_queryGroup_allfields are unchanged
but simply have indentation fixed).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Following the new naming style introduced in Exports.hs, this patch
renames the other resources to export non-qualified names (fieldMap as
opposed to nodeFieldMap), and to use qualified module imports.

Also fixes a haddock issue in a docstring.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Two test helper files didn't have export lists (lost during the split
of tests). This patch finally reintroduces them, to hopefully make the
export list saner and with fewer changes for purely-internal changes.

Additionally, a few missing docstrings are added as well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

When file storage is disabled at ./configure time, we shouldn't pass
opcodes containing DTFile/DTSharedFile/StorageFile to Python for
validation, as they will fail.

This patch implements this by simply tweaking the Arbitrary instances
for DiskTemplate and StorageType (which IMHO is a nice and clean
way!), and also fixing the generation of arbitrary IPolicies to use
the correct 'allDiskTemplates' list (otherwise we'd loop forever
trying to generate a list of all templates).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

'MakeLegacyNodeInfo' is not the best place for this, but we'd have to
duplicate it if we wanted a LVM-less version, so the easiest is to add
an optional parameter that allows it to accept/skip LVM-less results.

It still requires at most one VG result, so its behaviour isn't
changed in this respect.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was meant to be since "the beginning" but was never submitted as
somehow it failed with python 2.4. Now that the minimum python version
has been increased it can be added.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Until now all tests depended on simple AND/OR conditions. With the
introduction of virtual cluster support, some tests can only run if
virtual clusters are not in use (e.g. not yet supported or not
possible). Instead of introducing more logic for enabling/disabling
tests, callable values are supported instead. “ht.WithDesc” should be
used on them to continue to provide a readable description.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If these new options are set, a virtual cluster is created and used.
Virtual clusters consist of 1..N virtual nodes on a single physical (or
virtual) node. The design of virtual clusters is described in
“doc/design-virtual-clusters.rst”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

- Commit 4a90bd4f contained a rather large number of style violations:
  - Indentation/sequence formatting
  - Wrapping of long lines
  - Docstrings whose first line was wrapped
  - A stray backslash in a docstring
- opcodes: Don't check for list or tuple. None of the other opcodes does
  it explicitely. As long as the length and the items match the value is
  accepted.
- server/noded: “if variable” doesn't test for None, but False

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Most RAPI resources do not require authentication for the “GET” method.
In some setups it can be desirable to always require authentication.
This patch adds a command line parameter to always require it.

Some unrelated minor typos in the “ganeti-rapi” man page are also fixed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This will remove the tests for the getNetworkUuid method.
The method will be obsolete after the changes regarding
identification of networks by UUID and not name are merged
into master. Will send another patch for removing the
actual method.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These two functions permit operating in bulk on only the Left or Right
values in the original list, then reassembling the list back in the
original order.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>