While looking at the hypervisor types in Constants.hs, I saw that the
'hyperTypes' list is using strings instead of names. This is due to
the fact that we require the entire elements in the list to be
identified (homogeneous lists), but the string "xen-pvm" is declared
by both 'defaultEnabledHypervisor' and 'htXenPvm'.

We can improve convert-constants by adding a list of names that we
know (statically) don't declare, but instead reuse values. Unless we
have an ADT parser, this is the best we can do, I think.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

These test that we encode requests correctly and decode good messages,
that a message with a too old/new timestamp is never accepted, and
that a message signed with a different key is never accepted.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

The Qlang module defines ResultStatus, but it was already defined in
Ganeti/Luxi.hs; let's remove the duplicate definition from there since
the proper place is in the newer module.

Also, in order to ease testing, we split some confd functions into a
separate module; this can be imported easily into QC for testing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is a very big hack for testing the equivalence of Python and
Haskell opcode definitions. See the docstring for details; I'm not
very happy with the solution but it does the job.

An alternate option would be to launch the Python code when
initialising the tests, pass (somehow) the resource through all of the
test suite to this function, and then use normal QuickCheck
properties. Or maybe we find a better solution later; for now, this
does the job.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While (again) trying to test Python/Haskell encoding interoperability,
I found another bug: the disk index is declared in Python as
ht.TPositiveInt, but in Haskell just as Int, so it can take negative
values too. Clearly we can do better, so let's add a wrapper type that
protects creation of invalid indices via a smart constructor
(http://www.haskell.org/haskellwiki/Smart_constructors

, the runtime
checking variant).

This means that outside of OpCodes.hs, it's not possible to load
invalid values, and this also applies to de-serialisation from JSON
data.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While testing Haskell⇔Python interoperability for opcode
serialisation, I found this bug: the Haskell code doesn't treat
optional fields with 'null' values as missing, which the Python code
does, leading to differences.

Investigating all uses of 'maybeFromObj' and the single use of
'fromObjWithDefault' shows that these are only used in cases where we
the rules are indeed "null == missing", so let's update the functions
and their docstrings accordingly.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

For now, we only test that we don't define extra opcodes in the
Haskell code; once we have parity, we can enable the (for now)
commented-out lines that check the reverse.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will be used in the future to make tests/assertions for
Python/Haskell codebase equivalence (e.g. checking that the same
opcodes are defined, etc.).

(Side-note: this might not work in VPATH builds; I'll see how it goes)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will allow us to use exactly the same method as for defining and
using QuickCheck properties. The differentiation is based on name,
which is not very nice but is the same method used in
test-framework-th, for example (so we will be able to switch
trivially).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

With the change to test-framework, where individual tests are listed,
QCHelper tries to remove a "prop_$group_" prefix from the function
names, for shorter names. However, some test groups (LUXI) and test
functions are not uniform, leading to long names, so we should fix
them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch replaces our home-grown, and quite limited, test runner
infrastructure with test-framework
(http://batterseapower.github.com/test-framework/

). The rationale for
doing so is as follows:

- we will need to add support for HUnit tests, so either we add more
  custom code or we switch to an existing library
- test-framework is mature and already packaged, at least in
  Debian/Ubuntu
- it supports more features: parallel test running, better test
  selection, etc.

As you can see, the changes are trivial, and don't touch the tests at
all; if/when we split the QC.hs file into per-module files, then we
could drop QCHelper too, and replace it with test-framework-th, which
does the same, but even more automated (auto-discovery, without having
to list the tests at all).

Dependencies are updated in devnotes.rst; note that I've already added
the hunit dependencies since we're going to use that soon.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since now we have the recvMsgExt, we can remove the handleEOF
function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We didn't close the server correctly, so for test sizes bigger than
the max-open-files ulimit, the tests were failing in accept().

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We were missing many docstrings in THH.hs, so let's add at least some
of them, and fix some unquoted '/'. Additionally one style change has
been done.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This can be used for cross-checking with the Python code for
consistency on defined opcodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We'll need this in another place shortly, so let's abstract it and add
proper verification of whether we were passed a type name correctly;
the previous version would have failed with a pattern match failure,
instead of an explicit message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch adds the tags field to the objects which were missing it
(all except Cluster), implements handling the LuxiCall QueryTags, and
then enables the use of the query socket in cli.ListTags, used by all
commands, and in the RAPI client.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, RAPI uses queries (as in QueryNode with field tags) for
getting the tags, whereas the CLI uses the QueryTags Luxi
command. Since this discrepancy is not good, let's switch RAPI to use
the same method as the CLI.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Like in the python code, this is a bit more complex since groups are
indexed by UUID, so we must fallback to lookup via the name (slow; but
the number of groups should be small).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will allow us to use the simpler applicative form (<$>, <*>)
instead of monadic liftM, liftM2, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will allow safer code when we implement the tags query.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

The actual query definitions are now in Qlang.hs, so let's use the
ItemType from there instead of luxi-defined type (which is also
incomplete).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This implements the same query=True|False functionality as in
GetClient for cli.py, however since the RAPI code is much more
unit-tested (and the unit-test clients are mocked, for the most part,
without support for addresses) we have to do many adaptations in the
tests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

And expand cli.GetClient() to allow opening the query socket, instead
of the main master socket.

Finally, enable the query socket use in gnt-cluster version, since
that is already implemented fully in Queryd.hs/hconfd.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Due to the (now removed) custom filter field, we needed a conversion
function. Since now that field is gone, we can move to a simpler Luxi
TH implementation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While starting to use the new filter types, I realised that what is
currently implemented is the equivalent of `lib/qlang.py', not
`lib/query.py', since we only deal with data types for now and not the
actual query runtime functionality (RPC, config, etc.).

Let's rename the file to be more consistent with the Python code base.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Until now, since we didn't have a proper type for the encoded query
filters, we were ignoring the filters and handled them as null values
(JSNull and respectively ()). With the current Query2 implementation,
we can actually use correct filters (we can send filters the client
side, and on server side we can decode them).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This adds support for encoding/decoding Query2 filters to/from JSON,
in (hopefully) the same format as the Python code generates.

It also adds a simple unit-test to check that this conversion is
idempotent. Of note here is that, since the Filter data type is
recursive, we have to manually ensure that the generator for it
correctly "shrinks" at each step (first version crashed hard my
workstation after eating ~8GB of ram :).

Compared to the current Query2 implementation, the following changes
were done:

- style: shortened some names to match the Python ones (LessEqualThan
  → LE, etc.)
- changed FilterValue from string to an ADT that can encode both
  quoted strings and numeric values, since this is actually what
  qlang.py generates
- added support for EmptyField, which in hindsight it's an obvious
  missing part :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since these will be needed for Haskell's implementation of query2.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Commit 5a1e31b4 (Add infrastructure for, and two extra hlint rules)
was intended to add two *extra* hlint rules, but I didn't realise at
that time that "--hint" when first used overrides the built-in
lints. As such, since then we were basically running with just those
two rules, which resulted in many uncaught warnings/errors.

This patch fixes that (by importing the standard lint rules in our
custom hints file), and then goes to fix all the warnings that a
current hlint gives me. Compared to our current style, we have just a
few additions:

- zipWithM instead of map foo . zip …
- 'exitSuccess' instead of 'exitWith ExitSuccess'
- more uses of '.'

Additionally, we have to silence a case where hlint doesn't realise
why we are using '\e -> const (return False (e :: IOError)' instead of
just '\e -> return False' or even 'const (return False').

One warning that is generated by hlint ("Use void") can't be fixed
until we deprecate GHC 6.x, as only GHC 7 has the 'void' function in
Control.Monad.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This enables the query functionality in confd.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is just a new module that exports a runQueryD function, that can
be imported to run a separate thread handling the luxi requests.

Currently it needs access just to the configuration, in the future it
will need access to an RPC runner too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This makes the implementation a bit nicer for both for server and
client side: we add a wrapper function with a better result type, and
a few extra functions for building the response.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Note that since we don't have yet a way to nicely handle two-level
optional parameters, the Filled/Partial types and filling function are
all manually built.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This adds a few missing/incomplete definitions. We're still missing
the special parameters (disk params, hvparams, os_hvp).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since we now handle Containers uniformly, we can remove all traces of
the special handling for this field type.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is the first part of the changes related to the 'Container' type.

We currently handle this type as follows: it's a simple type alias
over the Data.Map type, which means:

- it's easy to use the Data.Map functions to change the type
- however, since Data.Map already has a JSON instance, we have to very
  carefully always use custom show/read routines to handle this type

The second point leads to potential bugs which are not caught by the
type system, so let's improve the situation by making it a proper
newtype, which can have its own JSON instance (with our desired
behaviour). Once we do this change, accessing the type requires an
extra function call, but it's as safe as before. On the positive side,
we can use the implicit read/show JSON, which means we can remove (in
the next patch) the "container" special casing.

The patch also moves the type to outside of THH, since not all users
of this will want to import that (as opposed to JSON.hs, which is
smaller).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch addresses two issues with our TH code:

- using non-unique names (e.g. "std" for a local name, instead of
  "std_XXXX" random names), which can leads to conflicts; on the other
  hand, this makes the generated code a bit harder to parse
- since only a few Python/JSON names have dashes in them, we didn't
  handle those, resulting in variables named like "disk-templates",
  which is not good; we now handle it the same as '_', i.e. we use it
  as a breaker for camel-casing

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Also add this new field and the other generic fields to the cluster
object.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is not used yet, but is helpful when developing the Luxi endpoint
for hconfd.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>