Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.6:
  Instance autorepair design

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This design describes a tool that will perform automatic repairs on
instances when they are detected to be unhealthy (living on offline or
drained nodes, at the moment). These repairs can be scheduled
automatically or requested as a one-off by a tool or person.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was missed; we add the definition and the de-serialisation
support.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is the initial support for Query2: basic infrastructure (except
filtering) and node query support (without RPC).

It implements all the fields (tests by comparison with list-fields on
the Python side), except that:

- filter is not done
- since RPC is not integrated yet, the runtime gathering/computing is
  simply stubbed out

However, the infrastructure seems pretty reasonable, so I'm sending as
is.

Note that I've split the functions/declarations into multiple files,
to keep each file clean and readable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This fixes an old TODO for moving these; now they can be reused in the
Haskell Query2 implementation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This adds the VTYPE_* conversion to VType, used for QFT equivalence
later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This enhances convert-constants to not flatten dicts completely, but
also generate a so-called association list for them.

This allows either direct use of the 'lookup' function, or (for
performance) conversion to Data.Map and optimised lookup later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This adds two related improvements to THH:

- for parameters, we declare a list with all their fields, so that
  Query2 can build the list of fields (e.g. for hvparams, or ndparams)
  automatically

- we declare a new type class for "DictObjects", i.e. objects which
  can be converted to a [(String, JSValue)] list of pairs; while this
  applies to all our objects, it will be used first for filled
  parameters, so that we can implement the lookup functions
  generically

Note that we re-export the new class from Objects.hs, so that other
modules don't have to import THH.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is a simple type declaration for NodeRole, a NdParamObject type
class, and a few related helper functions for nodes and node groups.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

I'd rather not introduce this typo in the Haskell code too, and also
not having them identical is not good.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

These mirror the TaggableObject in Python, in the sense that we will
be able to define generic functions for querying such fields.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch corrects the definitions in Qlang.hs to match what Python
expects on the wire; this means replacing some manual data type
definitions with 'buildObject' so that we get serialisation (and field
names) for free, adding (manually) JSON instances for types which are
not represented as objects in JSON, etc. Due to more TH usage, I had
to shift some definitions around, since TH breaks the "define in any
order" property (☹️

).

After that, we simply add a call into the stub Query/Query.hs module
which, for all queries, responds with "query not supported". The
reason for the deep directory structure is because I want to separate
the functionality into multiple submodules, for smaller module size
and readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, the query daemon would simply close the connection to the
client without issuing a response, if parsing the arguments failed;
the error was just logged.

Since this is very ugly from the client's point of view, we change it
so that the error is both logged and sent back to the
client. Furthermore, the execution block reporting is also cleaned up
a bit, to make the code more uniform.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Reporting things such as the following in our error messages
(indentation added by me, not originally present, so it's even worse):

  JSArray [JSArray [JSRational False (1 % 1),JSString
                    (JSONString {fromJSString = "a"})],
           JSArray [JSRational False (2 % 1),
                    JSString (JSONString {fromJSString = "b"})]]

Is not really nice. This patch adds more 'pp_value' conversions so
that the above becomes:

  [[1, "a"], [2, "b"]]

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch adds a simple function to ensure a socket doesn't exist
before binding to it. Additionally, we cleanup the exports of
Queryd.hs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since we have now access to all cluster parameters, we can "fill" the
parameters, and can finally enable it in CLI/RAPI.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch adds the missing parameters in the cluster/group objects,
for now as simple maps (dictionaries), without type safety. The
rationale for adding them as such is:

- we need something to enable query functionality
- since we don't modify the values, we don't risk introducing bugs
- we can improve the types later, once we find a good way to declare
  them

Also, I renamed a few of the parameters, dropping capitalisation of
acronyms (NIC → Nic, etc.).

At this point, I believe that the definitions are complete, with any
missing items being just oversight, and not intentionally removed (due
to lack of types, etc.).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is similar to the Python version, objects.py:FillDict.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

As opposed to all other places in the code, we imported this as
Data.Map, leaving to longer names. Let's make it similar to the rest
of the code (import qualified Data.Map as Map).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While looking at the hypervisor types in Constants.hs, I saw that the
'hyperTypes' list is using strings instead of names. This is due to
the fact that we require the entire elements in the list to be
identified (homogeneous lists), but the string "xen-pvm" is declared
by both 'defaultEnabledHypervisor' and 'htXenPvm'.

We can improve convert-constants by adding a list of names that we
know (statically) don't declare, but instead reuse values. Unless we
have an ADT parser, this is the best we can do, I think.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

These test that we encode requests correctly and decode good messages,
that a message with a too old/new timestamp is never accepted, and
that a message signed with a different key is never accepted.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

The Qlang module defines ResultStatus, but it was already defined in
Ganeti/Luxi.hs; let's remove the duplicate definition from there since
the proper place is in the newer module.

Also, in order to ease testing, we split some confd functions into a
separate module; this can be imported easily into QC for testing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is a very big hack for testing the equivalence of Python and
Haskell opcode definitions. See the docstring for details; I'm not
very happy with the solution but it does the job.

An alternate option would be to launch the Python code when
initialising the tests, pass (somehow) the resource through all of the
test suite to this function, and then use normal QuickCheck
properties. Or maybe we find a better solution later; for now, this
does the job.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While (again) trying to test Python/Haskell encoding interoperability,
I found another bug: the disk index is declared in Python as
ht.TPositiveInt, but in Haskell just as Int, so it can take negative
values too. Clearly we can do better, so let's add a wrapper type that
protects creation of invalid indices via a smart constructor
(http://www.haskell.org/haskellwiki/Smart_constructors

, the runtime
checking variant).

This means that outside of OpCodes.hs, it's not possible to load
invalid values, and this also applies to de-serialisation from JSON
data.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While testing Haskell⇔Python interoperability for opcode
serialisation, I found this bug: the Haskell code doesn't treat
optional fields with 'null' values as missing, which the Python code
does, leading to differences.

Investigating all uses of 'maybeFromObj' and the single use of
'fromObjWithDefault' shows that these are only used in cases where we
the rules are indeed "null == missing", so let's update the functions
and their docstrings accordingly.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

For now, we only test that we don't define extra opcodes in the
Haskell code; once we have parity, we can enable the (for now)
commented-out lines that check the reverse.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will be used in the future to make tests/assertions for
Python/Haskell codebase equivalence (e.g. checking that the same
opcodes are defined, etc.).

(Side-note: this might not work in VPATH builds; I'll see how it goes)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will allow us to use exactly the same method as for defining and
using QuickCheck properties. The differentiation is based on name,
which is not very nice but is the same method used in
test-framework-th, for example (so we will be able to switch
trivially).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

With the change to test-framework, where individual tests are listed,
QCHelper tries to remove a "prop_$group_" prefix from the function
names, for shorter names. However, some test groups (LUXI) and test
functions are not uniform, leading to long names, so we should fix
them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch replaces our home-grown, and quite limited, test runner
infrastructure with test-framework
(http://batterseapower.github.com/test-framework/

). The rationale for
doing so is as follows:

- we will need to add support for HUnit tests, so either we add more
  custom code or we switch to an existing library
- test-framework is mature and already packaged, at least in
  Debian/Ubuntu
- it supports more features: parallel test running, better test
  selection, etc.

As you can see, the changes are trivial, and don't touch the tests at
all; if/when we split the QC.hs file into per-module files, then we
could drop QCHelper too, and replace it with test-framework-th, which
does the same, but even more automated (auto-discovery, without having
to list the tests at all).

Dependencies are updated in devnotes.rst; note that I've already added
the hunit dependencies since we're going to use that soon.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since now we have the recvMsgExt, we can remove the handleEOF
function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We didn't close the server correctly, so for test sizes bigger than
the max-open-files ulimit, the tests were failing in accept().

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We were missing many docstrings in THH.hs, so let's add at least some
of them, and fix some unquoted '/'. Additionally one style change has
been done.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This can be used for cross-checking with the Python code for
consistency on defined opcodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

We'll need this in another place shortly, so let's abstract it and add
proper verification of whether we were passed a type name correctly;
the previous version would have failed with a pattern match failure,
instead of an explicit message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This patch adds the tags field to the objects which were missing it
(all except Cluster), implements handling the LuxiCall QueryTags, and
then enables the use of the query socket in cli.ListTags, used by all
commands, and in the RAPI client.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Currently, RAPI uses queries (as in QueryNode with field tags) for
getting the tags, whereas the CLI uses the QueryTags Luxi
command. Since this discrepancy is not good, let's switch RAPI to use
the same method as the CLI.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Like in the python code, this is a bit more complex since groups are
indexed by UUID, so we must fallback to lookup via the name (slow; but
the number of groups should be small).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This will allow us to use the simpler applicative form (<$>, <*>)
instead of monadic liftM, liftM2, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>