These are tested for equivalence with the Python code using existing
tests. These tests are not perfect - for many "TDict" we simply send
empty dicts, as we don't have a way to generate proper data, but the
tests did catch many trivial errors during writing of this patch
series.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This does a partial conversion of existing opcodes to the new
non-empty string type, which is needed so that we don't add even more
opcodes without this safety net (and to have consistent "target_node",
etc. in opcodes).

By partial I mean that only the opcode are changed, and not the entire
htools types/workflow, so we use error to abort if, when submitting
the opcodes, we detect an empty node name. FIXME added.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a bulk add of all node parameters needed for node and cluster
opcodes. The parameters are defined with a few helper functions in
this module, and, opposite from opcodes.py, there won't be any (new)
opcode attributes declared in-line (without a separate field
definition).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These are needed for completing the node opcode parameters.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This fixes a FIXME actually recommending this move :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The QR_VIA_OP/QR_VIA_LUXI types in Python are using yet another
validation mode: QR_VIA_OP is the base type, and QR_VIA_LUXI extends
it (when doing luxi queries). But on the wire they have the same
representation.

To accommodate this properly, we split the ItemType in two: a
QueryTypeOp and a QueryTypeLuxi, joining them back together in
ItemType itself. This requires custom serialisation/deserialisation,
but allows us to express correctly that at opcode level, we only
accept a QueryTypeOp, but at Luxi query level, we accept either of
them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a very basic type and 'Objects' is a heavy-weight module. By
moving it to 'types' we simplify (in the future) the import chains.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, we have some types that we kept as dictionaries in the
objects representation (due to inconsistent behaviour, or other
technical reasons). This should be improved in the future, but in the
meantime we can improve the Container type by allowing its keys to be
non-strings; this is needed, for example, for better safe in
DiskParams type, where we don't want arbitrary strings as keys, but
only the actually defined types.

To implement this change, we generalise the type (Container →
GenericContainer), and in the process we introduce a type class for
"things that can be represented as strings". This is very similar to a
combination of Read and Show, but with custom string
representation. The new type class is needed because in JSON
representation, object keys must be strings, so we need to be able to
serialised/deserialise the generic keys to/from plain strings.

We also add the instance for DiskTemplate at the same time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

These are needed both in opcodes and (potentially) outside, so we add
them in the main Types module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This mirrors the ht.py types PositiveInt, NonNegative, etc., except
that they work at a more generic level (any numeric type, respectively
any non-empty list).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

A new directory for haskell modules about block devices has been created
The parser is divided in two modules:
* one exports the data types describing the DRBD status
* one exports the parser itself

Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin@google.com: indentation/alignment fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

While writing the pipe-based reporting and trying various ways to
break the startup, I fought for a while trying to understand why error
reporting was _different_ when running the daemon as a user (with no
rights). It turns out that setupDaemonFDs wants to open the log file
in append mode way before, so we are not protected by the 'prepare'
phase.

This patch explicitly runs the 'setupDaemonFDs' function under the
same handler as the prepare phase, with the only change that here we
instruct handlePrepErr to not log the message via log*, since logging
is not yet set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the OpCode definitions are using normal constructors:

  data OpCode = OpTestDelay Double Bool [String]
              | OpInstanceFailover String Bool (Maybe String)
              …

While this works for a few opcodes, it becomes unwieldy when dealing
with a bigger number of opcode definitions and/or with opcodes having
many fields.

This patch changes the opcodes to record-based constructors, so that
we get for free accessor functions:

  data OpCode
    = OpTestDelay {
        opDuration :: Double,
        opOnMaster :: Bool,
        opOnNodes :: [String]
      }
      | OpInstanceFailover {
         opInstanceName :: String,
         opIgnoreConsistency :: Bool,
         opTargetNode :: Maybe String
      }
      …

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

While looking at the opcode docs and clicking accidentally on the
filter type, I saw that the haddock formatting is broken due to
non-escaped use of special chars. Let's convert the ascii-like
formatting to haddock, and have nicer apidoc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Due to TemplateHaskell stage restrictions, we can't define parameters
in the same module as we're using them for TH, so we have to define
all module parameters in a separate module.

This patch therefore splits OpCodes.hs in two, adding that module and
moves most code there (types, parameters, etc.). The remaining parts
in OpCodes.hs, the actual opcode definitions, now use more parameters
instead of direct field definitions (more will come later)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

There are already three cases where we copied type definitions between
the htools-specific types into the main ganeti code. Let's stop doing
this ☺️

 and create a common types module that holds these.

Note that there already exists BasicTypes.hs, but that refers to very
low-level types, and can't use TH derivation itself.

A side effect of this unification is that there is a small conflict
between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
in the InstanceStatus type.

The patch also moves the tests related to these types to a new test
module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, THH.hs "injects" into the built code names of library
functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
directly from strings, via (e.g.)

  varE (mkName "makeObj")

This means that the "makeObj" name must exist in the target module,
i.o.w. must be imported there. This leads to the strange case of
having to have imports that do not appear at all in the used
(template) code, but are needed to satisfy this "hidden" dependency;
look at Ganeti/Jobs.hs before this patch, for example.

This is also not very obvious, because we usually import Text.JSON
anyway; I only stumbled upon it while doing some cleanup work.

So to clean this up, the current patch changes the THH.hs to use not
string-derived, but identifier-derived names («'identifier» versus
«mkName "identifier"»); this is better, as the names must be
resolvable when compiling THH itself (once), and not when compiling
the multiple derived modules. As you can see, this allows removal of
extraneous imports from various modules.

Background information: an `mkName "foo"` results in a name of flavour
NameS (“An unqualified name; dynamically bound”) or alternatively to a
qualified name, but still dynamically bound. Whereas what we want is a
statically bound name: `'foo` results in a NameG flavour, “Global name
bound outside of the TH AST: An original name”.

One more explanation: the change is similar to going from 'x = eval
"map"' to 'x = map'; the name is no longer dynamically evaluated, but
statically when the module is compiled. In our case, previously names
were bound at target module compile time, now they are bound at THH.hs
compile time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

A new LUXI request is added, in both Python and Haskell.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sorry, I broke lint again :), by introducing two sub-standard changes.

Additionally, this silences an older existing warning that only
triggers with some versions of hlint (e.g. 1.8.28 which is present in
Wheezy).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, the code uses createFile, which has the effect of always
truncating the file. This is bad, as the content of the PID file is
wiped even when we wouldn't be able to lock it!

We switch to openFd (createFile is just a wrapper over that), and we
use an explicit set of flags; defaultFileFlags is already safe
(trunc=False), but I prefer to set it explicitly with our desired
flags.

Note that this bug doesn't manifest in normal usage, as daemon-util
won't try to start the daemon if already running. But if anyone or
anything does call ganeti-confd explicitly, the pid file will be
emptied and the daemon will keep trying to be restarted forever…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

We need to change a few things, most importantly CLI options defaults,
but otherwise we already used the path to files from functions which
were already in the I/O monad, so we don't have to change much of the
code flow.

Additionally, Path.hs now has an explicit export list, to stop
accidental leakage of symbols.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Currently, keyToFilename uses itself the default config path. In the
light of making that an function in the IO monad, let's remove the
"default" path functionality from this function and make it always
require the config path; its caller, readSSConfFile, can then do the
maybe/default path change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Some options have defaults that depend on the environment, and we
could handle these in two ways:

- use a place-holder value (e.g. data X a = Default | Custom a) that
  is later read from the environment
- move the options list to IO monad, where it can read the
  environment, etc.

The second option allows also displaying the actual defaults in the
`--help' output, even though it's not as nice, so I went with it.

This patch only changes the option types, without actually changing
any options yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The "starting" message is changed to match the Python one, and in case
the preparation fails, we also log the error (beside printing it on
stderr or writing it to the error reporting pipe), as at this time
logging is usually set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The luxi binding is now annotated for better readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the writing of the PID file uses a "standard" error
reporting: catch exception, transform it into a 'Result' type, leave
handling of that to the caller. However, for daemon startup, we
actually want exceptions to be propagated up until the handler which
will write the details to the pipe.

This patch removes the writePidFile wrapper (and stops exporting it),
and changes the code to simply annotate better the I/O error.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

It turns out that annotateIOError already exists in the standard
library, with a different purpose (this made me waste 10 minutes
trying to understand why the types were wrong…). Let's move this out
of BasicTypes, since it's used only in Daemon.hs, and also rename it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This changes the daemon reporting error to the same mechanism as in
Python: use a pipe which is written to from the forked children.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This does the address parsing earlier, before we fork, for
better/faster error reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will help with the general daemon split of prepare/run, and flag
errors earlier in the startup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, we keep information about the "target" of a tag operation
in a data type similar to (TagKind, Maybe String). This is unsafe, as
nothing (at the type level) prevents us from accidentally having
(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing).

To fix this problem, we rename the current TagObject type to TagType
(an internal utility type), and create TagObject as a better/safer
data type (see the definition), which doesn't allow such possibilities
in the future.

The downside is that, since at encoding level (both opcode and luxi)
this is done in an ugly way (type elements spread at the same level as
level as other value), we have to add custom encoders/decoders. The
encoder is shared between the OpCode and Luxi usage, the decoder is
different however as Luxi uses custom decoding.

This also fixes the recent breakage in confd w.r.t. QueryTags.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is needed so that we have more flexibility in generating Luxi
serialisation code (deserialisation is still custom). Also, only
exceptions are now using the 'simple' field types, so we might be able
later to convert and remove that TH code as well.

Since we will use custom serialisation fields in the future, we change
the order of serialisation for custom-save fields; Luxi uses
positional as opposed to name-based ordering, so we need to keep this
stable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This can be improved, by taking all hardcoded names as parameters, to
serve as a more-general "build save clause for a multi-constructor
data type". I'm not renaming the function as well, since I don't know
exactly how much we can abstract later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This should be the last module rename, promise!

We rename this to conform to the other hierarchies (e.g. Query), and
to not have both Confd.hs and Confd/*.hs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also, adjust comment to $(genOpCode) block to avoid repetition of
"only".

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is also
required for the "kind" attribute of OpTagsSet and OpTagsDel, which
are coming to OpCodes.hs next. Hence, we move TagObject there, and
adjust imports accordingly.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This mirrors the code in the Python code base, and is required for
clean error reporting during startup.

This patch implements the basic infrastructure; the confd daemon is
not yet modified to take advantage of this, just the types are
adjusted.

Also, the pipe-based error reporting will go in a future patch, once
we can actually use that for reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is only in master, so needed to be fixed separately.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This small patch fixes compatibility with a few newer Haskell libraries:

- base 4.6, included with ghc 7.6, removed the deprecated 'catch'
  function from Prelude, so our "import Prelude hiding (catch)" is now
  an error; we workaround by using fully-qualified
  Control.Exception.catch name

- containers 0.5 changed the signature of 'deleteFindMax'; we
  workaround by using separate 'findMax' and 'deleteMax'

- QuickCheck 2.5 removed the 'maxDiscards' test parameter, replacing
  it with a much better 'maxDiscardsRatio'; however, until we can
  depend on that, we workaround by just removing it (we don't control
  anymore the maxDiscards, instead leaving it default; for our default
  test size, this is no change, as the default value is already 500,
  which is our default as well) and not printing it anymore

Tested on Squeeze (+extra libs), Wheezy and experimental, which covers
all supported GHC versions.

Also, merging this in master will be a pain, but unless we want to
stop supporting 2.6…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This tests that the same Luxi calls are defined in Python and
Haskell. It doesn't test yet that their serialisation is correct
though.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>