GitLab

We detect an IPv6 vs V4 address based on columns, rather than passing
the family from the cluster object to be more future proof (in case
we'll ever support mixed clusters).

Unfortunately quite a bit more code is required to test this: we need an
arbitrary node that does the right thing w.r.t. ip addresses and also
test-only exports. As such we'll do this out of the stable branch.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

executeRpcCall is the function to be used for executing RPCs, so it makes sense
to use it as the single point for logging all thte RPC call errors.

Fixes Issue 293.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This function can be useful to multiple RPC calls, therefore it is moved
to the file containing the common RPC functions.

Also, it is made more generic by changing its signature.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This is straightforward, as the call has no parameters and a very simple return type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, an empty objects will generate warnings as the arguments of
various functions are unused. By adding conditional code for this, we
can support generation of empty objects, e.g. like needed in Rpc code.

Additionally, the patch also converts RpcCallVersion to THH, now that
it can build it. We change the serialisation for this (from JSNull to
JSObject []), but this shouldn't matter as this is not used in
production.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This replaces the very-basic parMap of IO actions (fully serialised,
as parMap won't work here), to the multi interface.

This makes a simple "time gnt-node list" on a 6-node cluster go from
3.2s to ~0.9s, and allows even better parallelisation - before,
curlGetString was blocking, whereas the new interface does allow some
interleaving.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch removes the node from the RPC error constructurs
CurlLayerError and OfflineNodeError. The rationale is that we anyway
return tuples (node, result), and removing this duplication allows
simplified signatures/calls in the execution of RPC calls.

Note: this was sent before as well, resending on top of current codebase.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Some curl option are request-specific, but not node specific: e.g. rpc
timeout, etc. The patch changes the HttpClientRequest type so that we
can pre-seed such options, instead of rebuilding the list in each
individual request execution.

Note: this was sent before and LGTMed, but on a different codebase, so
resending an updated version.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Currently, we support curl being optional via some sporting exercises:
ifdefs in the code, data types that represent 'Curl is disabled'
state, etc. However, with the future work on RPC, we would have to
even make the dependencies list conditional on it, etc. This is too
much work, when the curl library is included even in stable
distributions.

This patch changes curl from option to required, the same as the other
base libraries.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The flag is read from the configuration and passed to the RPC.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The flag is passed to the low-level functions that need it, but it's not
yet used. Also, RPCs get just a default value for now.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It turns out that optimising 'read' derived instances (via -O) for
complex data types (like OpCode, or the various objects) can be slow
to very slow. Disabling such instances results in (time make
$all_our_haskell_binaries) large compile-time savings and also smaller
(unstripped) binaries (by a significant amount):

ghc 6.12:        time  htools sz  hconfd sz
  with read:    4m50s 12,244,694 14,927,928
  no read:      3m30s 10,234,305 12,536,745
ghc 7.6:
  with read:   14m45s 13,694,761 15,741,755
  no read:      3m40s 11,631,373 13,245,134

So let's remove these instances, since we never use read in production
for our custom types, and even when debugging in GHCI, we can simply
use the 'show' representation to create the types, without needing to
actually parse from strings.

Note: for the very slow ghc 7.6 compilation time, I filled a ticket
(ghc #7450), since it is surprising(ly slow).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This fixes a FIXME actually recommending this move :)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a very basic type and 'Objects' is a heavy-weight module. By
moving it to 'types' we simplify (in the future) the import chains.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, THH.hs "injects" into the built code names of library
functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
directly from strings, via (e.g.)

  varE (mkName "makeObj")

This means that the "makeObj" name must exist in the target module,
i.o.w. must be imported there. This leads to the strange case of
having to have imports that do not appear at all in the used
(template) code, but are needed to satisfy this "hidden" dependency;
look at Ganeti/Jobs.hs before this patch, for example.

This is also not very obvious, because we usually import Text.JSON
anyway; I only stumbled upon it while doing some cleanup work.

So to clean this up, the current patch changes the THH.hs to use not
string-derived, but identifier-derived names («'identifier» versus
«mkName "identifier"»); this is better, as the names must be
resolvable when compiling THH itself (once), and not when compiling
the multiple derived modules. As you can see, this allows removal of
extraneous imports from various modules.

Background information: an `mkName "foo"` results in a name of flavour
NameS (“An unqualified name; dynamically bound”) or alternatively to a
qualified name, but still dynamically bound. Whereas what we want is a
statically bound name: `'foo` results in a NameG flavour, “Global name
bound outside of the TH AST: An original name”.

One more explanation: the change is similar to going from 'x = eval
"map"' to 'x = map'; the name is no longer dynamically evaluated, but
statically when the module is compiled. In our case, previously names
were bound at target module compile time, now they are bound at THH.hs
compile time.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

We need to change a few things, most importantly CLI options defaults,
but otherwise we already used the path to files from functions which
were already in the I/O monad, so we don't have to change much of the
code flow.

Additionally, Path.hs now has an explicit export list, to stop
accidental leakage of symbols.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Also add some more haddock structure to the module. The RPC call
itself is rather trivial.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

By accident, we sent the node object as call data in this rpc call,
instead the version request (i.e. nothing). This is due to the fact
that the 'call' data comes second, not first in the function argument.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Since ERpcError is still not a monad, a simple foldr will have to
do for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

parseHttpResponse is such a simple function that we can actually
do the entire unpacking in rpcResultParse.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now ERpcError is still not a monad, but we can still simplify
rpcResultFill implementation for different calls simply by abstracting
it into two helpers.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There is no need (at least right now) for some of the resulting
types to be in additional monad, so let us just have more
"normal" functions.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As per Iustin's suggestion, this patch aligns parts of typeclass
instance on "=".
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As per Iustin's suggestion, show is not a good solution - so instead
the name is now explainRpcError.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch implements single instance info call - somewhat similar to
all_instances_info, except we give a specific instance name.

Current implementation of reading the InstanceInfo value is somewhat
counter-intuitive because when we query a node on thich there is
no instance with such name, this is not considered an error - but
it does produce an empty answer. Therefore the resulting type is
optional.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For storage_list call, the result type depends on the call parameters.
Therefore, we have to add call as an argument for rpcResultFill - and
by extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implementation of node version query.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Previous version of RPC calls implementation in Haskell did not take
into account that the actual result type for queries is a list, not
a dictionary.

This patch aims at fixing the problem "for now" - it is not a pretty
solution, but it does work. Note that parsing of the result is now
split into two parts - first, we check if server's aswer is positive,
then if it is, we procede with decoding the actual result.

Values and order of some fields in the result type were changed to
reflect actual order of arguments from server responses.

AllInstancesInfo call was particularly tricky, because it returns a
dictionary where keys are instance names - and the response from
a given node is correct if all the instances were deserialized, not
just some.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was repeated a lot, so we should abstract this into a type.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Curl error messages are self contained and don't need "error" prefix;
added RpcResult Error which will be used once proper deserialization
of rpc response is in place.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds missing Path entry and uses it in Rpc. It was not in
place before due to different build settings - i.e. if curl was
disabled, there were no problems.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 2cdaf225, “Re-enable standard hlint warnings”, got it almost
right. The only problem is that (confusingly) the default set of hints
is not in HLint.Default, but in HLint.HLint (it includes Default and
some built-ins).

After changing the lint file to correctly include the defaults, we had
another 128 suggestions:

  - Error: Eta reduce (2)
  - Error: Redundant bracket (4)
  - Error: Redundant do (17)
  - Error: Redundant lambda (7)
  - Error: Redundant return (1)
  - Warning: Avoid lambda (2)
  - Warning: Redundant $ (42)
  - Warning: Redundant bracket (35)
  - Warning: Use : (1)
  - Warning: Use String (4)
  - Warning: Use camelCase (10)
  - Warning: Use section (3)

which are fixed by the current patch. Note that the 10 "Use camelCase"
were all due to hlint not “knowing” the idiom of ‘case_’ (it does for
‘prop_’), for which I filled
http://code.google.com/p/ndmitchell/issues/detail?id=558

.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

These two files are not htools-specific, so let's move them out of the
HTools subdirectory/module hierarchy and directly under Ganeti.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Commit 5a1e31b4

 (Add infrastructure for, and two extra hlint rules)
was intended to add two *extra* hlint rules, but I didn't realise at
that time that "--hint" when first used overrides the built-in
lints. As such, since then we were basically running with just those
two rules, which resulted in many uncaught warnings/errors.

This patch fixes that (by importing the standard lint rules in our
custom hints file), and then goes to fix all the warnings that a
current hlint gives me. Compared to our current style, we have just a
few additions:

- zipWithM instead of map foo . zip …
- 'exitSuccess' instead of 'exitWith ExitSuccess'
- more uses of '.'

Additionally, we have to silence a case where hlint doesn't realise
why we are using '\e -> const (return False (e :: IOError)' instead of
just '\e -> return False' or even 'const (return False').

One warning that is generated by hlint ("Use void") can't be fixed
until we deprecate GHC 6.x, as only GHC 7 has the 'void' function in
Control.Monad.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

node_info call takes hypervisors and vgs to ask for information about
node and returns bootid and results from hypervisors and volume groups.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

instance_list call takes a list of hypervisors and returns a list
of running instances.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

all_instances_info call takes a list of hypervisors and returns a list
of (name, memory, state, vcpus, time) - one element for each
instance.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Prepare and execute RPC call, prepare result for the call. These
procedures are generic in the sense that they only require the
Call and Result types to be connected.
We use curl library for sending http requests for RPC; as the library's
usage can be disabled, we need to use CPP preprocessor macros for some parts
of the code.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>