GitLab

It is implemented as a single binary receiving, as its first parameter,
the name of the actual data collector to be run.
This way, its structure can be used for all the future data collectors.

Also, factored out of bdev.py into constants.py the location of the DRBD
status file.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implementation of the network objects and address pool. Functionality
as in the reverted commit b9a616e1, but
now using only the vector library and not the bit-vector library.
Tested with vector library version 0.9 and 0.10., which are
also installed on the buildbot machines now.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It also uses values from “lib/errors.py”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This reverts commit b9a616e1, which
depends on the "bitvec" library. We need to investigate a bit further that
dependency, as it in turns depends on "vector" 0.9.1 or *below*, since
0.10.* removed support for Data.Vector.Unboxed.Safe which it uses.
Signed-off-by: Helga Velroyen <helgav@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Implementation of the network and address pool class in
Haskell. Not complete yet. Includes unit tests that cover
all functionality that is so far implemented.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sometimes a node is not removed properly from a cluster (especially
during development). This new tool stops all daemons and removes (after
making copies) the most critical files.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The design for this is in “doc/design-node-add.rst”. The tool receives a
JSON data structure on stdin and configures the node's daemon after
verifying the received values.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

epydoc allows overriding configuration values via environment variables.
While this might be useful in certain use cases, but as no prefix
whatsoever is used, conflicts are easily created. Some people have the
environment variable “NAME” set, effectively overriding the project name
set in epydoc.conf. A bug in epydoc causes an error if non-ASCII
characters, such as German umlauts, are used in NAME.

$ NAME=Täscht make py-apidoc
[…]
UNEXPECTED ERROR:
'ascii' codec can't decode byte 0xc3 in position 73: ordinal not in range(128)

$ parse=false make py-apidoc
[…]
epydoc: error: Invalid option combination: --parse-only and --introspect-only.

This patch changes the call in Makefile to reset the environment given
to epydoc save for PATH and PYTHONPATH.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This module implements some algorithms on Data.Graph data structures.
At the moment its main functionality is an LF-color implementation
(greedy coloring in descending order of degree). There are also a few
extra functions to calculate the degree order, and convert the node to
color mapping to color to nodes.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Just some cleanup:

- don't run hlint over hpc-htools, since it's just a symlink to
  htools.hs (so we'd get duplicate warnings)
- build rpc-test in make really-all, via HS_ALL_PROGS variable
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since hlint doesn't check style issues but rather code only issues,
let's add a simple Makefile check for too-long lines, and fix the
single bad case we have currently.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Indentation within rules is fixed to always be 2 spaces (as opposed to
mixed tabs/2 spaces/4 spaces). Additionally, the check-dirs rule,
which was more complex than necessary, has been simplified: moved
auto4mte.cache to DIRCHECK_EXCLUDE, and simplify the find call and the
exit condition (no need for if test …; then exit 0; else exit 1; fi).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of having two different lists of directories, now there is only one.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add the design document for opportunistic locking.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The serialization itself is done by Text.JSON, so the tests deal with checking
that Text.JSON objects are created correctly from the DRBD parser data
structures.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It turns out that in newer GHC versions, the suffix shouldn't be
prefixed with a dot; otherwise name mangling is broken.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The hpc tool, on squeeze, does not handle utf8 encoded files correctly.
The Attoparsec test is just needed to verify whether the parser handles
uft8 correctly, so the solution is to exclude it from coverage checks.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Until now ssconf would limit the amount read from files to 128 KiB and
silently ignored files larger than that. With this patch a check is
added by using fstat(2) on the file descriptor while it's being read.

There were no tests file ssconf at all, so some are added.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It is no longer “--enable-remote-commands”, but rather
“--enable-restricted-commands”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

This adds tests that existing test files can be parsed by the Haskell
parser as well, plus one new test file.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A new directory for haskell modules about block devices has been created
The parser is divided in two modules:
* one exports the data types describing the DRBD status
* one exports the parser itself
Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin@google.com: indentation/alignment fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

Due to TemplateHaskell stage restrictions, we can't define parameters
in the same module as we're using them for TH, so we have to define
all module parameters in a separate module.

This patch therefore splits OpCodes.hs in two, adding that module and
moves most code there (types, parameters, etc.). The remaining parts
in OpCodes.hs, the actual opcode definitions, now use more parameters
instead of direct field definitions (more will come later)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

There are already three cases where we copied type definitions between
the htools-specific types into the main ganeti code. Let's stop doing
this ☺ and create a common types module that holds these.

Note that there already exists BasicTypes.hs, but that refers to very
low-level types, and can't use TH derivation itself.

A side effect of this unification is that there is a small conflict
between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
in the InstanceStatus type.

The patch also moves the tests related to these types to a new test
module.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

- Remote unnecessary quote characters
- Change some uses of “test "$foo"” to “test -n "$foo"”
- Merge “if/else/if” into “if/elif”
- Wrap lines longer than 80 characters and add a test to “make check”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some versions of hlint crash on the Attoparsec test file because they
are not able to correctly handle unicode characters if the proper encoding
(--utf8) is not specified.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

More design details will be added regarding setting up the node daemon,
so this is more about adding a node than just setting up SSH.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As per design document (doc/design-remote-commands.rst), a number of
rather strict tests is applied to any incoming request, a delay is
inserted upon errors and returned error messages are very generic
(unless it's the actual command that failed). There are unit tests for
all of the newly added code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

By default remote commands are disabled and need to be explicitely
enabled at build time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This allows offlining nodes that don't respond if they are part of a
linux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This allows controlling the cluster master role if the nodes are part of
a linux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unlike existing tests, this actually tests RAPI at the interface with
the HTTP server. This way authentification can also be tested. A test
for “/2/query/…” is included as it's a bit special.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Some GHC flags are very useful, but only appear in more recent GHC
versions. To support the use of such flags while still supporting
older compilers, let's add conditional checks and enabling based on
the results.

Currently only `-fwarn-incomplete-uni-patterns` is enabled, which
detects refutable patterns in lambda expressions and pattern bindings
(e.g. "let (Just x) = y"); such constructs are bad as they can lead to
runtime exceptions.

Additionally, fix an existing such bad construct in a test case; we
workaround it by using error, since that should never fail.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Attoparsec is known to have had issues with parsing non-ASCII strings.
This test makes sure that parsing of Unicode characters works fine.
Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin: small doc string fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

"make regen-vcs-version" is run at every ./devel/upload time, in order
to have reasonably up-to-date information in the uploaded
scripts/binaries. However, this means it currently triggers
recompilation of (at least) Ganeti/Version.hs, and sometimes more than
that.

Since we only care about the contents of the file and not the
timestamp, let's change the make rule so that it only updates the file
if there are actually changes in the content, as checked by
"cmp". This results in a much faster ./devel/upload, especially for
small changes in the (non-Haskell) code, or when the code has already
been compiled.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This should be the last module rename, promise!

We rename this to conform to the other hierarchies (e.g. Query), and
to not have both Confd.hs and Confd/*.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

We had twice in the past days questions about devel/upload being
"broken", since bash re-reads shell scripts during their run and this
file can get regenerated due to Makefile changes.

Since we only need this to be dynamically built for 3 variables, let's
make the file static and read those three variables when it is run,
instead of when it is built, which allows us to re-read the "latest
version" of these vars as well.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Rename xm-console-wrapper to xen-console-wrapper
- Pass the xen command to use as a parameter
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It has been superseeded by “prepare-node-join”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Five modules under the HTools/ directories are backend
implementations, so let's move them to a separate directory, to more
clearly show the hierarchy. I wanted to do this for a while, but
merging between branches is always an issue, so let's do it know since
we have an opportunity.

This patch contains the actual renames, the required changed module
names, imports, etc., but no other changes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>