This will help with the general daemon split of prepare/run, and flag
errors earlier in the startup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, we keep information about the "target" of a tag operation
in a data type similar to (TagKind, Maybe String). This is unsafe, as
nothing (at the type level) prevents us from accidentally having
(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing).

To fix this problem, we rename the current TagObject type to TagType
(an internal utility type), and create TagObject as a better/safer
data type (see the definition), which doesn't allow such possibilities
in the future.

The downside is that, since at encoding level (both opcode and luxi)
this is done in an ugly way (type elements spread at the same level as
level as other value), we have to add custom encoders/decoders. The
encoder is shared between the OpCode and Luxi usage, the decoder is
different however as Luxi uses custom decoding.

This also fixes the recent breakage in confd w.r.t. QueryTags.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is needed so that we have more flexibility in generating Luxi
serialisation code (deserialisation is still custom). Also, only
exceptions are now using the 'simple' field types, so we might be able
later to convert and remove that TH code as well.

Since we will use custom serialisation fields in the future, we change
the order of serialisation for custom-save fields; Luxi uses
positional as opposed to name-based ordering, so we need to keep this
stable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Doh, this is exactly the opposite of what we wanted… good thing no
test failed so far :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This can be improved, by taking all hardcoded names as parameters, to
serve as a more-general "build save clause for a multi-constructor
data type". I'm not renaming the function as well, since I don't know
exactly how much we can abstract later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This way constants like “rapi.RAPI_ACCESS_WRITE” can be used in
documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 208a6cff just included empty docstrings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of being given the priority for acquiring locks by means of a
parameter, mcpu will now call back. This is in preparation for
implementing a command to change a job's priority on the fly and allows
to change it while locks are being acquired (taking effect on the next
lock acquire).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

* devel-2.6:
  http/__init__.py: Remove extraneous argument
  rapi.testutils: Add utility to format HTTP headers
  rapi.testutils: Return headers from mock utility
  http: Add wrapper for mimetools.Message
  Add missing tests for commit f0d22861

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.6:
  http/__init__.py: Remove extraneous argument
  rapi.testutils: Add utility to format HTTP headers
  rapi.testutils: Return headers from mock utility
  http: Add wrapper for mimetools.Message
  Add missing tests for commit f0d22861

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

pylint complained, I fixed it, and unfortunately pushed too early.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Once again this will be used by forthcoming RAPI test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A newly added test for RAPI will also verify the returned headers. A
test in ganeti.rapi.client_unittest.py is split into smaller stand-alone
tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A newly added piece of code will also have to parse headers, so having
this wrapper saves us from copying this part of code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This should be the last module rename, promise!

We rename this to conform to the other hierarchies (e.g. Query), and
to not have both Confd.hs and Confd/*.hs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also, adjust comment to $(genOpCode) block to avoid repetition of
"only".

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is also
required for the "kind" attribute of OpTagsSet and OpTagsDel, which
are coming to OpCodes.hs next. Hence, we move TagObject there, and
adjust imports accordingly.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit f0d22861 changed the logic of
gnt_instance._ConvertNicDiskModifications to also allow a parameter
named “modify”. Unfortunately the corresponding unittest was not
updated. An “if”/“else” condition is also merged.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This mirrors the code in the Python code base, and is required for
clean error reporting during startup.

This patch implements the basic infrastructure; the confd daemon is
not yet modified to take advantage of this, just the types are
adjusted.

Also, the pipe-based error reporting will go in a future patch, once
we can actually use that for reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is only in master, so needed to be fixed separately.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instead of having to explicitely increment the value (“… += 1”), a call
to next() is enough. These numbers should in no case be re-used (they
are used for ordering tasks). Using “itertools.count” is useful here as
it guarantees that a returned number won't be returned another time.
Manual code for this could, over the course of time, gain unintended
bugs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

* devel-2.6:
  Fix compatibility with newer Haskell libraries
  Fix gnt-instance console with xl

Conflicts:
        Makefile.am (reordering, fixed)
        htools/Ganeti/Confd/Server.hs (hlint fixes on master)
        htools/Ganeti/Daemon.hs (hlint)
        htools/Ganeti/HTools/Backend/Rapi.hs (hlint)
        htools/Ganeti/HTools/ExtLoader.hs (hlint)
        htools/Ganeti/HTools/QC.hs (file renamed/split in master, fixed)
        htools/test.hs (we don't use maxDiscards, ignored)
        lib/constants.py (move to pathutils)
        lib/hypervisor/hv_xen.py (move to pathutils)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* stable-2.6:
  Fix compatibility with newer Haskell libraries
  Fix gnt-instance console with xl

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We had twice in the past days questions about devel/upload being
"broken", since bash re-reads shell scripts during their run and this
file can get regenerated due to Makefile changes.

Since we only need this to be dynamically built for 3 variables, let's
make the file static and read those three variables when it is run,
instead of when it is built, which allows us to re-read the "latest
version" of these vars as well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This small patch fixes compatibility with a few newer Haskell libraries:

- base 4.6, included with ghc 7.6, removed the deprecated 'catch'
  function from Prelude, so our "import Prelude hiding (catch)" is now
  an error; we workaround by using fully-qualified
  Control.Exception.catch name

- containers 0.5 changed the signature of 'deleteFindMax'; we
  workaround by using separate 'findMax' and 'deleteMax'

- QuickCheck 2.5 removed the 'maxDiscards' test parameter, replacing
  it with a much better 'maxDiscardsRatio'; however, until we can
  depend on that, we workaround by just removing it (we don't control
  anymore the maxDiscards, instead leaving it default; for our default
  test size, this is no change, as the default value is already 500,
  which is our default as well) and not printing it anymore

Tested on Squeeze (+extra libs), Wheezy and experimental, which covers
all supported GHC versions.

Also, merging this in master will be a pain, but unless we want to
stop supporting 2.6…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Rename xm-console-wrapper to xen-console-wrapper
- Pass the xen command to use as a parameter

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This replaces direct calls to “os.access” and
“os.path.exists”/“os.path.isfile”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This tests that the same Luxi calls are defined in Python and
Haskell. It doesn't test yet that their serialisation is correct
though.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* devel-2.6:
  Disable E1101 on ganeti/http/server.py:424
  Fix live migration under xl
  Don't check for xend port when using xl

Conflicts:
	lib/hypervisor/hv_xen.py: trivial

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.6:
  Disable E1101 on ganeti/http/server.py:424
  Fix live migration under xl
  Don't check for xend port when using xl

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Until now the only way to make live migration work in conjunction with
"xl" was to add ssh known_hosts keys for every node's secondary ip on
every other node.

With this command we remove the target key verification: this is not
worse than what we were doing before with "xm", and allows the migration
to happen under either toolstack, without extra manual work. Of course
the full security of ssh is not used by live migration, then.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

If the toolstack is set to "xl" we shouldn't ping xend for liveness
before attempting a live migration.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Up until now “IsBelowDir("/", …)” would never return True. The reason
was that an additional slash was added to the root path resulting in
“//", which is “implementation-defined” in posix and treated specially
by “os.path.normpath”.

This patch fixes the behaviour for this special case and adds tests
(also for IsNormAbsPath). A typo in the docstring is fixed. Calls to
“assert_” and “assertFalse” are changed to pass a message by keyword
argument.

It is a bit of a mess, but I hope the resulting behaviour is correct.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dato Simó <dato@google.com>

The name “priority” is already used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The function in is simplified in its structure and duplicated checks
have been merged.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

This change is mostly cosmetic. Previously, the literal "cluster" was
used for the 'name' field of tag operations on the cluster (as opposed
to a node or an instance). Since this field has a type of TMaybeString
specifically for the case of the cluster, it seems more correct to use
None, rather than an arbitrary string (that is not used by the callee).

Additionally: note in opcodes.py that groups also expect a name; the
previous comment only referred to nodes and instances.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

A call to _CalculateGroupIPolicy wasn't refactored during the merge.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>