This patch changes “gnt-node add” to use the newly added
“prepare-node-join” tool. Hereby Paramiko is no longer a hard dependency
for setting up SSH on nodes.

In “gnt_cluster.py”, a positional parameter is no longer passed as a
keyword parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of building the dictionary locally, the global version in
“ssh.py” can be used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This new function returns the file paths for all of a user's SSH-related
files (RSA, DSA and authorized_keys).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch changes “utils.RunCmd” to accept a file-like object or a
numeric file descriptor which will be used as the command's standard
input. One use-case will be to pass all necessary data to
“prepare-node-join”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will also be used for changing jobs' priorities. All parameters to
the common function are non-optional.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This will be useful in “gnt-node add”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This dictionary will also be useful in “gnt-node add”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Other places, such as “ssh.GetUserFiles”, use a structure where the
private key comes before the private key. Until now prepare-node-join
did the opposite, that is the public key came first. To avoid confusion
and potential bugs, this is changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A public key already includes the necessary prefix (“ssh-rsa” or
“ssh-dss”), so there is no need to add it again.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Without this parameter, either an error would be raised or “.ssh” would
have to be created. Now it is possible to retrieve the paths without
requiring the “.ssh” directory to exist.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Five modules under the HTools/ directories are backend
implementations, so let's move them to a separate directory, to more
clearly show the hierarchy. I wanted to do this for a while, but
merging between branches is always an issue, so let's do it know since
we have an opportunity.

This patch contains the actual renames, the required changed module
names, imports, etc., but no other changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Sorry!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Thanks Dato for catching this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Testing with a newer hlint found a few minor issues; but all are real,
valid recommendations:

- don't use "if cond then f x else f y", but "f (if cond then x else y)"
- "if a then b else True" is equivalent to the simpler "not a || b"
- and as usual, one more ignore to our "testing basic properties"
  module

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is very THH specific, and applies to all serialisations generated
by THH, so I'm adding it in its own module.

Probably we should add some more generic tests, but in general THH
code is tested by the various definitions; this new field type however
is not (yet), so this is why I want this specific unittest.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This follows a conversation we had for how to deal with
optional-but-required fields in JSON serialisations: fields which are
optional (can be either a given type or 'null'), but where the 'null'
value is required. There are just a few of these in the Python code,
but we should support them nevertheless.

The patch changes the 'isOptional' attribute from boolean to a custom
ADT, three-typed. This allows us to keep the same path on load (which
deals with both cases), but use a custom save path where we explicitly
save the 'null' value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Change {exp,act}Code to {exp,act}Ver, which gives a better idea that
the integer fields represent version numbers.

Also:

  - errors.py: update OpPrereqError's docstring to note that an error
    code is always expected as the second argument (it was previously
    optional).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Note that this commit has no Makefile.am changes, as the files were
not actually used. So it's better to actually remove them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The isOptional function is no longer used after a1505857 (“Convert
opcode TH code to the use of Field type”).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix a verb tense and add a missing verb.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

An earlier version of this patch series verified all paths in cmdlib in
the master daemon. With this change all that verification code is moved
to bdev to run inside the node daemon. The checks are much stricter
now--it is no longer possible to use forbidden paths (e.g. /bin) to
manipulate file storage devices (once these checks are being used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A new function will be added to change a job's priority.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

If the option is used elsewhere, the numeric value is directly
available.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Somehow this was missed in commit 0422250e.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

It is not actually used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This options file for ghci preseeds the correct include paths, so that
interactive sessions don't need to always pass these args.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch converts all the call paths from 'Result' (which contains
just string errors) to 'ErrorResult', which holds
GanetiException-encoded errors. We can now return proper
OpPrereq/OpExec errors to the clients of the luxi/query socket.

The patch touches many files as we had to convert the entire call
chains in a single round. But it should be pretty straightforward
otherwise:

- change 'Result' into 'ErrorResult'
- add error annotations: change "Bad msg" into "Bad (XXXEror msg)"
- add a helper function for confd, where we don't send to client
  formatted exceptions, to convert back from ErrorResult into Result
- change tests similarly, where needed

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In Python, formatError also returns the exit code, but I find that
splitting them leads to clearer code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

As described in the module doc string, while writing this it dawned
upon me that we're mixing all errors together into a single hierarchy
(well, type on the Haskell side), which is not good. Some errors are
used purely within noded, some in the CLI frontends, etc. so these
should not be the same type; frontend functions should only be able to
raise frontend errors, not backend ones.

As to this patch itself, I've used again Template Haskell to generate
both the data type and the serialisation functions, as the initial
version, hand-written, seemed too prone to errors due to string
matching.

A small unittest for checking serialisation consistency is also added.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These are almost generic, so let's change the signatures a bit a make
them fully so.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

This 'simple' way of defining objects will be used also for errors, so
let's make it less Luxi-specific.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The 'dumb-allocator' has been removed almost two years ago (commit
6f547f96, “Remove dumb-allocator”), let's remove this special casing
for it from devel/upload.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

While running with a wrong --prefix/--sysconfdir, I saw that
devel/upload actually uses a hardcoded path for the init script, even
though it installs it in the correct place.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This documents the status (or wanted status) of some example ocf modules
I've written for Ganeti. They are far from perfect, but they should be
shipped as a starting point for other people who want to run this to
improve on.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

To people not used to it, it was completely non-obvious why
"./devel/upload" didn't do anything.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, when we have an assertion error raised from cmdlib, it looks like this:

  [cluster] root@node4:~# gnt-instance grow-disk instance1 0 1G
  Failure: command execution error:

This is very very confusing. This patch adds a bit of traceback
formatting to improve this as follows:

  [cluster] root@node4:~# gnt-instance grow-disk instance1 0 1G
  Failure: command execution error:
  Internal assertion error: please report this as a bug.
  Error message: ''; location:
    File "/usr/lib/python2.6/dist-packages/ganeti/cmdlib.py", line 11954, in CheckPrereq
      assert False

This is not perfect, but at least it shows better what the problem is.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>