This patch fixes some inconsistencies betwen the network
design doc and the actual user interface. Adding reserved
ips works with option --add-reserved-ips and removing them
with --remove-reserved-ips. There is no toggling of the
reservation status of ips.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 32e3d8b1 (“opcodes: Network parameter improvements and fixes”)
changed a few parameters in the network add, connect and set params
opcodes, but some of the changes are buggy. The patch changed the type
to TMaybe(), whereas the default should have been changed from None to
NoDefault - the network, mode and link are required. It also wrongly
unified all types as _CheckCIDRNetNotation, whereas some are
AddrNotation.

The patch fixes these (correctly, I hope) and also add some new
aliases for readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We have many cases in the code where we write TOr(TNone, a), so let's
introduce a combinator that simplifies this case.

Beside replacing the above with TMaybe(a), I did a few other parameter
fixes:

- noop change TOr(TNone, TDict) to TMaybeDict
- noop change TOr(TNone, TNonEmptyString) to TMaybeString
- OpClusterSetParams, change master netmask from any integer to a
  non-negative integer

(I can move the last one to a separate patch if desired)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The network patches and an existing test added function-call based
dict construction as opposed to literal sintax.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: slightly improved code flow]
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of initiating many SSH connections to copy files using “scp”, a
JSON structure is passed to a program running on the node to be added.
The design is similar to the one used for SSH setup.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

commaInt now recognizes only the first 3 digits for numbers without commas.

It was erroneously recognizing numbers of any size before the first comma.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

So far, when running 'gnt-cluster verify' on a cluster which
has instances that violate the instance policy, those
violations were shown as an error. This patch makes them a
warning.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iusin@google.com>

This is just trivial fixes; I think I caught all of them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

I'm ignoring the use of Control.Arrow.*** since it's a non-obvious
function, and we're not widely using Arrows in the code base, so I
think switching to it would make the code less readable to most
people.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since we generate arbitrary strings and then we send them over a pipe
to Python, we can (and will) generate non-ascii chars, which means
when run in a non-UTF8 locale, this will break with ghc 6.12 and
later. It didn't break on my workstation, since I have a "nice"
locale.

So to fix this, we replace all name generations with explicit
genName/genNameNE and similar, to ensure we only send "nice" strings.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These are tested for equivalence with the Python code using existing
tests. These tests are not perfect - for many "TDict" we simply send
empty dicts, as we don't have a way to generate proper data, but the
tests did catch many trivial errors during writing of this patch
series.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This does a partial conversion of existing opcodes to the new
non-empty string type, which is needed so that we don't add even more
opcodes without this safety net (and to have consistent "target_node",
etc. in opcodes).

By partial I mean that only the opcode are changed, and not the entire
htools types/workflow, so we use error to abort if, when submitting
the opcodes, we detect an empty node name. FIXME added.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a bulk add of all node parameters needed for node and cluster
opcodes. The parameters are defined with a few helper functions in
this module, and, opposite from opcodes.py, there won't be any (new)
opcode attributes declared in-line (without a separate field
definition).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These are needed for completing the node opcode parameters.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This makes it easy to generate sets of "things" that don't even need
to have arbitrary instances, but which have bounded/enum instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

This fixes a FIXME actually recommending this move :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The QR_VIA_OP/QR_VIA_LUXI types in Python are using yet another
validation mode: QR_VIA_OP is the base type, and QR_VIA_LUXI extends
it (when doing luxi queries). But on the wire they have the same
representation.

To accommodate this properly, we split the ItemType in two: a
QueryTypeOp and a QueryTypeLuxi, joining them back together in
ItemType itself. This requires custom serialisation/deserialisation,
but allows us to express correctly that at opcode level, we only
accept a QueryTypeOp, but at Luxi query level, we accept either of
them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a very basic type and 'Objects' is a heavy-weight module. By
moving it to 'types' we simplify (in the future) the import chains.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, we have some types that we kept as dictionaries in the
objects representation (due to inconsistent behaviour, or other
technical reasons). This should be improved in the future, but in the
meantime we can improve the Container type by allowing its keys to be
non-strings; this is needed, for example, for better safe in
DiskParams type, where we don't want arbitrary strings as keys, but
only the actually defined types.

To implement this change, we generalise the type (Container →
GenericContainer), and in the process we introduce a type class for
"things that can be represented as strings". This is very similar to a
combination of Read and Show, but with custom string
representation. The new type class is needed because in JSON
representation, object keys must be strings, so we need to be able to
serialised/deserialise the generic keys to/from plain strings.

We also add the instance for DiskTemplate at the same time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

These are needed both in opcodes and (potentially) outside, so we add
them in the main Types module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on
<ganeti-devel@googlegroups.com>.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on
<ganeti-devel@googlegroups.com>.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on
<ganeti-devel@googlegroups.com>.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It is no longer “--enable-remote-commands”, but rather
“--enable-restricted-commands”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds a new command, “gnt-node restricted-command”. Since the
semantics are different from “gnt-cluster command”, the same subcommand
shouldn't be re-used.

The included man page update also includes a small description of how to
configure and use remote commands.

Restricted commands can be run on a whole node group or on any number of
manually chosen nodes.

The output is similar to “gnt-cluster command”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

By setting the “delay_iallocator” parameter one could delay the call to
the instance allocator. This was not used for some time now, but the
code was still there.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUInstanceFailover and LUInstanceMigrate use TLMigrateInstance and had
the essentially same code for expanding names and declaring locks. In
LUInstanceMigrate.ExpandNames there was a mistake which led to node
resource locks not being declared properly. The two DeclareLocks methods
were exactly the same, down to the byte.

TLMigrateInstance had a lot of keyword parameters. Since it is only used
on two places, all parameters were made positional, making it easier to
determine the value of a parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Lint complains about too many variables in the CheckPrereq
method. While I think that this method/class would need
some refactoring in general, at least this patch makes lint
shut up for now.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>

When modifying an instance, so far the specs were not checked against
the ipolicy. This patch fixes this issue.

Note that for backend parameters which have a minimum and a maximum
value (currently only memory), it checks both limits against the
ipolicy. Because locking of the instance's node group was necessary, a
TODO of commit b8925b86 was fixed as well.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Upcoming changes will add opportunistic locking to “locking.LockSet”.
Doing so will require additional code in “LockSet.__acquire_inner”, at
which point the existing “want_all” parameter does not always apply.
This patch replaces it with a “mode” parameter. For opportunistic
locking a new mode will be added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The examples given in the RAPI documentation are not exhaustive. This
patch tries to clarify this by adding ellipses where appropriate. One
example's indentation is also fixed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

“locking.PipeCondition” and “locking.SharedLock” define “__repr__”,
which until now was not tested at all.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Now that 2.6 is essentially finished and 2.7 going to be branched
soon-ish, I thought it would be a good moment to replace some single
quotes in test/*.py. Merge pains should be limited.

In one place in test/ganeti.locking_unittest.py, spaces are added for
assignments.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Quote characters
- Docstrings

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Also update configure.ac.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Some type descriptions are rather long. If "None" is listed at the end
or somewhere in between it is easily missed. Therefore it should be at
the beginning, e.g. "None or (long description)".

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fails if the default value of an opcode parameter doesn't verify.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>