Until now ssconf would limit the amount read from files to 128 KiB and
silently ignored files larger than that. With this patch a check is
added by using fstat(2) on the file descriptor while it's being read.

There were no tests file ssconf at all, so some are added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Missing spaces around operator.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

There is no good reason why these two constants should be inside the
class. This patch moves them to the module so they can be used without
“self”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit b0dcdc10 removed the last user of this class.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Include filename in error message
- s/ip/IP/
- Stop using superfluous variable

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Extend _GetNetworkStatsField() to accept 4 arguments.
This fixes Issue 323.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add uuid and serial_no to _NETWORK_SIMPLE_FIELDS in order to export
them in network queries. Modify gnt-network to list them too.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

RE_VALID_SSCONF_NAME is not used anywhere.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Commit 9b7e05a6 didn't quite work as expected: glm.is_owned expects a
locking level, not a lock name.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

A new tool to configure the node daemon will also have to load and
verify JSON data.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Most tools had their own “SetupLogging” function, but they were all
essentially the same. This patch adds a generic version to “utils.log”
and provides unit tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* Now the parser completely consumes the input, up to the end of the text.
* Name of the test suite module changed to adhere to naming conventions.
* Some reformatting of the source code.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Block instance allocations when all node locks will be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The execution context ID can be passed right away.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Check whether the BGL is actually held instead of just testing
“REQ_BGL”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The Log* functions are re-exported through the LU. Inline use of string
formatting (“%”) was converted to additional arguments to Log* (unlike
“feedback_fn”, these functions support it). Some punctuation has been
removed from messages, as well as some small re-wordings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Don't use inline versions of CommaJoin.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

… instead of constructing an object, only to fill it separately.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

A new method is added to check whether the LockSet-internal lock is
held. This is the case after LockSet.acquire was called with
locking.ALL_SET.

Unit tests are updated, including one where the list of names must be
empty.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 8716b1db added a new lock level, LEVEL_NODE_ALLOC. It is ahead of
LEVEL_INSTANCE. The latter was hardcoded in mcpu to be locked right
after the BGL, effectively ignoring LEVEL_NODE_ALLOC.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The bug was in the test itself, not in the tested code.

Also, fixed a line longer than 80 characters in the same file.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The new lock is similar to the BGL in the sense that it has its own
level and there is only one. It is called “node allocation lock”.
Logical units will use it to synchronize with instance creations, which
in turn will start using opportunistic locks on nodes.

Additionally, an assertion in GanetiLockManager gained a message.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These new tests substitute the HTests, as by the suggestion
received after submitting the previous patch.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch fixes some inconsistencies betwen the network
design doc and the actual user interface. Adding reserved
ips works with option --add-reserved-ips and removing them
with --remove-reserved-ips. There is no toggling of the
reservation status of ips.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 32e3d8b1 (“opcodes: Network parameter improvements and fixes”)
changed a few parameters in the network add, connect and set params
opcodes, but some of the changes are buggy. The patch changed the type
to TMaybe(), whereas the default should have been changed from None to
NoDefault - the network, mode and link are required. It also wrongly
unified all types as _CheckCIDRNetNotation, whereas some are
AddrNotation.

The patch fixes these (correctly, I hope) and also add some new
aliases for readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We have many cases in the code where we write TOr(TNone, a), so let's
introduce a combinator that simplifies this case.

Beside replacing the above with TMaybe(a), I did a few other parameter
fixes:

- noop change TOr(TNone, TDict) to TMaybeDict
- noop change TOr(TNone, TNonEmptyString) to TMaybeString
- OpClusterSetParams, change master netmask from any integer to a
  non-negative integer

(I can move the last one to a separate patch if desired)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The network patches and an existing test added function-call based
dict construction as opposed to literal sintax.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: slightly improved code flow]
Reviewed-by: Iustin Pop <iustin@google.com>

Instead of initiating many SSH connections to copy files using “scp”, a
JSON structure is passed to a program running on the node to be added.
The design is similar to the one used for SSH setup.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

commaInt now recognizes only the first 3 digits for numbers without commas.

It was erroneously recognizing numbers of any size before the first comma.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

So far, when running 'gnt-cluster verify' on a cluster which
has instances that violate the instance policy, those
violations were shown as an error. This patch makes them a
warning.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iusin@google.com>

This is just trivial fixes; I think I caught all of them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

I'm ignoring the use of Control.Arrow.*** since it's a non-obvious
function, and we're not widely using Arrows in the code base, so I
think switching to it would make the code less readable to most
people.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since we generate arbitrary strings and then we send them over a pipe
to Python, we can (and will) generate non-ascii chars, which means
when run in a non-UTF8 locale, this will break with ghc 6.12 and
later. It didn't break on my workstation, since I have a "nice"
locale.

So to fix this, we replace all name generations with explicit
genName/genNameNE and similar, to ensure we only send "nice" strings.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These are tested for equivalence with the Python code using existing
tests. These tests are not perfect - for many "TDict" we simply send
empty dicts, as we don't have a way to generate proper data, but the
tests did catch many trivial errors during writing of this patch
series.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This does a partial conversion of existing opcodes to the new
non-empty string type, which is needed so that we don't add even more
opcodes without this safety net (and to have consistent "target_node",
etc. in opcodes).

By partial I mean that only the opcode are changed, and not the entire
htools types/workflow, so we use error to abort if, when submitting
the opcodes, we detect an empty node name. FIXME added.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a bulk add of all node parameters needed for node and cluster
opcodes. The parameters are defined with a few helper functions in
this module, and, opposite from opcodes.py, there won't be any (new)
opcode attributes declared in-line (without a separate field
definition).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>