Also do some minor code re-formatting.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This was not only copied for the networking fields in commit 306bed0e,
but commit cfcea7ef fixed wrongly ordered parameters and didn't fix the
original. Either way, this patch merges the two cases again. The newly
added function is already tested through the tests for
_GetLiveNodeField.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This makes entering the command easier.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

They should be in the same style as other descriptions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is not a trivial s/frozenset/compat.UniqueFrozenset/, but rather
only replaces “frozenset” where appropriate. Most of the places are
“static” information that doesn't change after the module has been
loaded.

Some docstrings and code formatting (e.g. empty lines) issues are
addressed as well. Some lines got too long and were wrapped.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch removes code which is no longer used due to refactoring:

- http.InitSsl, last usage removed in commit 33231500 (“Convert RPC
  client to PycURL”)
- rapi.baserlib.MakeParamsDict, last usage remove in commit 4e5a68f8
  (“RAPI: rlib1 removal”)
- rapi.baserlib.ExtractField, last usage removed in commit 028c6b76
  (“RAPI:Fix root list and unittest for it.”)
- qa.qa_utils.AssertNotEqual, last real usage removed in commit
  2f4b4f78 (“Simplify QA commands”) (but it was still imported for a
  while, until we extended pylint to run over the QA code as well)
- qlang._IsHostname, added in commit f8638e28 (“Detect globbing
  patterns as query arguments”) but never used
- cmdlib._BuildNetworkHookEnvByObject, last usage removed in commit
  f22433c0 (“Locking related fixes for networks”)
- NLD constants, which are obsolete—the nbma project is no longer
  maintained and it's not working with current ganeti

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch is enabling split queries for jobs for gnt-node and rapi
access (only for job listing, not job waiting).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Unit tests updated and expanded with an inheritance check.

The flag has no effect yet.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If an instance is offline we definitely shouldn't start it up.
But shutting it down, should it be up by mistake is not "that" bad.
Still, we only allow it with --force, as it still performs an action on
an instance we shouldn't touch. This should make everybody happy.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If an instance is running (eg. ERROR_up) and at the same time offline,
there's no way to either shut it down or reonline it. This allows
onlining it. Offlining is still disabled for running instances.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some fixes, added more information in a few points, removed a stale (5+
year old) TODO comment.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 141d1489 was a bit too enthusiastic. The three parameters added to
the list of parameters to be checked default to a value not evaluating
to false, leading to a failure on cluster initialization.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LEVEL_NODE_ALLOC should be aquired too if LEVEL_NODE is ALL_SET.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

“_SPICE_ADDITIONAL_PARAMS” is supposed to be the full list of
SPICE-related KVM hypervisor parameters with the exception of
“HV_KVM_SPICE_BIND”. The new test checks if all parameters starting with
“HV_KVM_SPICE_*” are included. Three previously missing parameters are
added to the list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This list is static at runtime and doesn't need to be recreated every
time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

When non-DRBD disks are used for an instance,
“lu.needed_locks[locking.LEVEL_NODE]” is set to “locking.ALL_SET” (which
is None). The assertion will then fail as None evaluates to False.

Reported by Constantinos Venetsanopoulos.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Doing so avoids triggering an assertion in mcpu which cross-checks the
node lock and node allocation lock acquisition mode.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit f0f8d060 (“Show old primary/secondary node on disk
replacement”) added two wrong uses of feedback_fn, which results in
log entries such like these:

      "log": [
        [
          7, [1351258326, 466214], "message", "Replacing disk(s) 0 for instance 'instance1.example.com'"
        ],
        [
          8, [1351258326, 749391], "Current primary node: %s", "node1"
        ],
        [
          9, [1351258326, 774829], "Current seconary node: %s", "node4"
        ],
        [
          10, [1351258329, 620410], "message", "STEP 1/6 Check device existence"
        ],

(the first and fourth are OK, the 2nd and 3rd are broken).

The API of jqueue._OpExecCallbacks.Feedback should probably be changed
to prevent such an occurrence in the future.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

For some opcodes, the output is not "stable", and depends on the exact
input values; this makes it harder to check consistency against
Haskell code.

To compensate for this, we add a way to override the formatting of the
OP_DSC_FIELD; by default, this is always "%s", but if the
OP_DSC_FORMATTER is defined (must be a callable), it is used to format
the actual value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Introduced in commit d4752785.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a "better than nothing" support, just for kvm and just joining
the machine to the opevswitch bridge with the right command.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This should be done because in the case of --no-name-check there
is no 'hostname.ip' attribute, causing an execution error.
'check_ip' is always set (in CheckArguments) even if --no-name-check
is passed in the command line.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The master daemon should not directly read files written by the node
daemon. This patch adds a new RPC to read the watcher pause file and
changes the master code to use it.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Instead of just setting the watcher pause file, which isn't replicated,
RPC is used to set it on all nodes (where possible). This means that
after an outage of the master node and a following master-failover, the
watcher will still be paused.

The RPC is attempted on all nodes. Errors for nodes marked offline are
ignored. All other errors are returned in the form of an exception
(there is no other option given the LUXI API).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The watcher pause file should be set/unset on all nodes at once, not
only the master node. For that a new RPC is needed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This addresses issue 218. When the number of inotify watches is
exhausted, for example by being set too low from the beginning or by
other programs, waiting for a job to change would just report a lost job
(e.g. “Error checking job status: Job with id 7817 lost”).

This patch changes the job watcher to no longer catch
“errors.InotifyError” and, this is by far the larger part of this patch,
adds unittests for this situation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Add more checks, some of them are deliberately redundant
- Descriptive error messages
- Add comment describing order to “tools.ensure_dirs”
- Avoid copying a list in an assertion in “tools.ensure_dirs”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Doing so simplifies to code a bit and never had a practical use.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Change the "--disks" option validation, to just check the format
of the dict and do not check whether the keys are included in the
IDISK_PARAMS constant at OpCode level. This allows the passing of
arbitrary parameters at the CLI, which will then be logically
checked at LU level (inside CheckArguments).

Currently, the check wrt to IDISK_PARAMS already happens in all
related LUs (InstanceSetParams, InstanceCreate, InstanceRecreateDisks),
with the use of ForceDictType(), so there is no need for additional
checking code.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed a unittest that needed adjustement]
Reviewed-by: Iustin Pop <iustin@google.com>

It will be needed by the DRBD data collector, that will be added shortly.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit ebd437a4 added two new entries to tools.ensure_dirs, but did so in
the wrong order. Patch forthcoming to improve the unittest's error
message.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

If the files were created by a user different, e.g. due to a switch from
running masterd as root to running it as a dedicated user, they couldn't
be modified/removed anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Wrap lines in a consistent manner (uid/gid on the second line) if
wrapping is necessary at all. “git diff --color-words” shows no
difference at all as only whitespace changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Until now, the flag was unset on a master failover unless the
“$localstatedir/lib/ganeti/queue/drain” file existed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

When used instead of a plain call to “frozenset”, this would have
avoided the issue fixed in commit e2dd6ece. The new function is located
in the “compat” module as it will be used at module load time in most
places and should therefore reside in a place with very few
dependencies.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Unfortunately there was a bug in commit 518a45e5 whereby ECODE_TEMP_NORES
had the same value as ECODE_NORES, leading to failures in a Haskell
test. Of course this would also have affected other users of the constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>