This fixes issue 125 (http://code.google.com/p/ganeti/issues/detail?id=125

)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Per the mailing list discussion, rename _oss to _os, both in cluster parameters
and in the rest of the code.

This is just an s/_oss/_os, with the exception of a small bit of cleanup
around the helper_os function in cmdlib.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This helps to find a value for complex opcodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch changes the behaviour of OpDiagnoseOS with regards to the
'valid' field to be similar to the one for the hidden/blacklisted
fields: unless this field is requested, invalid OSes are filtered out.

The rationale is that, except for the gnt-os info/diagnose, all other
users of this opcode are requesting the valid field just to filter out
invalid OSes, and not for any other use. Thus, changing this behaviour
makes these callers simpler.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This changes the behaviour of LUDiagnoseOS significantly.

The addition of hidden/blacklisted OSes would mean that each user-facing
client would have to filter intentionally such OSes from display, which
is not a good choice. Rather, the patch makes LUDiagnoseOS not return
any hidden or blacklisted OSes unless the hidden or respectively the
blacklisted status is requested.

While unconventional, this makes `gnt-instance reinstall --select-os`
work as intended without any changes; similar for gnt-os list. gnt-os
diagnose/gnt-os info are changed to query for, and display the new
fields.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

These were deprecated since Ganeti 2.0, it's time to remove them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

They'll be used for job priorities. Also add an empty line to
gnt-os where it's missing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is accomplished by keeping a list of waiting threads instead
of just their number inside the lock-internal condition. A few
other tweaks to the output format are also made.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch adds an initial implementation of a lock monitor, accessible
for the user through “gnt-debug locks”. It currently shows all resource
locks: BGL, nodes and instances. Config and job queue locks could be
shown too, but wouldn't be of much help.  The current owner(s) and mode
are also shown.

Showing pending acquires will require further changes on the SharedLock
internals and is not yet implemented.

Example output:
$ gnt-debug locks -o name,mode,owner
Name            Mode      Owner
BGL/BGL         shared    JobQueue19/Job147
instances/inst1 exclusive JobQueue19/Job147
instances/inst2 -         -
instances/inst3 -         -
instances/inst4 -         -
nodes/node1     exclusive JobQueue19/Job147
nodes/node2     exclusive JobQueue19/Job147

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

When removing the field in commit 9bdab621, I forgot to remove some parts.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Based on manual inspection (that the command only does a submit of some
jobs/opcodes), we re-add the dry-run option on a subset of the existing
commands.

A few more commands could use dry-run, but the code doesn't deal nicely
with no results, so fixing those is left for later patches.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

These commands use or display the result from the LU, so in case of
dry-run, they will crash or display just 'None'. At least checking that
the result is 'true' (in the boolean sense) will make them work better.

As for gnt-os modify, it didn't pass the 'opts' parameter properly to
SubmitOpCode, so the dry-run option was silently ignored.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Test multiple opcodes, also with failure.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

“gnt-backup export” requires the target node. Until now, the master
daemon would complain that the “parameter
'OP_BACKUP_EXPORT.target_node' fails validation”. With this patch,
an additional check is done in the client program.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

This patch fixes two issues with job archival. First, the
LoadJobFromDisk can return 'None' for no-such-job, and we shouldn't add
None to the job list; we can't anyway, as this raises an exception:

  node1# gnt-job archive foo
  Unhandled protocol error while talking to the master daemon:
  Caught exception: cannot create weak reference to 'NoneType' object

After fixing this, job archival of missing jobs will just continue
silently, so we modify gnt-job archive to log jobs which were not
archived and to return exit code 1 for any missing jobs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, we require both --force and --force-multiple for skipping the
confirmation on instance reinstalls. After offline conversations, this
has been deemed to be excessive, and this patch changes the meaning of
--force-multiple to be a “stronger” force, and not require both.

So, to skip the prompts:
- single instance reinstallation requires either --force or
  --force-multiple
- multiple instance reinstallation requires --force-multiple

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This can be used from shell-scripts to quickly check the status of the
master node, before launching a series of jobs (and handling the failure
of the jobs due to masterd other issues).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Most (all?) of our commands use dash-separator: replace-disks,
verify-disks, add-tags, etc. “gnt-cluster masterfailover” is an old
exception to this rule.

The patch replaces it with master-failover, add a compatiblity alias,
and updates the documentation for this change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The new name is then displayed by the clients.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

This patch fixes a bug when gnt-instance rename was invoked with
--no-name-check. It renames the internal variables to be consistent with
the ones in equivalent instance add code. Furthermore it checks whether
and instance rename is invoked with --no-name-check but without
--no-ip-check and throws an exception if so.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This doesn't allow addition/removal of individual volumes, only
wholesale replace of the entire list. It can be improved later, if we
ever get generic container parameters.

The man page changes replaces some tabs with spaces (hence the
whitespace changes).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is needed as now the parameter is no longer boolean, but tri-state.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This is in preparation for the rename of the opcode 'live' parameter to
'mode'.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Due to a small mistake I missed three non-critical pylint complaints for
commit e58f87a9. They're fixed with this patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since migration live/non-live is more stable (e.g.) for Xen-PVM versus
Xen-HVM, we introduce a new parameter for what mode we should use by
default (if not overridden by the user, in the opcode).

The meaning of the opcode 'live' field changes from boolean to either
None (use the hypervisor default), or one of the allowed migration
string constants. The live parameter of the TLMigrateInstance is still a
boolean, computed from the opcode field (which is no longer passed to
the TL).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This new opcode and gnt-debug sub-command test some aspects of the
job queue, including the status of a job. The bug fixed in commit
2034c70d was identified using this test. A future patch will
run this test automatically from the QA scripts.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Just calling utils.SafeEncode on the log message failed when it
wasn't of the type ELOG_MESSAGE and not a string. Now non-message
log entries are formatted using repr().

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch moves network utility functions to a dedicated module.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This introduces a new instance status field, named "oper_vcpus".
It contains the actual number of VCPUs an instance is using as
seen by the hypervisor.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a cluster parameter to hold the iallocator that will be used by default
when required and no alternative (manually-specified iallocator or
manually-specified node(s)) is given.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, the way to signal export failures is by the return value.
This means that if a client doesn't check the values (e.g. burnin), any
failure is being ignore. And this is what we've been doing forever in
burning (not actually testing that the export is successful).

This patch changes the behaviour of ExportInstance: it will abort with
an exception for any error, and removes the custom handling from
gnt-backup. This makes the behaviour consistent for any client (e.g.
RAPI), and it prevents false positives. If, for a given instance, a
subset of disks should not be backed up, the OS scripts should handle
that case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, GenericManyOps ignores the actual success or failure results
from the invididual jobs. We change this to return '0' (i.e. success)
only when all jobs failed, as many times we have just one job.

Together with the JobExecutor change, this will report failures
correctly when used with a drained queue and submit only, or when used
normally and the opcode actually fails.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>