Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is compatible with the previous version, but also allows specifying
more than one nic, by giving a "nics" list of dicts. The two methods
(individual fields for the first nic, and list of all nics) are
incompatible with each other.

The default remains one nick with no parameters specified.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This CL updates QueryInstanceData to return NICs in the new format (mac,
ip, mode, link) and fixes gnt-instance info to properly display them.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Compatibility with the old parameters is maintained, by allowing to
query for "bridge" "nic.bridges" and "nic.bridge/N", but None is
returned in that case for routed nics.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently the OSes have a special, customized error handling: the OS
object can represent either a valid OS, or an invalid OS. The associated
function, instead of raising other exception or failing, create custom
OS objects representing failed OSes.

While this was good when no other RPC had failure handling, it's
extremely different from how other function in backend.py expect
failures to be signalled.

This patch reworks this completely:
  - the OS object always represents valid OSes (the next patch will
    remove the valid/invalid field and associated constants)
  - the call_os_diagnose returns instead of a list of OS objects, a list
    of (name, path, status, diagnose_msg); the status is then used in
    cmdlib to determine validity and the status and diagnose_msg values
    are used in gnt-os for display
  - call_os_get returns either a valid OS or a RPC remote failure (with
    the error message)
  - the other functions in backend.py now just call backend.OSFromDisk()
    which will return either a valid OS object or raise an exception
  - the bulk of the OSFromDisk was moved to _TryOSFromDisk which returns
    status, value for the functions which don't want an exception raised

The gnt-os list and diagnose commands still work after this patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This is a big change, because we need to cleanup its users too.

The call and thus LUVerifyDisks LU used to differentiate between failure
at node level and failure at LV level, by returning different types in
the RPC result. This is way too complicated for our needs.

The patch changes to new style result (easy change), and then:
  - changes LUVerifyDisks.Exec() to return a tuple of 3-elements
    instead of 4-elements; we collapse the «nodes not reachable» and
    «nodes with LVM errors» in a single dict
  - changes gnt-cluster to parse 3-element results and simplifies the
    different by-error handling code

Note that the status is added in ganeti-noded, and not in the function
itself, as the function is used in other places too.

This was tested with down nodes and broken VGs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since default_bridge is deprecated, avoid displaying it when querying
for cluster information.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

And remove the -b option, as default nic parameters can be used instead.
We could support the option, but that would add more code, and since
cluster init is not a frequent operation, it's better to keep the code
clean.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Commit 55efe6da "Convert instance
reinstall to multi instance model" actually broke instance reinstall for
single-instance cases. This one-liner fixes it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Don't check for non-existant BE parameters, ForceDictType will do it
Use utils.FillDict to populate the defaults

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If --no-etc-hosts is passed in at cluster init time we set a new
parameter in the cluster's object to false, and avoid adding nodes to
the hosts file. The UpgradeConfig function is used to set the value to
True, when upgrading from an old configuration version.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This (somewhat big) patch adds support for remotely rebooting the nodes
via whatever support the hypervisor has for such a concept.

For KVM/fake (and containers in the future) this just uses sysrq plus a
‘reboot’ call if the sysrq method failed. For Xen, it first tries the
above, and then Xen-hypervisor reboot (we first try sysrq since that
just requires opening a file handle, whereas xen reboot means launching
an external utility).

The user interface is:

    # gnt-node powercycle node5
    Are you sure you want to hard powercycle node node5?
    y/[n]/?: y
    Reboot scheduled in 5 seconds

The node reboots hopefully after sending the reply. In case the clock is
broken, “time.sleep(5)” might take ages (but then I suspect SSL
negotiation wouldn't work).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch converts ‘gnt-instance reinstall’ from single-instance to
multi-instance model; since this is dangerours, it's required to pass
“--force --force-multiple” to skip the confirmation.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This small patch changed the batch create functionality to use the job
executor instead of single-job submits.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch modifies the start instance script, opcode and logical unit
to support temporary startup parameters.

Different from 1.2, where only the kernel arguments were supporting
changes (and thus xen-pvm specific), this version supports changing all
hypervisor and backend parameters (with appropriate checks).

This is much more flexible, and allows for example:
  - start with different, temporary kernel
  - start with different memory size

Note: in later versions, this should be extended to cover disk
parameters as well (e.g. start with drbd without flushes, start with
drbd in async mode, etc.).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently QueryJob returns "None" when a wrong job ID is passed.
Handle this in gnt-job list, by printing an error for each wrong job,
and still giving output for all the jobs which actually do exist.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently doing a gnt-cluster-modify --no-lvm-storage is silently
ignored, as it passes a None value in vg_name, which is the same as not
modifying that parameter. Explicitely set the passed value to '', so the
non-true not-None value can be evaluate to actually remove a volume
group.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Even if we cannot modify all of them, they are useful information about
the current cluster.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The _TransformPath function is not used anymore in 2.0, let's remove it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently “gnt-debug submit-job” takes a single argument and has
non-trivial startup-costs; in order to exercise the job system, it is
better to be able to submit multiple jobs with a single invocation of
the script.

This patch extends it to take multiple argument, de-serialize the
opcodes and then submit all of them as fast as possible, in order to
increase pressure on the master daemon.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Alexander Schreiber <als@google.com>

The current implementation of “gnt-cluster getmaster” doesn't work on
non-master nodes, which is a regression from 1.2. This patch implements
it (again) via ssconf.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Alexander Schreiber <als@google.com>

Don't show all instances info by default, but require --all to be passed
for this time consuming operation.

Reviewed-by: iustinp

This patch adds a couple of both externally and internally reported
issues:
  - missing SGML tags (Issue 54), report and patch by superdupont
  - wrong variable used in the init.d script, report and patch by
    Karsten Keil <karsten-keil@t-online.de>
  - man page for gnt-instance reinstall needs clarification (Issue 56)
  - gnt-instance man page missing --disks documentation for
    replace-disks
  - gnt-node modify help output is unclear about the -C/-D/-O input
    format, and the man page doesn't document this command at all
  - “gnt-node modify -C yes” for offline or drained nodes had wrong
    error message
  - “gnt-instance reinstall --select-os” has wrong prompt, we only
    accept a number for the OS and not the template name

Reviewed-by: ultrotter

This patch raises an error in the master daemon in case the user
requests a locking query; accordingly, all clients were modified to send
only lockless queries. This is short-term fix, for proper fix the
clients should be modified to submit a job when the user request a
locking query.

The other approach would be to ignore the flag passed by the client;
this would be worse as client's wouldn't get at least an error.

The possible impact of this is multiple:
  - some commands could have been not converted, and thus fail; this
    can be remedied easily
  - the consistency of commands is lost; e.g. node failover will not
    lock the node *while we get the node info*, so we could miss some
    data; this is again in the thread of atomic operations which are
    missing in the current model of query-and-act from gnt-* scripts

Reviewed-by: imsnah, ultrotter

 # gnt-cluster queue foo
 Failure: prerequisites not met for this operation:
 Command 'foo' is not valid.

Reviewed-by: iustinp

Similar to the --disk fixes a while ago, --net is broken too. This patch
fixes it.

Reviewed-by: imsnah

This patch removes the extra_args parameter and instead switches the
instance to the HV_KERNEL_ARGS hypervisor option.

This is a big change, but it's a needed cleanup, this extra parameter on
all RPC calls is not generic and we also need to have a persistent value
here.

Reviewed-by: imsnah

Having hvattr descriptions is only confusing for the user, because even
if they explain better what an attribute is about, they don't help in
deciding what keyword should be used to actually set it. If in the
future we want descriptions they should probably live in constants.py,
and be displayed together with the key, rather than instead of it.

This patch also changes the handling of the vnc console connection
description, making compatible work with both kvm and xen-hvm.

Reviewed-by: iustinp

This patch adds back to the instance creation command (gnt-instace add,
gnt-backup import) the ‘-s’ short form option for specifying a
single-disk instance.

Also a small bug in gnt-backup import is fixed.

Reviewed-by: ultrotter

This patch changes the gnt-node and gnt-job list commands to accept
argument and list only the selected items, which is useful when having
many nodes or jobs.

It also removes the “--units” option from gnt-job list as we don't
actually use it.

Reviewed-by: imsnah

This patch changes the scripts so that the short name for the
“--iallocator” option is always ‘-I’.

Reviewed-by: ultrotter

Currently the batcher hypervisor parameter must be a dict with one
element (e.g. {"xen-hvm": { "acpi": true }}). This is overly complex and
hard to validate correctly; the patch splits it in two:
  - one "hypervisor" string parameter, with the name of the hypervisor
  - one "hvparams" dictionary, with the hypervisor parameters

The patch also changes the error handling in parsing the definition file
- since this is not a long-running file, we are less concerned with safe
closing of the file, and more with presenting meaningful error
messages.

Reviewed-by: killerfoxi

It's hvparams, not opts.hvparams.

Reviewed-by: iustinp

If hvparams is not set it will be [], so dict() will transform it to an
empty dict, which is safe in all cases.

Reviewed-by: iustinp

The patch sorts the instance list in gnt-node info output, in order to
make it more readable (and stable).

Reviewed-by: imsnah

The patch changes the pre-checks in node-add and re-add:
  - if the node is not already in the cluster, refuse to re-add
  - when re-adding, reuse the secondary IP from the cluster
    configuration
  - when re-adding, reset the offline and drained flags, so that RPC
    calls work (and we can actually upload the keys)

The patch also adds a missing log entry in LUSetNodeParams.

Reviewed-by: imsnah

We want all the hv/be parameters to have a known type, rather than a
random mix of empty string, boolean values, and None, so we declare the
type of each variable and we enforce/convert it.

- Add some new constants for enforceable value types
- Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
  holding not only the valid parameters but also their types
- Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
  the values from the type dict
- Convert all the default parameters to a valid type value
- Create a new ForceDictType utils function, to check/enforce a dict's
  element value types, with relevant unit tests
- Drop a few custom functions to check/convert the BE param types in
  utils and cli, in favor of ForceDictType
- Double-check the parameter types using ForceDictType in both scripts
  and LogicalUnits, when possible.

As a bonus:
- Remove some old commented-out code in gnt-instance
- Remove some already fixed FIXME
- Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
  in SetInstanceParams because the value was checked for validity before
  that transformation was made
- Fix a bug which prevented initing a cluster and passing hvparams to
  work at all
- ForceDictType allows an allowed_values for exceptions, which makes us
  able to do the checking even when some values must not be
  converted/typechecked (for example the 'default' string in
  SetInstanceParameters)

Reviewed-by: iustinp