Currently, we always list the api/variants, even if these are empty.
This patch changes so that we make clear distiction for empty values
("[no variants]" versus "[variants: ]"), and we only list variants and
parameters when the OS API indicates they should be supported.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This can be used to show the actual OS parameters and supported
variants, in a global manner (rather than per-node as gnt-os diagnose).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch exports all the way from backend a new field ‘api_version’
which holds the list of support API versions, and exposes the (already
computed) ‘parameters’ field.

The patch also reworks (again) the field calculation in its Exec()
method. All callers of LUDiagnoseOS pass in the 'valid' and 'variants'
parameters, thus having the special casing of whether to compute or not
the validity seems overkill. We move to a model where we always compute
these across-nodes arguments, in order to simplify the code, and we also
change the parameters set to be intersection of all node's values (which
means a change in description will drop the parameter from the list of
parameters).

Additionally, we update scripts/gnt-os, which was broken for multi-dir
OSes since the introduction of variants…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

We use _GetUpdatedParams in order to support removal too, and then
validate the OS parameters if the OS exists.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The patch also modifies the internal methods in LUDiagnoseOS and gnt-os
to deal with the format change of call_os_diagnose.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Introduce “gnt-os modify” command to make it possible to set the
per-os-hypervisor parameters.

Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In some cases pylint doesn't parse the import correctly, so we add
silences; but there are also many cases of unused imports, which we
simply remove.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Many of our functions have to follow a given API, and thus we have to
keep a given signature, but pylint doesn't understand this. Therefore,
we silence this warning.

The patch does a few other cleanups.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This removes unused variables in the rest of the code (outside lib/).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This patch should have only:

- pylint disables
- docstring changes
- whitespace changes

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

This patch removes the quotes from CommaJoin and converts most of the
callers (that I could find) to it. Since CommaJoin does str(i) for i in
param, we can remove these, thus simplifying slightly a few calls.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

List of OSes are displayed by gnt-os list, rapi, and gnt-instance
reinstall --select-os, and checked by burnin. In all of these show the
list with name+variant, if the os has variants.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

We already show the per-node os variants, also show the global ones.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Since >90% of the commands take the “--debug” option, and all should
actually take it (the gnt-job command is currently missing it), it makes
sense to simply remove this and add it by default in cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the indentation level for a command's definition depends on
the command name length, and this makes the commands dict unpleasant to
the eye. This patch changes it so that a new line begins right after the
left parenthesis starting the command tuple.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This can be used to generate the bash completion script automatically.
In the future it may allow for better command line validation as well.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently the OSes have a special, customized error handling: the OS
object can represent either a valid OS, or an invalid OS. The associated
function, instead of raising other exception or failing, create custom
OS objects representing failed OSes.

While this was good when no other RPC had failure handling, it's
extremely different from how other function in backend.py expect
failures to be signalled.

This patch reworks this completely:
  - the OS object always represents valid OSes (the next patch will
    remove the valid/invalid field and associated constants)
  - the call_os_diagnose returns instead of a list of OS objects, a list
    of (name, path, status, diagnose_msg); the status is then used in
    cmdlib to determine validity and the status and diagnose_msg values
    are used in gnt-os for display
  - call_os_get returns either a valid OS or a RPC remote failure (with
    the error message)
  - the other functions in backend.py now just call backend.OSFromDisk()
    which will return either a valid OS object or raise an exception
  - the bulk of the OSFromDisk was moved to _TryOSFromDisk which returns
    status, value for the functions which don't want an exception raised

The gnt-os list and diagnose commands still work after this patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch enables by default the old 'human-readable' option, but in a
slightly different model.

The option is now called "units" and takes either:
 - 'h' for automatic formatting
 - 'm', 'g' or 't' for mebi/gibi/tebibytes

If 'h' is used, we add a unit suffix, otherwise nothing is added so that
parsing is easy.

The default value of this unit is:
  - 'h' if a separator is not passed
  - 'm' if a separator is passed

Reviewed-by: ultrotter

Reviewed-by: imsnah

Some names were wrong, and similar stuff detected by pylint.
gnt-debug.GenericOpCode is still broken.

Reviewed-by: imsnah

Currently the gnt-* scripts are using a mix of print, logger.ToStd* and
sys.stderr.write. We convert them all to using cli.ToStdout/err. This
way, we can easily change the implementation for all at once.

Reviewed-by: imsnah

This reverts commit 976.

Reviewed-by: ultrotter

[Forward-port of the 1.2 branch patch]

This patch removes all the parameters and options from the output
"gnt-X" (i.e. the subcommand list for command). This is done in order to
uniformize the output, currently only some parameters are shown and they
are not always consistent (e.g. required versus important parameters).

Reviewed-by: ultrotter

Currently, the opcode DiagnoseOS is the only opcode that return a
structure of objects.OS (which is a custom class, and not a simple
python object) and furthermore all the processing of OS validity across
nodes is left to the clients of this opcode.

It would be more logical to have this opcode be similar to list
instances/nodes, in the sense that:
  - it should return a table of results
  - the fields in the table should be selectable

This patch does the above. The possible fields are:
  - name (os name)
  - valid (bool representing validity across all nodes)
  - node_status, which is a complicated structure required for ‘gnt-os
    diagnose’

With this patch, gnt-os list becomes a very simple iteration over the
list of results, filtering out non-valid ones. gnt-os diagnose is still
complicated, but no more than before.

The burnin tool has also been modified to work with the modified
results, and is simpler because of this (it only needs to know if an OS
is valid or not, not the per-node details).

Reviewed-by: imsnah

Modify backend.py so that DiagnoseOS only returns OS objects rather than
InvalidOS errors, and make sure gnt-os understands the new objects. Also delete
the deprecated helper functions from gnt-os.

Reviewed-By: iustinp

Ok, I've been battling with those for a while but it seems
in the end I forgot to get rid of them! :( Doing it explicitely now.

Reviewed-By: iustinp

This patch changes whitespace only. It fixes all the functions in script/gnt-os
by putting a white line at the end of all the function descriptions and
removing it from just before the code.

Reviewed-By: iustinp

Reviewed-By: imsnah

Get rid of the tabulated form and create a simplier indented structure.
Now a valid OS looks something like:

OS: foo [global status: partial valid]
  Node: gsdtest4.euw.corp.google.com, status: valid (path: /usr/local/lib/ganeti/os/foo)
  Node: gsdtest5.euw.corp.google.com, status: valid (path: /usr/local/lib/ganeti/os/foo)
    [hidden] path: /usr/local/lib/ganeti/os2/foo, status: 'ganeti_api_version' file not found (ENOENT)
  Node: gsdtest3.euw.corp.google.com, status: 'ganeti_api_version' file not found (ENOENT) (path: /usr/local/lib/ganeti/os/foo)

Reviewed-By: imsnah

Use both _DiagnoseOSStatus and  _DiagnoseOSPath to calculate the message
associated with the first entry found on a node for an OS. Also call the rename
the relevant variables to _msg rather than just _status to reflect that they
contain a bit more than just the bare status message.

Reviewed-By: imsnah

Make _DiagnoseOSStatus only return a status message for the OS diagnose object
and create a new _DiagnoseOSPath function to get the path out. This will let us
be more flexible in the messages we display to the user.

Reviewed-By: imsnah

The predicate function gets build by a lambda function.
Also add a bit of debugging information to the 'unknown OS diagnose type'.


Reviewed-by: iustinp

Reviewed-by: iustinp

Put some order in gnt-os putting helper functions above and command functions
below. Also add a new _DiagnoseOSName() function and use it to make _GetAllOS
simplier to understand. Change its name to _DiagnoseByOS which makes more clear
what it does.



Reviewed-by: iustinp

To do this we abstract the creation of the all_os dictionary, which was
produced in DiagnoseOS and use it both for listing and diagnosing.


Reviewed-by: iustinp

The new version debugs Hidden OSes as well.


Reviewed-by: iustinp