1. 17 Jun, 2009 1 commit
    • Iustin Pop's avatar
      Fix handling of 'vcpus' in instance list · c1ce76bb
      Iustin Pop authored
      Currently running “gnt-instance list -o+vcpus” fails with a cryptic message:
        Unhandled Ganeti error: vcpus
      This is due to multiple issues:
        - in some corner cases cmdlib.py raises an errors.ParameterError but
          this is not handled by cli.py
        - LUQueryInstances declares ‘vcpu’ as a supported field, but doesn't handle
          it, so instead of failing with unknown parameter, e.g.:
            Failure: prerequisites not met for this operation:
            Unknown output fields selected: vcpuscd
          it raises the ParameteError message
      This patch:
        - adds handling of 'vcpus' to LUQueryInstances
        - adds handling of the ParameterError exception to cli.py
        - changes the 'else: raise errors.ParameterError' in the field handling of
          LUQueryInstance to an assert, since it's a programmer error if we reached
          this step
      With this, a future unhandled parameter will show:
        gnt-instance list -o+vcpus
        Unhandled protocol error while talking to the master daemon:
        Caught exception: Declared but unhandled parameter 'vcpus'
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
  2. 18 May, 2009 1 commit
  3. 27 Feb, 2009 1 commit
    • Iustin Pop's avatar
      Fix some epydoc style issues · 5fcc718f
      Iustin Pop authored
      99% of the epydoc return tags are "@return:", but each of the modified files
      had one "@returns:" line. We fix this for consistency.
      Reviewed-by: imsnah
  4. 13 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Remove two fixed FIXME and convert one to TODO · ea5a5b74
      Guido Trotter authored
      The cli FIXME is not something broken, but rather some better handling
      feature we'd rather have, and the two backend FIXME are done (disks have
      their read only parameter set, and the error is raised and thus reaches
      the master).
      Reviewed-by: iustinp
  5. 10 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Instance parameters: force typing · a5728081
      Guido Trotter authored
      We want all the hv/be parameters to have a known type, rather than a
      random mix of empty string, boolean values, and None, so we declare the
      type of each variable and we enforce/convert it.
      - Add some new constants for enforceable value types
      - Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
        holding not only the valid parameters but also their types
      - Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
        the values from the type dict
      - Convert all the default parameters to a valid type value
      - Create a new ForceDictType utils function, to check/enforce a dict's
        element value types, with relevant unit tests
      - Drop a few custom functions to check/convert the BE param types in
        utils and cli, in favor of ForceDictType
      - Double-check the parameter types using ForceDictType in both scripts
        and LogicalUnits, when possible.
      As a bonus:
      - Remove some old commented-out code in gnt-instance
      - Remove some already fixed FIXME
      - Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
        in SetInstanceParams because the value was checked for validity before
        that transformation was made
      - Fix a bug which prevented initing a cluster and passing hvparams to
        work at all
      - ForceDictType allows an allowed_values for exceptions, which makes us
        able to do the checking even when some values must not be
        converted/typechecked (for example the 'default' string in
      Reviewed-by: iustinp
  6. 05 Feb, 2009 1 commit
    • Iustin Pop's avatar
      Fix some issues for lockless queries · 2e7b8369
      Iustin Pop authored
      This patch converts some more jobs with only queries into cheaper luxi
      queries (no job created), and fixes some fallout from the lockless
      queries changes.
      Reviewed-by: ultrotter
  7. 04 Feb, 2009 1 commit
    • Iustin Pop's avatar
      Implement lockless query operations · ec79568d
      Iustin Pop authored
      This patch adds the framework for, and enables lockless OpQueryInstances. This
      means that instances will be shown in ERROR_up or ERROR_down state, even though
      this is not an error (but just an in-progress job).
      The framework is implemented as follows:
        - the OpQueryInstances, OpQueryNodes and OpQueryExports opcodes take
          an additional “use_locking” flag which will denote whether to lock
          or not; this patch only implements this for LUQueryInstances
        - the luxi query functions take an additional argument use_locking
          which is passed to the master daemon, and then passed to the above
        - cli.py export a new SYNC_OPT command line options which implement
          setting this flag to true
        - except for gnt-instance list, which uses this option, and for
          name-only queries (e.g. QueryNodes(fields=["names"])), all other
          callers are setting this flag to True
        - RAPI also sets the flag to True
      The patch was tested with a continuous (0.2s sleep in-between)
      gnt-instance list during a burnin, and no problems were observed.
      Reviewed-by: ultrotter
  8. 03 Feb, 2009 1 commit
    • Iustin Pop's avatar
      An attempt at fixing some encoding issues · 26f15862
      Iustin Pop authored
      This patch unifies the hardcoded re-encoding attempts into a single
      function in utils.py. This function is used to take either an unicode or
      str object and convert it to a ASCII-only str object which can be safely
      displayed and transmitted.
      We replace then the current manual re-encodings with this function. In
      mcpu we stop re-encoding the hooks output and instead we do it right at
      the hook generation in backend.py.
      This passes on my 'custom' lvs output with non-ASCII chars. But there
      are probably other places we will need to fix.
      Reviewed-by: ultrotter
  9. 27 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Rework the multi-instance gnt commands · 479636a3
      Iustin Pop authored
      This patch changes the multi-instance gnt-* commands (gnt-instance
      start/stop, gnt-node evacuate/failover) such that the individual
      operations are submitted in parallel, ideally improving the speed of the
      The patch does this by abstracting the job set functionality into a new
      class in cli.py, that takes care of the job submit, job poll and error
      Reviewed-by: ultrotter
  10. 12 Jan, 2009 1 commit
  11. 17 Dec, 2008 1 commit
    • Michael Hanselmann's avatar
      Add job queue size limit · f87b405e
      Michael Hanselmann authored
      A job queue with too many jobs can increase memory usage and/or make
      the master daemon slow. The current limit is just an arbitrary number.
      A "soft" limit for automatic job archival is prepared.
      Reviewed-by: iustinp
  12. 11 Dec, 2008 1 commit
    • Iustin Pop's avatar
      Fix epydoc format warnings · c41eea6e
      Iustin Pop authored
      This patch should fix all outstanding epydoc parsing errors; as such, we
      switch epydoc into verbose mode so that any new errors will be visible.
      Reviewed-by: imsnah
  13. 28 Nov, 2008 1 commit
  14. 27 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Small change to job failure output · 0e050889
      Iustin Pop authored
      Currently, job failures are done by raising OpExecError(job result). For
      a one-opcode job that failed, this is very non-intuitive:
      Failure: command execution error:
      [u'Disk size change not possible, use grow-disk']
      This patch changes the output in two ways:
       - for a job which didn't have opcodes which succeeded, it changes it
         Failure: command execution error:
         Disk size change not possible, use grow-disk
       - for a job with partial completion, to:
         Failure: command execution error:
         partial failure (opcode 1): Cannot remove the last disk of an
      The actual error type is not changed. A job that doesn't have at least
      an error-ed opcode is shown as before.
      Reviewed-by: imsnah
  15. 23 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Enable auto-unit formatting in script output · 9fbfbb7b
      Iustin Pop authored
      This patch enables by default the old 'human-readable' option, but in a
      slightly different model.
      The option is now called "units" and takes either:
       - 'h' for automatic formatting
       - 'm', 'g' or 't' for mebi/gibi/tebibytes
      If 'h' is used, we add a unit suffix, otherwise nothing is added so that
      parsing is easy.
      The default value of this unit is:
        - 'h' if a separator is not passed
        - 'm' if a separator is passed
      Reviewed-by: ultrotter
  16. 21 Nov, 2008 2 commits
    • Iustin Pop's avatar
      Make cli.py use FieldSet for matching fields · 00430f8e
      Iustin Pop authored
      This changes cli.py to FieldSet usage so that gnt-instance list will
      format nicely the disk.size/*, and the count of disks/nics.
      Reviewed-by: ultrotter
    • Michael Hanselmann's avatar
      Reuse HTTP client pool for RPC · 4331f6cd
      Michael Hanselmann authored
      ganeti-masterd: Add initialization and shutdown of RPC pool. It needs
      to be shutdown before forking.
      ganeti.cli: Add decorator function to initialize and shutdown RPC pool.
      ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw
      exception when used without proper initialization.
      gnt-cluster, gnt-node: Use decorator function to initialize and shutdown
      RPC pool.
      Reviewed-by: iustinp
  17. 18 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Allow querying of variable number of parameters · 71c1af58
      Iustin Pop authored
      This patch adds support for querying in gnt-instance list of:
        - disk.count
        - nic.count
        - disk.size/$N
        - nic.(ip|mac|bridge)/$N
      The patch also disables the exception raised when the header description
      is missing, that should be fixed in the future (together with better
      handling of variable numeric fields).
      Reviewed-by: imsnah
  18. 23 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Fix gnt-* command with --submit · d8353c3a
      Iustin Pop authored
      Currently we don't catch the JobSubmittedException (although the code to
      format it nicely is there). The patch adds the exception name to the
      catch list and makes --submit work.
      Reviewed-by: imsnah
  19. 20 Oct, 2008 2 commits
    • Iustin Pop's avatar
      Remove the logger.py module · 82d9caef
      Iustin Pop authored
      Since now we use only one function from the logger module
      (SetupLogging), we move it to utils.py (which is already imported by all
      users of this function), and we remove the module.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Convert cli.py to logging · 46fbdd04
      Iustin Pop authored
      We also add two function for printing messages, so that scripts won't
      have to import logger to get these. They are a simple extension over the
      logger ones, as they accept the call style from logging:
       ToStdout("Message: %s", msg)
      (instead of requiring formatting by the client.
      Reviewed-by: imsnah
  20. 15 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Implement the job queue drain flag · 686d7433
      Iustin Pop authored
      We add a (per-node) queue drain flag that blocks new job submission.
      There is not yet an interface to add/remove the flag (will come in next
      Reviewed-by: imsnah
  21. 14 Oct, 2008 2 commits
    • Iustin Pop's avatar
      Add cli function for beparams validation · a604f165
      Iustin Pop authored
      This is used to validate the parameters in both “gnt-instance add” and
      “gnt-backup import”.
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Implement parameter removal in SplitKeyVal · fcd62d84
      Iustin Pop authored
      This patch adds paramter removal in SplitKeyVal, by prefixing a
      value-less key with "-"; this is needed in resetting parameters back to
      cluster defaults, but care must be applied now that None can come from
      the parser.
      Reviewed-by: imsnah
  22. 13 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Add two new options types for CLI usage · a8469393
      Iustin Pop authored
      For the new 2.0-style command line options, we need to parse strings of
      the type:
      This patch adds two new option builders for these two, which return
      (ident, {key=val,}) and {key=val,} for the above two formats. It also
      handles specially constructs of type “key” (val is set to True) and
      “no_key” (val is set to False, and the ‘no_’ prefix is stripped).
      Reviewed-by: imsnah
  23. 06 Oct, 2008 3 commits
    • Iustin Pop's avatar
      Implement job auto-archiving · 07cd723a
      Iustin Pop authored
      This patch adds a new luxi call that implements auto-archiving of jobs
      older than a certain age (or -1 for all completed jobs), and the gnt-job
      command that makes use of this (with 'all' for -1).
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Add a simple timespec parsing function · 2241e2b9
      Iustin Pop authored
      This function will be used for auto-archiving jobs via the command line.
      The function is pretty simple, we only support up to weeks since months
      and higher are not 'precise' entities, and dealing with them would
      require us to start using calendar functions.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Fix gnt-job list with empty timestamps · e0ec0ff6
      Iustin Pop authored
      In case the job object doesn't have a timestamp (which is a separate
      issue), the listing should not break. We fix this by changing the
      FormatTimstamp function itself to return '?' in case the timestamp
      doesn't look good (note that it still can break if non-integers are
      returned, but this is unlikely).
      Reviewed-by: imsnah
  24. 01 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Change the results from cli.PollJob · 53c04d04
      Iustin Pop authored
      Curently PollJob accepts a generic job, but will return (history
      artifact) only the first opcode result. This is wrong, as it doesn't
      allow polling of a job with multiple results.
      Its only caller (for now) is also changed, so no functional changes
      should happen.
      Reviewed-by: ultrotter, amishchenko
  25. 30 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Abstract the timestamp formatting into cli.py · 3386e7a9
      Iustin Pop authored
      Currently we format the timestamp inside the gnt-job info function. We
      will need this more times in the future, so move it to cli.py as a
      separate, exported function.
      Reviewed-by: imsnah
  26. 10 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Rewrite the 'only submit job' handling in scripts · e9d741b6
      Iustin Pop authored
      The "sys.exit(0)" was not nice as you couldn't differentiate it from
      other exit codes. We change this to a specially defined exception for
      this, so that multi-opcode commands can handle this nicely.
      Reviewed-by: imsnah
  27. 28 Aug, 2008 1 commit
  28. 27 Aug, 2008 1 commit
    • Michael Hanselmann's avatar
      Make sure that client programs get all messages · 6c5a7090
      Michael Hanselmann authored
      This is a large patch, but I can't figure out how to split it without
      breaking stuff. The old way of getting messages by always getting the
      last one didn't bring all messages to the client if they were added
      too fast, thereby making commands like “gnt-cluster verify” less than
      useful. These changes now introduce some sort a serial number per
      log entry to keep track what message a client already received. They
      also remove the log lock per opcode to make reading log entries thread
      Reviewed-by: ultrotter
  29. 11 Aug, 2008 2 commits
  30. 07 Aug, 2008 1 commit
  31. 04 Aug, 2008 1 commit
    • Iustin Pop's avatar
      Implement job submission for scripts · 94428652
      Iustin Pop authored
      This patch adds the infrastructure for executing a job in background,
      instead of foreground, via a new “--submit” option. The behaviour is
      that the job ID is printed and the script will immediately exit.
      The patch also converts gnt-node list to this model (yes, this will be a
      query in the future).
      Reviewed-by: imsnah
  32. 31 Jul, 2008 1 commit
    • Iustin Pop's avatar
      Split cli.SubmitOpCode in two parts · 0a1e74d9
      Iustin Pop authored
      The current SubmitOpCode function is not flexible enough to be used for
      submitters that don't want to wait for the job finish.
      The patch splits this in two, a SendJob function and a PollJob one, and
      the old SubmitOpCode becomes a wrapper. Note that the new SendJob takes
      a list of opcodes (and not a single opcode anymore).
      Reviewed-by: imsnah
  33. 30 Jul, 2008 2 commits
    • Iustin Pop's avatar
      Make gnt-* commands fail nicely on non-masters · b33e986b
      Iustin Pop authored
      This patch adds a check that we are on the master after failing to
      connect to the socket, and log nicely the master name.
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Fix pylint-detected issues · 38206f3c
      Iustin Pop authored
      This is mostly:
        - whitespace fix (space at EOL in some files, not all, broken
          indentation, etc)
        - variable names overriding others (one is a real bug in there)
        - too-long-lines
        - cleanup of most unused imports (not all)
      Reviewed-by: ultrotter