1. 28 Dec, 2009 1 commit
  2. 14 Dec, 2009 1 commit
  3. 07 Jul, 2009 2 commits
  4. 19 May, 2009 2 commits
    • Iustin Pop's avatar
      Fix the SafeEncoding behaviour · d392fa34
      Iustin Pop authored
      
      
      Currently we have bad behaviour in SafeEncode:
        - binary strings are actually not handled correctly (ahem)
        - the encoding is not stable, due to use of string_escape
      
      For this reason, we replace the use of string_escape with part of the
      code of string escape (PyString_Repr in Objects/stringobject.c); we
      don't escape backslashes or single quotes, since that is that makes it
      nonstable. Furthermore, we only use the encode('ascii', ...) for unicode
      inputs.
      
      The patch also adds unittests for the function that test basic
      behaviour.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      d392fa34
    • Iustin Pop's avatar
      Move more hypervisor strings into constants · 835528af
      Iustin Pop authored
      
      
      This patch adds constants for the mouse and boot order strings; while
      there are still some issues remaining, we're trying to cleanup hardcoded
      strings from the hypervisors.
      
      Since the formatting of frozensets is currently wrong, we also add an
      utility function for this and change all the error messages to use it.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      835528af
  5. 05 May, 2009 3 commits
  6. 04 May, 2009 1 commit
  7. 02 Mar, 2009 1 commit
  8. 27 Feb, 2009 2 commits
    • Guido Trotter's avatar
      Create runtime dir in bootstrap · 9dae41ad
      Guido Trotter authored
      Some hypervisors (KVM) need RUN_GANETI_DIR to exist even at cluster init
      time. This patch creates it in InitCluster just before hv parameter
      checking. Since the code to make list of directories is already repeated
      twice in the code, and this would be the third time, we abstract it into
      an utils.EnsureDirs function and we call that one from ganti-noded,
      ganeti-masterd and bootstrap.
      
      Reviewed-by: iustinp
      9dae41ad
    • Iustin Pop's avatar
      Fix some epydoc style issues · 5fcc718f
      Iustin Pop authored
      99% of the epydoc return tags are "@return:", but each of the modified files
      had one "@returns:" line. We fix this for consistency.
      
      Reviewed-by: imsnah
      5fcc718f
  9. 26 Feb, 2009 1 commit
  10. 12 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Apply the right permissions to /etc/hosts · 9b977740
      Guido Trotter authored
      In the current Ganeti version when modifying /etc/hosts we mistakenly
      give it the permissions of the temporary file we create to define its
      content, which is by default 0600. This breaks most non-root
      applications, and thus must be corrected. This patch forces the mode to
      be 0644 (but we might decide to just use the mode of the previous
      /etc/hosts, if we want to be more polite against any eventual
      administrative choice). We also add a new assertFileMode() method for
      unit tests and actually check in the SetEtcHostsEntry and
      RemoveEtcHostsEntry tests that the mode is correct, to be sure not to
      reintroduce this bug again. Also, a FIXME is added in the original
      functions stating that it would be nice to use WriteFile+fn() rather
      than reimplementing its functionality again.
      
      Reviewed-by: iustinp
      9b977740
  11. 10 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Instance parameters: force typing · a5728081
      Guido Trotter authored
      We want all the hv/be parameters to have a known type, rather than a
      random mix of empty string, boolean values, and None, so we declare the
      type of each variable and we enforce/convert it.
      
      - Add some new constants for enforceable value types
      - Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
        holding not only the valid parameters but also their types
      - Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
        the values from the type dict
      - Convert all the default parameters to a valid type value
      - Create a new ForceDictType utils function, to check/enforce a dict's
        element value types, with relevant unit tests
      - Drop a few custom functions to check/convert the BE param types in
        utils and cli, in favor of ForceDictType
      - Double-check the parameter types using ForceDictType in both scripts
        and LogicalUnits, when possible.
      
      As a bonus:
      - Remove some old commented-out code in gnt-instance
      - Remove some already fixed FIXME
      - Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
        in SetInstanceParams because the value was checked for validity before
        that transformation was made
      - Fix a bug which prevented initing a cluster and passing hvparams to
        work at all
      - ForceDictType allows an allowed_values for exceptions, which makes us
        able to do the checking even when some values must not be
        converted/typechecked (for example the 'default' string in
        SetInstanceParameters)
      
      Reviewed-by: iustinp
      a5728081
  12. 03 Feb, 2009 2 commits
    • Guido Trotter's avatar
      Correct a typo in ReadPidFile's docstring · 1de62f37
      Guido Trotter authored
      Reviewed-by: iustinp
      1de62f37
    • Iustin Pop's avatar
      An attempt at fixing some encoding issues · 26f15862
      Iustin Pop authored
      This patch unifies the hardcoded re-encoding attempts into a single
      function in utils.py. This function is used to take either an unicode or
      str object and convert it to a ASCII-only str object which can be safely
      displayed and transmitted.
      
      We replace then the current manual re-encodings with this function. In
      mcpu we stop re-encoding the hooks output and instead we do it right at
      the hook generation in backend.py.
      
      This passes on my 'custom' lvs output with non-ASCII chars. But there
      are probably other places we will need to fix.
      
      Reviewed-by: ultrotter
      26f15862
  13. 23 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Fix batcher for 2.0-style disks and nics · 9939547b
      Iustin Pop authored
      This patch fixes the gnt-instance batch-create command, and in doing so
      also slightly changes two other functions:
        - we change utils.ParseUnit so that it accepts integer values also
          (both ParseUnit(5) and ParseUnit("5") return the same value)
        - a bridge 'None' in LUCreateInstance will be converted to the default
          bridge; currently only missing bridges will be accepted to mean the
          default one
      
      The main changes to batcher were the change to variable number of disks
      and NICs.
      
      The patch also adds a batcher-instances.json example file copied from
      the 1.2 branch and properly modified.
      
      Reviewed-by: imsnah, killerfoxi
      9939547b
  14. 20 Jan, 2009 3 commits
    • Iustin Pop's avatar
      Update the logging output of job processing · d21d09d6
      Iustin Pop authored
      (this is related to the master daemon log)
      
      Currently it's not possible to follow (in the non-debug runs) the
      logical execution thread of jobs. This is due to the fact that we don't
      log the thread name (so we lose the association of log messages to jobs)
      and we don't log the start/stop of job and opcode execution.
      
      This patch adds a new parameter to utils.SetupLogging that enables
      thread name logging, and promotes some log entries from debug to info.
      With this applied, it's easier to understand which log messages relate
      to which jobs/opcodes.
      
      The patch also moves the "INFO client closed connection" entry to debug
      level, since it's not a very informative log entry.
      
      Reviewed-by: ultrotter
      d21d09d6
    • Michael Hanselmann's avatar
      .gitignore: Don't exclude whole /autotools/ dir, but only files · ae59efea
      Michael Hanselmann authored
      This way newly added files will be not be excluded by default. Fixes
      also a small whitespace error in utils.py.
      
      Reviewed-by: iustinp
      ae59efea
    • Iustin Pop's avatar
      Add a TailFile function · f65f63ef
      Iustin Pop authored
      This patch adds a tail file function, to be used for parsing and returning in
      the job log OS installation failures.
      
      Reviewed-by: ultrotter
      f65f63ef
  15. 09 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Rework the daemonization sequence · 7d88772a
      Iustin Pop authored
      The current fork+close fds sequence has deficiencies which are hard to
      work around:
        - logging can start logging before we fork (e.g. if we need to emit
          messages related to master checking), and thus use FDs which we
          can't track nicely
        - the queue locks the queue file, and again this fd needs to be kept
          open which is hard from the main loop (and this error is currently
          hidden by the fact that we don't log it)
      
      Given the above, it's much simpler, in case we will fork later, to close
      file descriptors right at the beginning of the program, and in Daemonize
      only close/reopen the stdin/out/err fds.
      
      In addition, we also close() the handlers we remove in SetupLogging so
      that the cleanup is more thorough.
      
      Reviewed-by: imsnah
      7d88772a
  16. 18 Dec, 2008 1 commit
  17. 17 Dec, 2008 1 commit
    • Michael Hanselmann's avatar
      utils.KillProcess: Use waitpid() to wait for child processes · 7167159a
      Michael Hanselmann authored
      Sometimes the proc filesystem doesn't reflect the current status of
      a process. By calling waitpid(), we make sure to get the current
      information, at least for child processes. The timeout is still
      kept for child processes to make sure the proc filesystem is updated.
      
      Reviewed-by: iustinp
      7167159a
  18. 16 Dec, 2008 1 commit
  19. 14 Dec, 2008 2 commits
  20. 03 Dec, 2008 1 commit
  21. 02 Dec, 2008 1 commit
    • Guido Trotter's avatar
      Add utils.CheckBEParams · 42ce2e13
      Guido Trotter authored
      This function will be used in LUCreateInstance, LUSetInstanceParams,
      LUSetClusterParams and InitCluster to check the backend parameters
      validity and convert the relevant values to integer, without duplicating
      code. It lives in utils as bootstrap.py is calling it too.
      
      Reviewed-by: imsnah
      42ce2e13
  22. 23 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Enable auto-unit formatting in script output · 9fbfbb7b
      Iustin Pop authored
      This patch enables by default the old 'human-readable' option, but in a
      slightly different model.
      
      The option is now called "units" and takes either:
       - 'h' for automatic formatting
       - 'm', 'g' or 't' for mebi/gibi/tebibytes
      
      If 'h' is used, we add a unit suffix, otherwise nothing is added so that
      parsing is easy.
      
      The default value of this unit is:
        - 'h' if a separator is not passed
        - 'm' if a separator is passed
      
      Reviewed-by: ultrotter
      9fbfbb7b
  23. 21 Nov, 2008 2 commits
    • Iustin Pop's avatar
      Move FieldSet class to utils.py · a2d2e1a7
      Iustin Pop authored
      Since we can use the FieldSet class in cli.py to nicely format disk
      sizes and such, we move it to utils.py and also move its associated
      unittest. I didn't remove the cmdlib.py unittest file as that's not the
      good direction :)
      
      Reviewed-by: ultrotter
      a2d2e1a7
    • Michael Hanselmann's avatar
      Get rid of node daemon password · ec17d09c
      Michael Hanselmann authored
      With the new SSL client certificate stuff it's no longer needed.
      
      Reviewed-by: iustinp
      ec17d09c
  24. 12 Nov, 2008 2 commits
    • Michael Hanselmann's avatar
      utils.SetupLogging: Remove previously setup handlers · 6346a9e5
      Michael Hanselmann authored
      If a logging function is called before the logging module is setup
      using utils.SetupLogging, it calls logging.basicConfig, which adds
      a StreamHandler, on its own. If we leave it in, all log output will
      be printed twice on stderr. This patch makes sure to remove all
      handlers before adding our own.
      
      Reviewed-by: iustinp
      6346a9e5
    • Iustin Pop's avatar
      Fix utils.KillProcess · e1bd0072
      Iustin Pop authored
      Rev 1978 introduced a breakage on the SIGKILL finall signal to the
      process, due to mistyped variable.
      
      Reviewed-by: francis.perron
      e1bd0072
  25. 11 Nov, 2008 3 commits
    • Iustin Pop's avatar
      Remove zombie special case from IsProcessAlive · 44bf25ff
      Iustin Pop authored
      Based on the discussion on the mailing list, we remove the special
      casing of zombie processes.
      
      Note we don't use kill -0 since that has a different meaning than "check
      process is alive", so this function is still linux-specific.
      
      Reviewed-by: imsnah
      44bf25ff
    • Iustin Pop's avatar
      Add cleanup of processes to utils.KillProcess · ff5251bc
      Iustin Pop authored
      In case the process we want to signal is our own process, and the signal
      we send is a deadly one, we should also cleanup after the process.
      
      This patch adds a new parameter waitpid to this function that does this,
      and changes its unittest to pass this new parameter.
      
      Reviewed-by: imsnah
      ff5251bc
    • Iustin Pop's avatar
      Implement working directory for utils.RunCmd · 8797df43
      Iustin Pop authored
      In order to replace usage of the shell just for 'cd %s; ...' we need to
      be able to specify the working directory for childs.
      
      The patch also changes the default working directory (when not
      specified) to "/", as opposed to the current dir; since the daemos run
      with cwd=/ anyway, this should not change things.
      
      Reviewed-by: imsnah
      8797df43
  26. 27 Oct, 2008 1 commit
  27. 24 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Modify utils.RunCmd to write output to file · 36117c2b
      Iustin Pop authored
      Currently we launch processes via the shell in a few places only to
      redirect standard output and error to a log file ("&> $file"). It is
      better to do such redirection from within RunCmd itself.
      
      This patch splits RunCmd in two parts, the setup and the execution part,
      the latter being implemented in two different functions depending on
      whether we write to a file or not.
      
      We also update the unittests with this new case.
      
      Reviewed-by: imsnah
      36117c2b