1. 05 May, 2009 1 commit
    • Iustin Pop's avatar
      Fix compatibility with DRBD 8.3 · 01e2ce3a
      Iustin Pop authored
      DRBD 8.3 changes two more things compared to 8.2:
        - /proc/drbd format changed in multiple ways; the part we're
          interested is the ‘st:’ to ‘ro:‘ change (in the changelog named as
          “Renamed 'state' to 'role'”
        - “drbdsetup /dev/drbdN show” changed the ‘device’ stanza from:
            device "/dev/drbd0";
            device                  minor 0;
      This patch fixes these both and adds data files and unittests for DRBD
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
  2. 16 Feb, 2009 1 commit
    • Iustin Pop's avatar
      Convert IOErrors for /proc/drbd into our errors · f6eaed12
      Iustin Pop authored
      If /proc/drbd can't be opened, this raises an IOError, but all the
      error-handling behaviour in backend treats only BlockDeviceErrors. This
      creates a plain failure in cluster verify and in other RPC calls.
      This patch simply converts EnvironmentErrors into BlockDeviceErrors, and
      also changes the RPC result for NV_DRBDLIST and its handling to be able
      to show the error. The other RPC calls work by default now, due the
      existing error handling.
      Reviewed-by: ultrotter
  3. 12 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Apply the right permissions to /etc/hosts · 9b977740
      Guido Trotter authored
      In the current Ganeti version when modifying /etc/hosts we mistakenly
      give it the permissions of the temporary file we create to define its
      content, which is by default 0600. This breaks most non-root
      applications, and thus must be corrected. This patch forces the mode to
      be 0644 (but we might decide to just use the mode of the previous
      /etc/hosts, if we want to be more polite against any eventual
      administrative choice). We also add a new assertFileMode() method for
      unit tests and actually check in the SetEtcHostsEntry and
      RemoveEtcHostsEntry tests that the mode is correct, to be sure not to
      reintroduce this bug again. Also, a FIXME is added in the original
      functions stating that it would be nice to use WriteFile+fn() rather
      than reimplementing its functionality again.
      Reviewed-by: iustinp
  4. 10 Feb, 2009 1 commit
    • Guido Trotter's avatar
      Instance parameters: force typing · a5728081
      Guido Trotter authored
      We want all the hv/be parameters to have a known type, rather than a
      random mix of empty string, boolean values, and None, so we declare the
      type of each variable and we enforce/convert it.
      - Add some new constants for enforceable value types
      - Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
        holding not only the valid parameters but also their types
      - Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
        the values from the type dict
      - Convert all the default parameters to a valid type value
      - Create a new ForceDictType utils function, to check/enforce a dict's
        element value types, with relevant unit tests
      - Drop a few custom functions to check/convert the BE param types in
        utils and cli, in favor of ForceDictType
      - Double-check the parameter types using ForceDictType in both scripts
        and LogicalUnits, when possible.
      As a bonus:
      - Remove some old commented-out code in gnt-instance
      - Remove some already fixed FIXME
      - Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
        in SetInstanceParams because the value was checked for validity before
        that transformation was made
      - Fix a bug which prevented initing a cluster and passing hvparams to
        work at all
      - ForceDictType allows an allowed_values for exceptions, which makes us
        able to do the checking even when some values must not be
        converted/typechecked (for example the 'default' string in
      Reviewed-by: iustinp
  5. 03 Feb, 2009 1 commit
    • Iustin Pop's avatar
      Fix unittest encoding breakage · 7b80424f
      Iustin Pop authored
      Due to the fact that we sanitize now the output from environment
      scripts, the unittest needs to be adjusted. My bad for not checking it.
      Reviewed-by: imsnah
  6. 29 Jan, 2009 3 commits
  7. 23 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Introduce more configuration consistency checks · 4a89c54a
      Iustin Pop authored
      This patch enhances the duplicate DRBD minors checks (currently just a
      few) and adds automatic checks of configuration consistency at
      configuration file writing time.
      In order to do so and show meaningful error messages, the
      _UnlockedComputeDRBDMap function is changed to not raise errors in case
      of duplicates, but instead return both the minors map and the duplicate
      list, and its callers now raise the error. This allows the VerifyConfig
      function to return a complete list of duplicates.
      The new checks required some small updates to the unittests for the
      config module.
      Reviewed-by: ultrotter
  8. 20 Jan, 2009 3 commits
    • Iustin Pop's avatar
      DRBD: check for in-use minor during Create · 767d52d3
      Iustin Pop authored
      In order to prevent errors with old, in-use DRBD minors, we check and
      abort at create time if our minor is already in use. For this we need to
      also modify DRBD8Status to be able to parse cs:Unconfigured devices.
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Add a TailFile function · f65f63ef
      Iustin Pop authored
      This patch adds a tail file function, to be used for parsing and returning in
      the job log OS installation failures.
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Unify some unittest functions · 51596eb2
      Iustin Pop authored
      This patch adds unified temporary file handling to the
      testutils.GanetiTestCase class, which adds easy creation and automated
      cleanup of temporary files.
      The patch allows a simpler handling in a couple of test cases but
      requires all child classes to call the parent setUp and tearDown
      Reviewed-by: ultrotter
  9. 19 Dec, 2008 1 commit
  10. 18 Dec, 2008 1 commit
  11. 17 Dec, 2008 1 commit
  12. 04 Dec, 2008 1 commit
  13. 03 Dec, 2008 1 commit
  14. 02 Dec, 2008 1 commit
  15. 28 Nov, 2008 1 commit
  16. 23 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Enable auto-unit formatting in script output · 9fbfbb7b
      Iustin Pop authored
      This patch enables by default the old 'human-readable' option, but in a
      slightly different model.
      The option is now called "units" and takes either:
       - 'h' for automatic formatting
       - 'm', 'g' or 't' for mebi/gibi/tebibytes
      If 'h' is used, we add a unit suffix, otherwise nothing is added so that
      parsing is easy.
      The default value of this unit is:
        - 'h' if a separator is not passed
        - 'm' if a separator is passed
      Reviewed-by: ultrotter
  17. 21 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Move FieldSet class to utils.py · a2d2e1a7
      Iustin Pop authored
      Since we can use the FieldSet class in cli.py to nicely format disk
      sizes and such, we move it to utils.py and also move its associated
      unittest. I didn't remove the cmdlib.py unittest file as that's not the
      good direction :)
      Reviewed-by: ultrotter
  18. 18 Nov, 2008 3 commits
    • Iustin Pop's avatar
      Skip another testWaitingXblocksY · a143be68
      Iustin Pop authored
      The waiting sharer blocks exclusive is another not-possible to test
      right now.
      Sorry for missing it the first time; there are no other testWaiting...
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Add a FieldSet class for variable parameter sets · 6de7c41d
      Iustin Pop authored
      This patch adds a _FieldSet class that can be used for the new variable
      parameter sets: e.g. the sda_size will change to disk/0.size (or
      similar) and we need to both check validity and extract the index of the
      parameter easily.
      The patch also sorts the unittest list in Makefile.am.
      Reviewed-by: ultrotter
    • Iustin Pop's avatar
      Skip the waitingExclusiveBlockSharer test · e6416152
      Iustin Pop authored
      Until we have a way to properly test this, we'll skip the test.
      Reviewed-by: imsnah
  19. 12 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Convert the locking unittests to repetition-test · 4607c978
      Iustin Pop authored
      Currently the locking tests are using timeouts to ensure an event will
      'never happen'. However, this is suboptimal.
      The patch converts all of these to instead: not wait, but sequence the
      operations logically and expect that they execute as such. In case of
      not so, they will timeout with a big (60s) timeout.
      The 'never happen' is tested by multiple repetitions of the same test -
      this is not perfect, but again 'X will never happen' is not actually
      This patch reduces the runtime of the tests from ~5.4 to ~0.8 seconds
      (with 8 repetitions of each test thas has 'never happen' situations).
      Reviewed-by: imsnah
  20. 11 Nov, 2008 3 commits
    • Iustin Pop's avatar
      Remove zombie special case from IsProcessAlive · 44bf25ff
      Iustin Pop authored
      Based on the discussion on the mailing list, we remove the special
      casing of zombie processes.
      Note we don't use kill -0 since that has a different meaning than "check
      process is alive", so this function is still linux-specific.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Add cleanup of processes to utils.KillProcess · ff5251bc
      Iustin Pop authored
      In case the process we want to signal is our own process, and the signal
      we send is a deadly one, we should also cleanup after the process.
      This patch adds a new parameter waitpid to this function that does this,
      and changes its unittest to pass this new parameter.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Implement working directory for utils.RunCmd · 8797df43
      Iustin Pop authored
      In order to replace usage of the shell just for 'cd %s; ...' we need to
      be able to specify the working directory for childs.
      The patch also changes the default working directory (when not
      specified) to "/", as opposed to the current dir; since the daemos run
      with cwd=/ anyway, this should not change things.
      Reviewed-by: imsnah
  21. 24 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Modify utils.RunCmd to write output to file · 36117c2b
      Iustin Pop authored
      Currently we launch processes via the shell in a few places only to
      redirect standard output and error to a log file ("&> $file"). It is
      better to do such redirection from within RunCmd itself.
      This patch splits RunCmd in two parts, the setup and the execution part,
      the latter being implemented in two different functions depending on
      whether we write to a file or not.
      We also update the unittests with this new case.
      Reviewed-by: imsnah
  22. 21 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Move some LU logging to use proc.Log* · 86d9d3bb
      Iustin Pop authored
      We change some messages which are obviously intended for the user to use
      the proc.Log* functions instead of (only) the logging ones.
      We also fix some wrong uses of feedback_fn.
      Reviewed-by: imsnah
  23. 20 Oct, 2008 4 commits
    • Iustin Pop's avatar
      Generalize the reading of test file data · 149a5439
      Iustin Pop authored
      Currently we have two methods in ganeti.bdev_unittest.py of computing
      the test data file name - and, of course, they don't give the same
      The patch moves the functions to compute the test file name and reading
      of its contents to the GanetiTestCase class in testutils, which allows
      running the tests from the command line as well.
      We also change assertFileContent to use utils.ReadFile.
      Reviewed-by: imsnah
    • Alexander Schreiber's avatar
      Set default hypervisor at cluster init · 02691904
      Alexander Schreiber authored
      During cluster init, set the default hypervisor to be used for instances.
      Ensure that the default hypervisor belongs to the set enabled hypervisors
      for this cluster. Also fix a small bug with setting the default enabled
      hypervisor list.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Convert cli.py to logging · 46fbdd04
      Iustin Pop authored
      We also add two function for printing messages, so that scripts won't
      have to import logger to get these. They are a simple extension over the
      logger ones, as they accept the call style from logging:
       ToStdout("Message: %s", msg)
      (instead of requiring formatting by the client.
      Reviewed-by: imsnah
    • Michael Hanselmann's avatar
      Remove old HTTP server code · fa10bdc5
      Michael Hanselmann authored
      All users of this code have been migrated to the new and shiny HttpServer
      It also fixes a typo in the ApacheLogfile unittests. It has not yet been
      decided whether we should keep ApacheLogfile or not, hence leaving it in.
      Reviewed-by: ultrotter
  24. 17 Oct, 2008 1 commit
  25. 16 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Really fix with the zombie test (hopefully) · 09352fa4
      Iustin Pop authored
      So, instead of any timeouts for synchronization, we move to actual
      events. We still have one (huge) timeout for pathological cases (just to
      cleanup), but we don't sleep at all in normal usage.
      The patch moves the zombie/cleaned up child creation out from setUp(),
      and to on-demand by the functions. We remove all signal handler usage,
      instead we rely on the fact that when a process is cleaned up, its FDs
      are closed, so the read end of a pipe will get an event.
      Reviewed-by: imsnah
  26. 14 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Implement parameter removal in SplitKeyVal · fcd62d84
      Iustin Pop authored
      This patch adds paramter removal in SplitKeyVal, by prefixing a
      value-less key with "-"; this is needed in resetting parameters back to
      cluster defaults, but care must be applied now that None can come from
      the parser.
      Reviewed-by: imsnah
  27. 13 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Add two new options types for CLI usage · a8469393
      Iustin Pop authored
      For the new 2.0-style command line options, we need to parse strings of
      the type:
      This patch adds two new option builders for these two, which return
      (ident, {key=val,}) and {key=val,} for the above two formats. It also
      handles specially constructs of type “key” (val is set to True) and
      “no_key” (val is set to False, and the ‘no_’ prefix is stripped).
      Reviewed-by: imsnah
  28. 12 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Abstract checking own address into a function · caad16e2
      Iustin Pop authored
      Currently, we check if we have a given ip address (i.e. it's alive on
      one of our interfaces) but manually calling TcpPing(source=localhost).
      This works, but having it spread all over the code makes it hard to
      change the implementation.
      The patch abstracts this into a separate utils.OwnIpAddress(addr)
      function. We add a rpc call for it, which we use instead of the
      (single-use of) call_node_tcp_ping. We leave node_tcp_ping in, as seems
      useful and eventually it should be removed in a separate patch.
      Reviewed-by: imsnah
  29. 10 Oct, 2008 1 commit
    • Iustin Pop's avatar
      Convert rpc module to RpcRunner · 72737a7f
      Iustin Pop authored
      This big patch changes the call model used in internode-rpc from
      standalong function calls in the rpc module to via a RpcRunner class,
      that holds all the methods. This can be used in the future to enable
      smarter processing in the RPC layer itself (some quick examples are not
      setting the DiskID from cmdlib code, but only once in each rpc call,
      There are a few RPC calls that are made outside of the LU code, and
      these calls are left as staticmethods, so they can be used without a
      class instance (which requires a ConfigWriter instance).
      Reviewed-by: imsnah