1. 03 Sep, 2012 2 commits
    • Iustin Pop's avatar
      Fix warnings/errors with newer pylint · 8ad0da1e
      Iustin Pop authored
      
      
      To help developing Ganeti on newer distributions, let's try to fix
      pylint warnings/errors. I'm using pylint from current Debian wheezy:
      pylint 0.25.1, astng 0.23.1, common 0.58.0, and we have 3 things that
      needs fixing.
      
      First, a really wide "except", with the silencing in the wrong
      place. I'm not sure why this doesn't have "except Exception", so let's
      add it. However, pylint still complains about "Catching too general
      exception", even though we do want to catch both system and our
      exception, so let's add a silence for W0703. It's true that we
      shouldn't catch KeyboardInterrupt and friends, but that should be
      cleaned up on the master branch.
      
      Second, pylint complains about "redefining name builtin tuple",
      because we do some pattern matching in the except blocks in
      netutils. This seems to be a false positive, but let's clean the code
      around this.
      
      And finally, type inference again goes bad, so let's silence E1103
      with its "boolean doesn't have 'get' method".
      
      After this, I can run "make lint", and by extension "make
      commit-check" on Debian Wheezy, yay! We might be able to bump our
      required pylint versions to something not ancient…
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      8ad0da1e
    • Iustin Pop's avatar
      Fix decorator uses which crash newer pylint · fc3f75dd
      Iustin Pop authored
      
      
      Pylint version:
      
        pylint 0.25.1,
        astng 0.23.1, common 0.58.0
      
      crashes when passing the fully-qualified decorator name with:
      
        File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 161, in visit_function
          if not redefined_by_decorator(node):
        File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 116, in redefined_by_decorator
          decorator.expr.name == node.name):
      AttributeError: 'Getattr' object has no attribute 'name'
      
      I found out that simply using a shortened name will 'fix' this issue,
      so let's do this to allow running newer pylint versions.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      fc3f75dd
  2. 19 Jul, 2012 1 commit
  3. 11 Jul, 2012 1 commit
  4. 05 Jun, 2012 2 commits
    • Iustin Pop's avatar
      QA: add toggle for fewer common instance tests · b498540e
      Iustin Pop authored
      
      
      Currently, the "common" instance tests (shutdown, rapi stopped
      console, stopped modify, startup, rapi shutdown, rapi startup, list,
      info, modify, rapi modify , console, rapi console, reinstall, rename,
      rapi rename, rename and back, rapi rename and back, grow disk, reboot,
      tags, cluster verify, rapi test instance, node list, job list) are run
      4 times:
      
      - one time for plain instance created via RAPI, using client
      - one time for plain instance created via RAPI, without client
      - one time for plain instance created via gnt-instance
      - one time for DRBD instance created via gnt-instance
      
      This makes the QA long, and is over-doing it (for non-full QAs): an
      instance created via RAPI (either client) and via gnt-instance should
      have the same parameters, but our duplicate tests do not ensure that,
      just that instances behave OK.
      
      This patch adds a toggle so that we can skip the common tests for
      RAPI-created instances; the creation/deletion is still performed, but
      all the other operations are not. This reduces the time of a "quick"
      QA by ~20% (1h:15m → 1h).
      
      The common tests will still be run (unconditionally) for
      gnt-instance-created instances.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      b498540e
    • Iustin Pop's avatar
      QA: stop logging SSH arguments for each invocation · 710bc88c
      Iustin Pop authored
      
      
      Currently, this is how the QA log looks like:
      
      Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.DMzkuH -oControlMaster=no node14 exit
      Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes node8 exit
      Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes node18 exit
      
      And these arguments are repeated over and over. This patch proposes to
      log once the arguments, at the beginning of the QA (so that the SSH
      commands can be reproduced, if needed) and then drop them from the
      log. The new output looks like:
      
      SSH command for primary node: ssh -oEscapeChar=none -oBatchMode=yes -lroot -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.24lgrK -oControlMaster=no node14
      SSH command for other nodes: ssh -oEscapeChar=none -oBatchMode=yes -lroot -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes NODE
      (the above are the informational messages about parameters, then)
      Command: ssh node14 exit
      Command: ssh node8 exit
      Command: ssh node18 exit
      
      This makes the QA log much more readable, by dropping unneeded clutter
      (look how long the original lines were), and hopefully also a bit
      smaller (current QA log is > 100MB of text output).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarBernardo Dal Seno <bdalseno@google.com>
      710bc88c
  5. 14 May, 2012 1 commit
  6. 10 May, 2012 2 commits
  7. 04 May, 2012 1 commit
  8. 26 Apr, 2012 1 commit
  9. 19 Apr, 2012 1 commit
  10. 30 Mar, 2012 1 commit
  11. 31 Jan, 2012 1 commit
  12. 26 Jan, 2012 1 commit
  13. 17 Jan, 2012 1 commit
    • Iustin Pop's avatar
      Rework instance rename QA tests · 4c1a464b
      Iustin Pop authored
      
      
      After looking again at the QA speeds, it turned out that instance
      rename was run 12 times via the command line and 12 times via RAPI;
      outside of start/stop which we need for maintenance during QA, this is
      the most used test.
      
      This patch does a few changes with regard to instance rename:
      
      - it changes rename functions to rename the instance back to the
        original name
      - which allows us not to run said functions twice, A→B and then B→A
      
      This brings down the number of rename tests from 12 to 8 (for both
      cmdline and RAPI).
      
      Furthermore, the CLI rename test was doing too many actual instance
      renames (at Ganeti level), so we rework it to be simpler (brings down
      the average duration from ~27s to ~21s), and we reorganise the run of
      various tests such that we do fewer instances start/stop tests (10
      instead of 14).
      
      Overall we only gain about 3 minutes (~7%), but still it's a speedup.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      4c1a464b
  14. 05 Jan, 2012 1 commit
  15. 04 Nov, 2011 1 commit
  16. 17 Oct, 2011 1 commit
  17. 22 Sep, 2011 1 commit
  18. 30 Aug, 2011 2 commits
  19. 25 Jul, 2011 1 commit
  20. 21 Jul, 2011 1 commit
  21. 16 May, 2011 1 commit
  22. 21 Apr, 2011 2 commits
  23. 14 Apr, 2011 2 commits
  24. 15 Mar, 2011 1 commit
  25. 08 Mar, 2011 2 commits
  26. 28 Feb, 2011 1 commit
  27. 24 Feb, 2011 1 commit
  28. 18 Feb, 2011 1 commit
  29. 28 Jan, 2011 2 commits
  30. 12 Jan, 2011 3 commits
    • Iustin Pop's avatar
      Run pylint over QA code too · 3582eef6
      Iustin Pop authored
      
      
      Right now, the QA code is not covered by pylint, and this shows at
      least one low-impact bug.
      
      This patch does the necessary changes to make QA pylint-clean, and the
      changes the makefile to run pylint for it.
      
      Notable changes:
      
      - qa_utils.GenericQueryTest: randfields was not used at all, and my
        belief is that it was indented to be used in order not to modify the
        input list; so I replaced randfields with fields, so we only shuffle
        the our local copy
      - qa_node.TestOutOfBand was using it's own copy of AcquireNode(), so I
        replaced it with the existing version
      - qa_os: was using 'dir' in a couple of places, replaced with dirname
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      3582eef6
    • Iustin Pop's avatar
      QA: use a persistent SSH connection to the master · f7e6f3c8
      Iustin Pop authored
      
      
      The recent additions to QA (many more tests) make QA slow if the
      machine on which the QA runs is not very close to the tested nodes —
      or in general, when the SSH handhaske is costly.
      
      We discussed before about using a persistent connection, and here is
      the patch that implements it. On a very small QA (very very small), it
      cuts down a lot of time (almost half), so it should be useful even for
      a full QA.
      
      I've also thought about changing from external ssh to paramiko, but I
      estimated that it would be more work to correctly interleave the IO
      from the remote process than just running a background SSH.
      
      Also note that yes, the global dict is ugly, but I don't know of
      another simple way to implement this.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      f7e6f3c8
    • Iustin Pop's avatar
      QA: Fix duplicated OOB tests · 69df9d2b
      Iustin Pop authored
      Patch f55312bd
      
       added the OOB tests to TestClusterVerify, which is not
      actually a test for cluster verify, but a runner for cluster verify
      that is called multiple times, for each instance type, etc. This led
      to running the OOB commands multiple times, which is painful
      especially as this is a slow test.
      
      The patch moves this to a separate test, that is run only once.
      
      Furthermore, the way that data files are copied around is very
      inefficient: touch + mv + chmod + mv + rm for each node (5 times
      number of nodes), whereas it could be simply: touch on master, chmod
      on master, cluster copyfile, chmod on master, cluster copyfile,
      cluster command rm, i.e. only 5 fixed ssh calls to the master. The
      code is changed as such, for increased speed.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      69df9d2b