1. 08 Jul, 2008 6 commits
    • Guido Trotter's avatar
      mocks: create a FakeContext object · f4555db2
      Guido Trotter authored
      This will be passed to FakeLUs
      
      Reviewed-by: iustinp
      f4555db2
    • Guido Trotter's avatar
      Fix a typo in LUTestDelay docstring · 0b097284
      Guido Trotter authored
      Reviewed-by: iustinp
      0b097284
    • Guido Trotter's avatar
      Locking: remove LEVEL_CONFIG lockset · 08a6c581
      Guido Trotter authored
      Since the ConfigWriter now handles its own locking it's not necessary to
      have a specific level for the config in the Locking Manager anymore.
      This patch thus removes it, and all the unittest calls that used it, or
      depended on it being present.
      
      Reviewed-by: iustinp
      08a6c581
    • Guido Trotter's avatar
      ConfigWriter: synchronize access · f78ede4e
      Guido Trotter authored
      Since we share the ConfigWriter we need somehow to make sure that
      accessing it is properly synchronized. We'll do it using the
      locking.ssynchronized decorator and a module-private shared lock.
      
      This patch also renames a few functions, which were called inside the
      ConfigWriter, to a private version _UnlockedFunctionName, and exports
      the synchronized public ones. The internal callers, which are already
      synchronized, are then changed to use the _Unlocked version, to prevent
      double locking.
      
      Reviewed-by: iustinp
      f78ede4e
    • Guido Trotter's avatar
      Locking: add ssynchronized decorator · 42a999d1
      Guido Trotter authored
      This patch creates a new decorator function ssynchronized in the locking
      library, which takes as input a SharedLock, and synchronizes access to
      the decorated functions using it. The usual SharedLock semantics apply,
      so it's possible to call more than one synchronized function at the same
      time, when the lock is acquired in shared mode, and still protect
      against exclusive access.
      
      The patch also adds a few unit test to check the basic decorator's
      functionality, and to provide an example on how to use it.
      
      Reviewed-by: iustinp
      42a999d1
    • Guido Trotter's avatar
      ConfigWriter: remove _ReleaseLock · 5b4cd1b0
      Guido Trotter authored
      Remove empty function _ReleaseLock and all its calls. Since we only
      have one configwriter per cluster the locking needs to cover all the
      data in the object, and not just the file contents. Locking in
      ConfigWriter will be handled using the ganeti locking library.
      
      Reviewed-by: iustinp
      5b4cd1b0
  2. 04 Jul, 2008 2 commits
    • Iustin Pop's avatar
      Fix some issues with the watcher · 26517d45
      Iustin Pop authored
      This patch fixes two bugs:
        - the state file is not saved because we use the method for checking
          for udpated data
        - in two places 'Error' was used instead of 'Exception', which breaks
          error handling
      
      Additionally:
        - the unused 're' import has been removed
        - a variable named 'id' which collides with a builtin function has
          been renamed
      
      Note that comparing the serialized forms might create false negatives
      (due to the dicts being reordered) but that will just cause an extra
      write of the file, which is sub-optimal but harmless.
      
      Reviewed-by: ultrotter
      26517d45
    • Michael Hanselmann's avatar
      Add generic worker pool implementation · 76094e37
      Michael Hanselmann authored
      Reviewed-by: ultrotter
      76094e37
  3. 03 Jul, 2008 3 commits
    • Iustin Pop's avatar
      Reuse the luxi client in cli.SubmitOpCode · 2cb687ad
      Iustin Pop authored
      By a mistake, we don't reuse the luxi client. As such, we open and close
      the connection at each poll cycle and spam the server logs.
      
      Reviewed-by: ultrotter
      2cb687ad
    • Iustin Pop's avatar
      Add custom logging setup for daemons · 3b316acb
      Iustin Pop authored
      It's better for daemons if:
        - they log only to one log file
        - the log level is included
        - for debug runs, the filename/line number is included
      
      This patch moves the custom formatter from the watcher to the logging
      module and generalizes it; then it changes the master daemon to use this
      function instead of the generic logging (which might be deprecated
      anyway in the future).
      
      Reviewed-by: imsnah
      3b316acb
    • Iustin Pop's avatar
      Remove custom locking code from gnt-instance · fbe5b278
      Iustin Pop authored
      The gnt-instance script doesn't run in the same process anymore, so we
      can't and don't have to unlock.
      
      Reviewed-by: ultrotter
      fbe5b278
  4. 02 Jul, 2008 2 commits
  5. 01 Jul, 2008 10 commits
    • Guido Trotter's avatar
      Context: s/GLM/glm/ · 984f7c32
      Guido Trotter authored
      Make the GanetiLockManager instance of GanetiContext lowercase
      
      Reviewed-by: imsnah
      984f7c32
    • Michael Hanselmann's avatar
      Set locale when using docbook programs · 8ab8e969
      Michael Hanselmann authored
      At least docbook2man inserts a date formatted using the current
      locale into its output.
      
      Reviewed-by: iustinp
      8ab8e969
    • Iustin Pop's avatar
      Update .gitignore · 60053f31
      Iustin Pop authored
      Reviwed-by: imsnah
      60053f31
    • Iustin Pop's avatar
      Add a FirstFree function to utils.py · 7b4126b7
      Iustin Pop authored
      This function will return the first unused integer based on a list of
      used integers (e.g. [0, 1, 3] will return 2).
      
      Reviewed-by: imsnah
      7b4126b7
    • Guido Trotter's avatar
      Increase the thread size to 5 · a478cd7e
      Guido Trotter authored
      Now that we use the locking library to make sure running opcodes cannot
      step on each other toes we can have a bigger thread size, and
      potentially process many opcodes in a parallel manner.
      
      Reviewed-by: iustinp
      a478cd7e
    • Guido Trotter's avatar
      Processor: acquire the BGL for LUs requiring it · 04864530
      Guido Trotter authored
      If a LU required the BGL (all LUs do, right now, by default) we'll
      acquire it in the Processor before starting them. For LUs that don't
      we'll still acquire it, but in a shared fashion, so that they cannot run
      together with LUs that do.
      
      We'll also note down whether we own the BGL exclusively, and if we don't
      and we try to chain a LU that does, we'll fail.
      
      More work will need to be done, of course, to convert LUs not to require
      the BGL, but this basic infrastructure should guarantee the coexistance
      of the old and new world for the time being.
      
      Reviewed-by: iustinp
      04864530
    • Guido Trotter's avatar
      Processor: pass context in and use it. · 1c901d13
      Guido Trotter authored
      The processor used to create a new ConfigWriter when it was initialized.
      We now have one in the context, so we'll just recycle it. First of all
      we'll pass the context in when creating a new Processor object, then
      we'll just use context.cfg, which is granted to be initialized, wherever
      we used self.cfg, and stop checking whether the config is already
      initialized or not.
      
      In the future the Processor will be able to use the context also to
      acquire the BGL for LUs that require it, and to push the context down to
      LUs that don't in order for them to manage their own locking.
      
      Reviewed-by: iustinp
      1c901d13
    • Guido Trotter's avatar
      Add REQ_BGL LogicalUnit run requirement · 7e55040e
      Guido Trotter authored
      When logical units have REQ_BGL set (it is currently the default) they
      need to be the only ganeti operation run on the cluster, and we'll
      guarantee it at the master daemon level. Currently only one thread is
      running at a time, so this requirement is never broken.
      
      Reviewed-by: iustinp
      7e55040e
    • Guido Trotter's avatar
      Burnin doesn't need a Processor · 2da55f7c
      Guido Trotter authored
      In 2.0 burnin submits job to the master daemon, so it doesn't need to
      create an internal Processor anymore. Even if the processor is not used
      anywhere in the burnin code it was still initialized as a leftover of
      how burnin used to work. Fixing this.
      
      Reviewed-by: iustinp
      2da55f7c
    • Iustin Pop's avatar
      Implement “gnt-job list -o +...” · 7a5d3bbd
      Iustin Pop authored
      This adds the same “-o +...” functionality in gnt-job as in the node and
      instance scripts.
      
      Reviewed-by: imsnah
      7a5d3bbd
  6. 30 Jun, 2008 3 commits
    • Guido Trotter's avatar
      Fix sstore handling in Processor · c6868e1d
      Guido Trotter authored
      - no need to keep the sstore as an object member, remove it
      - don't reinitialize sstore only if self.cfg is None
          This is not an issue, as the Processor is recycled for every opcode,
          but in general we know that (a) we might need a different type of
          sstore for different opcodes and (b) initializating them is cheap
      - recreate sstore when chaining opcodes
          Without this fix chaining an opcode which requires a writable sstore
          to one which doesn't would fail. This doesn't happen today, but it's
          better to fix it anyway
      
      These changes are possible because nowadays all opcodes already require
      a working cluster/configuration.
      
      Reviewed-by: iustinp
      c6868e1d
    • Guido Trotter's avatar
      Remove duplicate code in hooks unittests · d2525573
      Guido Trotter authored
      All the tests there used to creare a cfg, a sstore, an opcode and a LU.
      Put all the duplicate code in the setUp function.
      
      Reviewed-by: iustinp
      d2525573
    • Guido Trotter's avatar
      ganeti-masterd: init and distribute common context · 39dcf2ef
      Guido Trotter authored
      This patch creates a new GanetiContext class, which is used to hold
      context common to all ganeti worker threads. As for the
      GanetiLockingManager class it is paramount that there is only one such
      class throughout the execution of Ganeti, so the class checks for that,
      and also forbids its own modification after it's been initialized. The
      context for now contains a ConfigWriter and a GanetiLockingManager and
      is created by the daemon and propagated to PoolWorker(s) and
      JobRunner(s).
      
      Reviewed-by: iustinp
      39dcf2ef
  7. 27 Jun, 2008 11 commits
    • Guido Trotter's avatar
      AddNode: move the initial setup to boostrap · 827f753e
      Guido Trotter authored
      From the master node we can't start ssh and connect to the remote node,
      nor we can do it from ganeti-noded as this ssh section will possibly ask
      for key confirmation and password. So the code to copy the ganeti-noded
      password and SSL key has been moved to bootstrap.py, and it's called by
      gnt-node before the AddNode opcode.
      
      Reviewed-by: iustinp
      827f753e
    • Guido Trotter's avatar
      AddNode: Check for node existance · 05ccd983
      Guido Trotter authored
      In the "new world" we'll need to setup ganeti-noded via ssh on the node
      before calling the AddNode opcode. Before doing it we'll check that the
      node is not already in the cluster, if --readd was not passed. This
      guarantees we're not going to restart ganeti-noded on a running node.
      
      This patch also incidentally fixes a non-style-guide conformant
      docstring.
      
      Reviewed-by: iustinp
      05ccd983
    • Guido Trotter's avatar
      LUAddNode: use node-verify to check node hostname · 5c0527ed
      Guido Trotter authored
      As we can't use ssh.VerifyNodeHostname directly, we'll set up a mini
      node-verify to do checking between the master and the new node. In the
      future networking checks, or more nodes, can be added as well.
      
      Reviewed-by: iustinp
      5c0527ed
    • Guido Trotter's avatar
      LUAddNode: use self.sstore, not a local ss · 3d1e7706
      Guido Trotter authored
      Since we're inside a LU we have access to self.sstore.
      No need to use ss, which separate instantiation will disappear in a few
      patches! ;)
      
      Reviewed-by: iustinp
      3d1e7706
    • Guido Trotter's avatar
      LUAddNode: upload files via rpc, not scp · b5602d15
      Guido Trotter authored
      We used to scp all the ssconf files, and the vnc password file to the
      new node. With this patch we use the upload_file rpc, specifying just
      the new node as a destination. All the files previously copied by scp
      are already allowed by the backend.
      
      Reviewed-by: iustinp
      b5602d15
    • Guido Trotter's avatar
      Allow VNC_PASSWORD_FILE to be rpc-uploaded · 90fae627
      Guido Trotter authored
      What could possibly go wrong?
      
      Reviewed-by: iustinp
      90fae627
    • Guido Trotter's avatar
      Change fping to TcpPing in two LUs · 937f983d
      Guido Trotter authored
      Two LUs are using RunCmd to call fping, in order to check for an IP
      presence on the network. Substituting it with TcpPing will get rid of
      it, which makes it not break in the new world order, where the master
      cannot fork.
      
      Reviewed-by: iustinp
      937f983d
    • Guido Trotter's avatar
      raise QuitGanetiException in LeaveCluster · 6d8b6238
      Guido Trotter authored
      Reviewed-by: iustinp
      6d8b6238
    • Guido Trotter's avatar
      ganeti-noded: Fix handling of QuitGanetiException · c3d7f69b
      Guido Trotter authored
      - s/GanetiQuitException/QuitGanetiException/
      - Look for the arguments in err.args, not err itself
      
      Reviewed-by: iustinp
      c3d7f69b
    • Guido Trotter's avatar
      Simplify QuitGanetiException instantiation · 9f9c8ee2
      Guido Trotter authored
      Rather than packing all the arguments in a tuple, let's pass them
      plainly. The superclass won't complain.
      
      Reviewed-by: iustinp
      9f9c8ee2
    • Michael Hanselmann's avatar
      logger: Set formatter for stderr · 5023934a
      Michael Hanselmann authored
      Having a timestamp on log messages is very useful. The default
      format string doesn't include a timestamp.
      
      Reviewed-by: ultrotter
      5023934a
  8. 26 Jun, 2008 3 commits
    • Guido Trotter's avatar
      When removing a node don't ssh to it · d489ca4f
      Guido Trotter authored
      Even in 1.2 this behaviour is broken, as the rpc call will remove the
      ssh keys before we get a chance to log in. Now the rpc takes care of
      shutting down the node daemon as well, so we definitely can avoid this.
      
      This makes the LURemoveNode operation work again with the threaded
      master daemon.
      
      Reviewed-by: iustinp
      d489ca4f
    • Guido Trotter's avatar
      ganeti-noded: quit on QuitGanetiException · 9ae49f27
      Guido Trotter authored
      Accoring to the usage documented in the QuitGanetiException docstring,
      if we receive such an exception we'll set the global _EXIT_GANETI_NODED
      variable to True, and then return either a valid value or an error
      message to the user. This will be the last request we serve, though,
      because the main loop will be interrupted and the daemon will terminate.
      
      Reviewed-by: iustinp
      9ae49f27
    • Guido Trotter's avatar
      Add errors.QuitGanetiException · e50bdd68
      Guido Trotter authored
      This exception does not signal an error but serves the purpose of making
      the ganeti daemon shut down after handling a request. Currently it will
      be used by ganeti-noded but in the future ganeti-masterd might make use
      of it as well. Its usage is documented in the docstring.
      
      Reviewed-by: iustinp
      e50bdd68