1. 23 Jul, 2008 5 commits
    • Michael Hanselmann's avatar
      Rename JobStorage to DiskJobStorage · 21cc1fbd
      Michael Hanselmann authored
      Reviewed-by: iustinp
    • Michael Hanselmann's avatar
      Fix logging with string job IDs · 205d71fd
      Michael Hanselmann authored
      The job ID is now a string, hence logging must use %s instead of %d.
      Reviewed-by: iustinp
    • Michael Hanselmann's avatar
      Make job ID a string · 3be9a705
      Michael Hanselmann authored
      The docstring says that _NewSerialUnlocked returns “a string
      representing the job identifier”. Until now it returned an
      integer and this patch changes it.
      Reviewed-by: iustinp
    • Iustin Pop's avatar
      Distribute the queue serial file after each update · c3f0a12f
      Iustin Pop authored
      This patch adds distribution of the queue serial file after each write
      to it (but before a new job is created and written with that ID, and
      before a response is returned, so we should be safe from crashes in
      Currently it only logs if a node cannot be contacted, it should abort if
      > 50% errors are seen.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Make the job storage init reuse a serial file · c4beba1c
      Iustin Pop authored
      This will be needed for master failover. If we don't have a valid queue
      directory, we need to reinitialize it, but we should keep the existing
      serial number.
      As such, we abstract the reading of the serial and if we find a valid
      serial, we do not reset it.
      Reviewed-by: imsnah
  2. 22 Jul, 2008 1 commit
    • Michael Hanselmann's avatar
      Make argument to CleanCacheUnlocked mandatory · 57f8615f
      Michael Hanselmann authored
      Not passing the argument means it has the value None. Iterating None
      doesn't work:
        >>> "123" in None
        Traceback (most recent call last):
          File "<stdin>", line 1, in ?
        TypeError: iterable argument required
      Hence I rename it to "exclude" instead of "exceptions", which may be
      confusing, and make it mandatory. If one wants to clean all cache
      entries, an empty list can be passed.
      Reviewed-by: iustinp
  3. 17 Jul, 2008 1 commit
  4. 15 Jul, 2008 1 commit
  5. 14 Jul, 2008 4 commits
    • Iustin Pop's avatar
      First version of user feedback fixes · f1048938
      Iustin Pop authored
      This patch contains a raw version for fixing feedback_fn.
      The new mechanism works as follows:
        - instead of a per-Processor feedback_fn, there's one for each
          ExecOpCode, so that feedback for different opcodes go via possibly
          different functions
        - each _QueuedOpCode gets a message buffer, a method for adding
          feedback and a method for retrieving (parts of) the feedback
        - the _QueuedJob object gets a new attribute that is equal to the
          index of the currently executing opcode
        - job queries get an extra parameter called 'ticker' that will return
          the latest message on the current executing opcode
        - the cli.py job completion poll will show the new status if different
          from the old one
      Of course, quick messages will be lost, as currently only the latest one
      is available. Also changes between opcodes are not represented at all.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Cache some jobs in memory · ac0930b9
      Iustin Pop authored
      This patch adds a caching mechanisms to the JobStorage. Note that is
      does not make the memory cache authoritative.
      The algorithm is:
        - all jobs loaded from disks are entered in the cache
        - all new jobs are entered in the cache
        - at each job save (in UpdateJobUnlocked), jobs which are not
          executing or queued are removed from the cache
      The end effect is that running jobs will always be in the cache (which
      will fix the opcode log changes) and finished jobs will be kept for a
      while in the cache after being loaded.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Fix JobStorage._GetJobIDsUnlocked · 8a70e415
      Iustin Pop authored
      The job ID returned must be an integer (and the regex enforces that),
      but we didn't convert it manually.
      Reviewed-by: imsnah
    • Iustin Pop's avatar
      Change JobStorage to work with ids not filenames · 911a495b
      Iustin Pop authored
      Currently some of the functions in JobStorage work with filenames (which
      is an implementation detail and should only be used when dealing with
      the storage) and not with job IDs. We need to change this in order to
      implement a job cache.
      Reviewed-by: ultrotter
  6. 11 Jul, 2008 2 commits
    • Michael Hanselmann's avatar
      Add experimental persistency to job queue · f1da30e6
      Michael Hanselmann authored
      It's not perfect and it's not finished, but it's a start.
      - Serial number is read only once, but written on each update
      - Jobs are kept only on disk (caching will be implemented)
      Reviewed-by: iustinp
    • Michael Hanselmann's avatar
      Make "gnt-job list" work again · af30b2fd
      Michael Hanselmann authored
      "gnt-job list" was broken after my recent changes in the RPC
      between clients and the master. This patch makes it work again.
      Reviewed-by: iustinp
  7. 10 Jul, 2008 3 commits
  8. 09 Jul, 2008 2 commits
  9. 13 Jun, 2008 1 commit
  10. 10 Apr, 2008 1 commit
    • Iustin Pop's avatar
      Add per-opcode results to job processing · 35049ff2
      Iustin Pop authored
      This patch changes the definition of a job and introduces per-opcode
      First, the result and status fields of a job are condensed into a single
      'status' attribute. Then, we introduce an opcode status and one result
      list, that allow jobs to return values.
      The gnt-job script is also modified to allow these new fields to be
      Note that the patch changes the opcode field to op_list, and it changes
      its return value from string to a list of (serialized) opcodes.
      Reviewed-by: ultrotter
  11. 07 Apr, 2008 1 commit
  12. 04 Apr, 2008 1 commit
    • Iustin Pop's avatar
      Add a simple gnt-job script · 7a1ecaed
      Iustin Pop authored
      This patch adds a very basic gnt-job script that allows job querying.
      This goes on top of the previous master daemon patches.
      Currently, because of the not-changed cmd lock, you can't query the jobs
      as long as a job is running - you have to rm the cmd lock and then you
      can query the jobs.
      Reviewed-by: imsnah
  13. 01 Apr, 2008 1 commit