Skip to content
Snippets Groups Projects
  1. Jul 14, 2009
    • Iustin Pop's avatar
      Update documentation for the new luxi backend · b9017fa7
      Iustin Pop authored
      b9017fa7
    • Iustin Pop's avatar
      Add support for luxi backend in CLI/hspace/hbal · 8e445e6d
      Iustin Pop authored
      This patch changes the backend selection method in CLI to prefer, in order:
        - a RAPI specification
        - a Luxi specification
        - and finally the node/instance files
      
      It also modifies hspace and hbal to provide a ‘-L’ command line option
      for enabling Luxi.
      8e445e6d
    • Iustin Pop's avatar
      Initial commit of the luxi backend · 53ec9022
      Iustin Pop authored
      This patch adds a luxi backend that allows direct query of the master
      daemon on the local node. This patch doesn't enable the backend to be
      used.
      
      There are a couple of things still missing in the implementation:
        - we don't have a master timeout in reads and writes, only a
      	per-recv/send-call one
        - the failure reporting is not as good as it could be
      53ec9022
    • Iustin Pop's avatar
      Introduce timeout in RAPI queries · 135a6c6a
      Iustin Pop authored
      The patch adds two constants in Types.hs for connect and query timeout,
      then modifies Rapi.hs to use them as the connect and general curl
      timeout.
      
      Rapi could be improved more, as currently we wait double the total
      timeout due to not aborting early in case the node queries failed.
      135a6c6a
  2. Jul 09, 2009
    • Iustin Pop's avatar
      Update NEWS file for the 0.1.5 release · d8de9112
      Iustin Pop authored
      This is basically a hspace release, so the changelog is small.
      htools-v0.1.5
      d8de9112
    • Iustin Pop's avatar
      Fix a haddoc issue · 7d11799b
      Iustin Pop authored
      7d11799b
    • Iustin Pop's avatar
      hspace: fix failure handling of tryAlloc results · 31e7ac17
      Iustin Pop authored
      Currently hspace doesn't handle failures from tryAlloc correctly; this
      patch changes the iterateDepth function in hspace to return a Result (…)
      so that errors can be propagated correctly.
      
      The patch also changes one output key to be more clear and a typo in
      Cluster.hs
      31e7ac17
    • Iustin Pop's avatar
      Change the tryAlloc/tryReloc workflow · 478df686
      Iustin Pop authored
      Currently, the tryAlloc and tryReloc function return a list with all the
      results, both failures and successes. This is fine for hail, which does
      one round of allocations, but is not so good for hspace, which does
      iterative rounds; since at each (successful) step we only take the best
      solution, it means that we're using lots of heap space to compute and
      store node lists which are thrown away at the end of the step.
      
      This patch changes these two functions and their callers in hail/hspace
      to only return the best solution, and error/success counters. This
      allows hspace to run in a much smaller space, and reduces GC cost
      greatly.
      
      Overall, it is a cleanup, as hail/hspace did a lot of work to chose this
      best solution, whereas now it's automatically promoted within
      Cluster.concatAllocs.
      478df686
  3. Jul 08, 2009
    • Iustin Pop's avatar
      Simplify the Cluster.tryAlloc structures · 685935f7
      Iustin Pop authored
      Currently the tryAlloc function calls the
      allocateOnSingle/allocateOnPair and the builds a new tuple with those
      functions's result plus the new node list. This is however suboptimal
      in two respects:
        - the new nodes added are the 'old' versions of the respective nodes,
          so even though we don't use more than their names, it's logically
          broken
        - we do an extra unpack/repack of the result, while we could simply
          pass it through if allocateOnX returned the correct result
      
      This patch makes the allocateOnX functions return the node list too and
      also removes them and applyMove from the export list, as these are only
      used within Cluster.hs.
      685935f7
    • Iustin Pop's avatar
      Slight change to the internal allocation results · 8880d889
      Iustin Pop authored
      Currently the Cluster.AllocSolution type is defined as a list of
      ‘(OpResult Node.list, …)’ and the results for applyMove are defined as
      ‘(OpResult Node.List, …)’. Both these means that the failure/success
      indication is hidden in the first elements of this tuple, which makes is
      harder to add other elements depending on the success/failure (like the
      score for the new node list).
      
      This patch moves the OpResult to outside of the tuple, in effect making
      all these ‘OpResult (…)’ which makes the internal tuple elements
      consistent. The patch is affecting the applyMove, tryAlloc and
      tryRealloc functions, and only briefly the hail and hspace programs.
      8880d889
    • Iustin Pop's avatar
      Add a 'tags' makefile target · 9ea446d2
      Iustin Pop authored
      This uses hasktags for building emacs TAGS.
      9ea446d2
    • Iustin Pop's avatar
      hspace: switch output to shell-script format · 2bbf77cc
      Iustin Pop authored
      This (big) patch changes the output of hspace from text-format
      (separated by ‘: ’) to a shell-snippet, in ‘key=value’ format.
      
      This will allow sourcing the output or parsing it via awk/sed/etc.
      2bbf77cc
    • Iustin Pop's avatar
      hspace: move instance count and score into CStats · de4ac2c2
      Iustin Pop authored
      Currently the instance count and cluster score are separated from the
      other initial/final phase stats, even though they are very similar. This
      patch moves computation of these two into totalResources/CStats and
      removes special printing in hspace.
      de4ac2c2
  4. Jul 07, 2009
    • Iustin Pop's avatar
      Fix unittests · 79a72ce7
      Iustin Pop authored
      The recent OpResult and CPU values additions broke unittests.
      79a72ce7
    • Iustin Pop's avatar
      Export more stats in hspace · 8c4c6a8a
      Iustin Pop authored
      This patch changes Cluster.totalResources to compute more resources and
      prints them in hspace.
      8c4c6a8a
    • Iustin Pop's avatar
      Show errors on stderr instead of stdout · 2795466b
      Iustin Pop authored
      Currently many of the exit and warning conditions mistakenly display error
      messages on stdout, which makes parsing the output of programs harder. This
      patch attempts to fix such occurrences.
      2795466b
    • Iustin Pop's avatar
      Fix score calculation to work with empty clusters · 16103319
      Iustin Pop authored
      Currently the cluster score calculation includes an offline instance
      percentage, expressed as “offline inst / (offline + online inst)”, which
      results in NaN for empty clusters. This patch changes the calculation
      such that clusters with no offline instances (this criteria includes
      completely empty clusters) get a score of zero on this component of the
      total score.
      16103319
    • Iustin Pop's avatar
      hspace: convert N1 error exit into FailN1 result · dca7f396
      Iustin Pop authored
      Currently hspace exits with an error if the cluster is not N+1 compliant
      at the beginning of the run. This patch changes hspace such that this
      condition is instead treated as a zero-allocation-possible, FailN1 mode.
      This allows the same stats to be reported in this case as in the normal
      case.
      dca7f396
    • Iustin Pop's avatar
      Some docstring updates · 58631b72
      Iustin Pop authored
      58631b72
    • Iustin Pop's avatar
      hspace: add display of instance spec · 7e74e7db
      Iustin Pop authored
      This is mostly for user-friendliness in the default mode, when we don't
      specify the instance parameters.
      7e74e7db
  5. Jul 06, 2009
    • Iustin Pop's avatar
      Optimize the Utils.stdDev function · e6f4f05c
      Iustin Pop authored
      This patch optimizes the stdDev function in two respects:
        - first, we don't do sum . map which builds an intermediate list, but
      	instead use a fold over the list to build incrementally the sum;
      	this should reduce both the time and space characteristics, as we
      	have fewer objects created
        - second, we move from “a ^ 2” to “a * a” as the latter has a much
      	simpler implementation and thus a higher performance
      
      Since the ‘square’ function is obsoleted by the above the patch also
      removes it.
      e6f4f05c
    • Iustin Pop's avatar
      Take the foldl out of Loader.fixNodes · d71d0a1d
      Iustin Pop authored
      Currently Loader.fixNodes is foldl' with a complicated function. It
      makes more sense to take foldl' out of this function (and put it into
      the caller) and let fixNodes be only this internal function.
      d71d0a1d
    • Iustin Pop's avatar
      Simplify Cluster.computeMoves · 41c3b292
      Iustin Pop authored
      This patch changes the function Cluster.computeMoves to use guards and a
      couple of subexpressions in order to greatly simplify it.
      41c3b292
    • Iustin Pop's avatar
      Fix hlint-generated warnings · 9f6dcdea
      Iustin Pop authored
      This big patch cleans up the code per hlint indications. Many removals
      of extra parentheses, replacements of concat . map with concabtMap,
      extra dollar signs, eta reductions, etc. were performed.
      
      The code still compiles and passes a couple of manual tests on sample
      files. The individual changes are also small enough as to be visually
      easy to confirm.
      9f6dcdea
  6. Jul 05, 2009
    • Iustin Pop's avatar
      Add computation of the failure reason in hspace · 44763b51
      Iustin Pop authored
      This patch enhances hspace to report why the allocation sequence
      stopped, both in absolute error count and for the top reason.
      44763b51
    • Iustin Pop's avatar
      Return correct failure data from Node.add* · c43c3354
      Iustin Pop authored
      This patch alters the Node.addPri/addSec to return correct failure data.
      It removes the computeFailN1 function from the module as that used to
      combine both mem and disk checks in the same function and thus the real
      error reason cannot be deduced when using it.
      c43c3354
    • Iustin Pop's avatar
      Introduce a new type for allocation results · f2280553
      Iustin Pop authored
      Currently the allocation/move operations workflow return ‘Maybe a’,
      which is very convenient but loses all details about the failure mode.
      
      This patch introduces a new data type which encodes the specific failure
      mode. It is not yet used correctly (e.g. all node operations result in
      FailN1), but the workflow is updated. Most of the changes are in the
      hail/hspace tools, the library code required only trivial adjustments.
      f2280553
    • Iustin Pop's avatar
      Remove hn1 and related code · 266aea94
      Iustin Pop authored
      hn1 was deprecated for a while and this patch removes it altogether. The
      support code in Cluster.hs is also removed.
      266aea94
  7. Jul 03, 2009
    • Iustin Pop's avatar
      Display two more stats in hspace · 78ecfa8f
      Iustin Pop authored
      This adds two new stats - sum of reserved ram and disk.
      78ecfa8f
    • Iustin Pop's avatar
      Fix totalResources avail disk computation · 301789f4
      Iustin Pop authored
      This uses the newly-added Node.availDisk to compute the actual available
      disk correctl, and display the total allocatable disk in hspace.
      301789f4
    • Iustin Pop's avatar
      Add an availDisk node function · fe3d6f02
      Iustin Pop authored
      This function returns the amount of available disk, which depends on
      whether a low disk limit has been configured or not and on the free disk
      space of the node.
      fe3d6f02
    • Iustin Pop's avatar
      Add two new autocomputed vars to Nodes · 836533fa
      Iustin Pop authored
      Currently we track the max disk usage/max vcpus as percentages, however
      sometimes it's easier to check against minimum free disk or maximum
      number of cpus, as units instead of percentages.
      
      This patch adds two new variables, lo_dsk, hi_cpu, which are recomputed
      whenever we change the m_dsk/m_cpu 'master' values.
      836533fa
  8. Jul 02, 2009
    • Iustin Pop's avatar
      Add a new type for cluster statistics · 1a7eff0e
      Iustin Pop authored
      Currently totalResources returns a 5-tuple of integers. This is not easy
      to handle, as each change on the return type means that each caller must
      be updated.
      
      This patch adds a new type for cluster stats and uses that instead as
      its return type. This simplifies its callers while complicating only a
      little the computation of the stats.
      1a7eff0e
    • Iustin Pop's avatar
      Enhance hspace resource display · 621de5b7
      Iustin Pop authored
      The display of cluster resources is extracted into a separate function
      and enhanced to display more stats.
      621de5b7
    • Iustin Pop's avatar
      Add display of more stats in hspace · e2af3156
      Iustin Pop authored
      This patch changes Cluster.totalResources to compute more details about
      the cluster status, and enhances hspace to display more of these.
      e2af3156
  9. Jun 16, 2009
  10. Jun 12, 2009
Loading