GitLab

Some options have defaults that depend on the environment, and we
could handle these in two ways:

- use a place-holder value (e.g. data X a = Default | Custom a) that
  is later read from the environment
- move the options list to IO monad, where it can read the
  environment, etc.

The second option allows also displaying the actual defaults in the
`--help' output, even though it's not as nice, so I went with it.

This patch only changes the option types, without actually changing
any options yet.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Small simplifications of other unit tests using the (==?)
operator when possible, and typo fixes.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Furthermore, a few messages have their capitalisation changed (fixed).
Signed-off-by: Helga Velroyen <helgav@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Before we reorganised the source tree, the 'Result' type was exported
from HTools/Types.hs. This changed during the reorg, but at that time
we didn't change the exports; instead, we kept re-exporting it from
the old module for compatibility.

In light of future changes to the Result type, let's stop this
re-export and cleanup the imports of the other modules.

One test is slightly rewritten with explicit types declaration (part
of the values already needed one, let's make it explicit).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This defines the arguments supported and then modifies the
--help-completion output to include them too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is, I believe, the last non-htools specific file that still lived
in the htools directory; it's already widely used in non-htools code,
so let's move it before we add more functionality to this module.

All changes are related to the name change, imports fixup, etc.; there
are no other changes in this patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, we test and require that each individual program (hbal,
etc.) defines/supports the generic options (currently --help and
--version). Even with the test, this is not optimal, since it requires
changes in many places whenever we modify the list of generic options,
hence we move these out of the individual programs and into the
generic CLI handling code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Commit 2cdaf225, “Re-enable standard hlint warnings”, got it almost
right. The only problem is that (confusingly) the default set of hints
is not in HLint.Default, but in HLint.HLint (it includes Default and
some built-ins).

After changing the lint file to correctly include the defaults, we had
another 128 suggestions:

  - Error: Eta reduce (2)
  - Error: Redundant bracket (4)
  - Error: Redundant do (17)
  - Error: Redundant lambda (7)
  - Error: Redundant return (1)
  - Warning: Avoid lambda (2)
  - Warning: Redundant $ (42)
  - Warning: Redundant bracket (35)
  - Warning: Use : (1)
  - Warning: Use String (4)
  - Warning: Use camelCase (10)
  - Warning: Use section (3)

which are fixed by the current patch. Note that the 10 "Use camelCase"
were all due to hlint not “knowing” the idiom of ‘case_’ (it does for
‘prop_’), for which I filled
http://code.google.com/p/ndmitchell/issues/detail?id=558

.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As suggested by Rene, I added comments to types for printKeys
and printFinal. Also, realigned some lines in Hspace.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As hspace and hcheck both use machine readable options, they require
similar functions - which are now generalized and moved to CLI and
Utils.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

While updating the confd code, I realised that we have _lots_ of
duplication in the exit model for the various programs.

So this patch attempts to abstract all the exits via a couple of new
functions; sorry for the somewhat big patch, but I hope the payoff is
worth the change: the actual exit conditions are much clearer.

Note that the patch (also) moves the exitIfBad function to Utils.hs,
since that is more logical.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For now we just pass in "1" as spindle usage and adapt the backend
later.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is only useful in the stand-alone command line tools:

- hbal
- hspace
- hinfo

This change also documents the flag in all man-pages.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It prints the information which hbal prints in a nice formatted manner and of
course without doing any actions. An example output looks like this:

$ hinfo -m xen.example.com
Loaded 4 nodes, 2 instances
Cluster has 1 node group(s)
Cluster coefficients:
   Field                Value Weight
   free_mem_cv     0.00195342  x1.00
   free_disk_cv    0.00000000  x1.00
   n1_cnt          0.00000000  x1.00
   reserved_mem_cv 0.00195342  x1.00
   offline_all_cnt 0.00000000  x4.00
   offline_pri_cnt 0.00000000 x16.00
   vcpu_ratio_cv   0.12500000  x1.00
   cpu_load_cv     0.50000000  x1.00
   mem_load_cv     0.50000000  x1.00
   disk_load_cv    0.00000000  x1.00
   net_load_cv     0.50000000  x1.00
   pri_tags_score  0.00000000  x2.00
Cluster score: 1.62890685
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch changes the allocation delta stats from RSpec to
AllocInfo. There's nothing interesting about it, but it should allow
more correct computations of Npus in hspace.

I also moved the AllocStats type alias from Cluster.hs to Types.hs
(just makes more sense).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

hspace computes the NPU (normalised CPU units) via the
(used|pool|avail) VCPU divided by the (currently cluster-wide) max
vcpu_ratio.

However, in the future we'll have variable vcpu_ratios, so we can't
the calculation as such. We temporarily change to computing the
average vcpu_ratio across the cluster, however long-term this needs to
be improved, as ∑ (Vi/Pi) ≠ ∑ Vi / ∑ Pi. In the meantime we implement
this patch to remove the use of a global mcpu variable.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This patch moves the call to getArgs/parseOpts into htools.hs, as
opposed to the current situation where htools.hs only calls the main
file of each program, which in turn calls these two functions.

By doing it only once we remove some duplicate code and we also export
each program's options, which allows us to test some common behaviour
in the future.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently, in the human-readable mode, hspace doesn't show any
messages when no instances can be allocated in tiered-alloc mode. This
is confusing, so let's add an explicit message.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The default disk template is the first enabled template.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This changes from the current hardcoded defaults to the cluster
policy. The command line options now override the defaults from the
cluster, and the tiered spec mode is always enabled.

Also fixes a tiny typo in the man page (together with the man page
updates).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This attribute is always initialised to the default, and is not (yet)
read/saved in the various backends.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

More consistency with the tiered allocation mode.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

For the standard specs, hspace currently takes separate options for
the memory, disk and VCPUs. For the tiered specs, which were
introduced later, it takes a single option with all three values
combined.

This patch adds a backward-incompatible change to the standard spec,
basically moving it to the same single option format, which is a bit
more simple (both in the code and on the command line). This is
"needed" for future changes, where the command line will just
overwrite what we get from cluster, instead of setting the starting
point.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

First, we update the recommended hlint version to what I used to get a
clean output (1.8.15). Most of the changes are:

- remove unneeded parentheses
- some simplifications (intercalate " " → unwords, maybe … id →
  fromMaybe, etc.)
- removal of some duplicate code (in previous patches)

There are still some warnings which I didn't clean out but plain
ignored:

- 'Eta reduce' in some specific files, because the type inference
  specialises the function on the first call, and annotating the type
  properly would be too verbose
- use of 'first', 'comparing', and 'on', since these don't seem to be
  widely or consistently used (outside ganeti/htools, I mean)
- use of Control.Exception.catch, as we only care about I/O errors; at
  one point yes, we will need to transition to this new API
- 'Reduce duplication', since hlint warns even for 3 duplicate lines,
  and abstracting that away seems overkill to me

After this patch, make hlint is clean and doesn't exit with an error
anymore; we could enable it automatically on 'make lint' if hlint is
detected (future patch).

Note that we explicitly skip the THH.hs file from checking because it
seems that hlint doesn't parse correctly for now the splice notation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

While testing with ghc 7.2, I saw that some imports we are using are
very old (from ghc 6.8 time), even though current libraries are using
different names.

We fix this and bump minimum documented version to ghc 6.12, as I
don't have 6.10 to test anymore (possibly still works with that
version, but better safe - both Ubuntu Lucid and Debian Squeeze ship
with 6.12 nowadays).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This moves the checking of results from the allocation functions to a
separate function, so that we have less code duplication. It also does
a bit of simplification in the printing functions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The node offline/mcpu is identical to hbal's setNodesStatus, so let's
move that to CLI.hs and reuse it in hspace (also, rename it and drop
one 's').

Also, the check for the number of nodes is obsolete, as we compute
that from the disk template.

The patch does a bit of other small cleanups.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This changes the names for some helper functions so that future
patches are touching less unrelated code. The change replaces
shortened prefixes with the full type name.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

This is very useful for testing/benchmarking.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Tested only on GHC 7.x, will test on 6.1x too before commit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adds parameter documentation for Cluster.iMoveToJob (I think it
was not clear if the new or old node list is needed) and fixes other
docstring style issues.

After this patch, all modules except for CLI.hs (which has many
obvious declarations for command-line options) and QC.hs (unittests)
have 100% doc-strings.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, hspace can only output a machine-readable format that
(while detailed) is hard to parse quickly by people. This patch adds
(and enables by default) a human-readable output that shows the most
important metrics in a simple format.

Most of the work of the patch is in moving the display of various
metrics from the 'main' function to separate functions, each of which
can output either a machine or human intended format.

The patch also corrects a bug in the CPU efficiency display: before,
the efficiency was computed as instance virtual CPUs divided by total
physical CPUs, which is almost always supra-unitary. More correct is
to divide by the total virtual CPUs, which shows a more meaningful
number (when the p-to-v CPU ratio has been defined correctly).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

hspace and hbal treat -O differently, and use aliases for short names
(although hbal succeeds in that, and hspace doesn't). Uniform this with
a name lookup, using the same functions we used for instance
selection/exclusion.

Some of the code is by the way a bit repetitive, and could probably be
merged in a single function. That needs to be a monadic one, though, so
I promise to do it as soon as I realize how to write them! ;)
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is used just in hspace, so let's help in making Cluster.hs
smaller. We also split the function in two, as computing the spec map
and formatting it are two different tasks.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>