Commit be468da0 authored by Iustin Pop's avatar Iustin Pop
Browse files

Switch hspace defaults to the cluster policy



This changes from the current hardcoded defaults to the cluster
policy. The command line options now override the defaults from the
cluster, and the tiered spec mode is always enabled.

Also fixes a tiny typo in the man page (together with the man page
updates).
Signed-off-by: default avatarIustin Pop <iustin@google.com>
Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
parent 304f9292
...@@ -134,7 +134,7 @@ data Options = Options ...@@ -134,7 +134,7 @@ data Options = Options
, optShowInsts :: Bool -- ^ Whether to show the instance map , optShowInsts :: Bool -- ^ Whether to show the instance map
, optShowNodes :: Maybe [String] -- ^ Whether to show node status , optShowNodes :: Maybe [String] -- ^ Whether to show node status
, optShowVer :: Bool -- ^ Just show the program version , optShowVer :: Bool -- ^ Just show the program version
, optStdSpec :: RSpec -- ^ Requested standard specs , optStdSpec :: Maybe RSpec -- ^ Requested standard specs
, optTieredSpec :: Maybe RSpec -- ^ Requested specs for tiered mode , optTieredSpec :: Maybe RSpec -- ^ Requested specs for tiered mode
, optReplay :: Maybe String -- ^ Unittests: RNG state , optReplay :: Maybe String -- ^ Unittests: RNG state
, optVerbose :: Int -- ^ Verbosity level , optVerbose :: Int -- ^ Verbosity level
...@@ -173,7 +173,7 @@ defaultOptions = Options ...@@ -173,7 +173,7 @@ defaultOptions = Options
, optShowInsts = False , optShowInsts = False
, optShowNodes = Nothing , optShowNodes = Nothing
, optShowVer = False , optShowVer = False
, optStdSpec = RSpec 1 4096 102400 , optStdSpec = Nothing
, optTieredSpec = Nothing , optTieredSpec = Nothing
, optReplay = Nothing , optReplay = Nothing
, optVerbose = 1 , optVerbose = 1
...@@ -383,7 +383,7 @@ oStdSpec :: OptType ...@@ -383,7 +383,7 @@ oStdSpec :: OptType
oStdSpec = Option "" ["standard-alloc"] oStdSpec = Option "" ["standard-alloc"]
(ReqArg (\ inp opts -> do (ReqArg (\ inp opts -> do
tspec <- parseISpecString "standard" inp tspec <- parseISpecString "standard" inp
return $ opts { optStdSpec = tspec } ) return $ opts { optStdSpec = Just tspec } )
"STDSPEC") "STDSPEC")
"enable standard specs allocation, given as 'disk,ram,cpu'" "enable standard specs allocation, given as 'disk,ram,cpu'"
......
...@@ -29,6 +29,7 @@ import Control.Monad ...@@ -29,6 +29,7 @@ import Control.Monad
import Data.Char (toUpper, isAlphaNum, toLower) import Data.Char (toUpper, isAlphaNum, toLower)
import Data.Function (on) import Data.Function (on)
import Data.List import Data.List
import Data.Maybe (fromMaybe)
import Data.Ord (comparing) import Data.Ord (comparing)
import System.Exit import System.Exit
import System.IO import System.IO
...@@ -398,12 +399,11 @@ main = do ...@@ -398,12 +399,11 @@ main = do
exitWith $ ExitFailure 1 exitWith $ ExitFailure 1
let verbose = optVerbose opts let verbose = optVerbose opts
ispec = optStdSpec opts
disk_template = optDiskTemplate opts disk_template = optDiskTemplate opts
req_nodes = Instance.requiredNodes disk_template req_nodes = Instance.requiredNodes disk_template
machine_r = optMachineReadable opts machine_r = optMachineReadable opts
orig_cdata@(ClusterData gl fixed_nl il _ _) <- loadExternalData opts orig_cdata@(ClusterData gl fixed_nl il _ ipol) <- loadExternalData opts
nl <- setNodeStatus opts fixed_nl nl <- setNodeStatus opts fixed_nl
let num_instances = Container.size il let num_instances = Container.size il
...@@ -431,21 +431,24 @@ main = do ...@@ -431,21 +431,24 @@ main = do
allocnodes <- exitIfBad $ Cluster.genAllocNodes gl nl req_nodes True allocnodes <- exitIfBad $ Cluster.genAllocNodes gl nl req_nodes True
-- Run the tiered allocation, if enabled -- Run the tiered allocation
case optTieredSpec opts of let tspec = fromMaybe (rspecFromISpec (iPolicyMaxSpec ipol))
Nothing -> return () (optTieredSpec opts)
Just tspec -> do
(treason, trl_nl, _, spec_map) <-
runAllocation cdata stop_allocation
(Cluster.tieredAlloc nl il alloclimit
(instFromSpec tspec disk_template) allocnodes [] [])
tspec SpecTiered opts
printTiered machine_r spec_map (optMcpu opts) nl trl_nl treason (treason, trl_nl, _, spec_map) <-
runAllocation cdata stop_allocation
(Cluster.tieredAlloc nl il alloclimit
(instFromSpec tspec disk_template) allocnodes [] [])
tspec SpecTiered opts
printTiered machine_r spec_map (optMcpu opts) nl trl_nl treason
-- Run the standard (avg-mode) allocation -- Run the standard (avg-mode) allocation
let ispec = fromMaybe (rspecFromISpec (iPolicyStdSpec ipol))
(optStdSpec opts)
(sreason, fin_nl, allocs, _) <- (sreason, fin_nl, allocs, _) <-
runAllocation cdata stop_allocation runAllocation cdata stop_allocation
(Cluster.iterateAlloc nl il alloclimit (Cluster.iterateAlloc nl il alloclimit
...@@ -454,4 +457,6 @@ main = do ...@@ -454,4 +457,6 @@ main = do
printResults machine_r nl fin_nl num_instances allocs sreason printResults machine_r nl fin_nl num_instances allocs sreason
-- Print final result
printFinal machine_r printFinal machine_r
...@@ -44,7 +44,6 @@ Output options: ...@@ -44,7 +44,6 @@ Output options:
DESCRIPTION DESCRIPTION
----------- -----------
hspace computes how many additional instances can be fit on a cluster, hspace computes how many additional instances can be fit on a cluster,
while maintaining N+1 status. while maintaining N+1 status.
...@@ -61,6 +60,10 @@ it is intended to interpreted as a shell fragment (or parsed as a ...@@ -61,6 +60,10 @@ it is intended to interpreted as a shell fragment (or parsed as a
output the additional information on stderr (such that the stdout is output the additional information on stderr (such that the stdout is
still parseable). still parseable).
By default, the instance specifications will be read from the cluster;
the options ``--standard-alloc`` and ``--tiered-alloc`` can be used to
override them.
The following keys are available in the machine-readable output of the The following keys are available in the machine-readable output of the
script (all prefixed with *HTS_*): script (all prefixed with *HTS_*):
...@@ -132,14 +135,13 @@ INI_MNODE_DSK_AVAIL, FIN_MNODE_DSK_AVAIL ...@@ -132,14 +135,13 @@ INI_MNODE_DSK_AVAIL, FIN_MNODE_DSK_AVAIL
Like the above but for disk. Like the above but for disk.
TSPEC TSPEC
If the tiered allocation mode has been enabled, this parameter holds This parameter holds the pairs of specifications and counts of
the pairs of specifications and counts of instances that can be instances that can be created in the *tiered allocation* mode. The
created in this mode. The value of the key is a space-separated list value of the key is a space-separated list of values; each value is of
of values; each value is of the form *memory,disk,vcpu=count* where the form *memory,disk,vcpu=count* where the memory, disk and vcpu are
the memory, disk and vcpu are the values for the current spec, and the values for the current spec, and count is how many instances of
count is how many instances of this spec can be created. A complete this spec can be created. A complete value for this variable could be:
value for this variable could be: **4096,102400,2=225 **4096,102400,2=225 2560,102400,2=20 512,102400,2=21**.
2560,102400,2=20 512,102400,2=21**.
KM_USED_CPU, KM_USED_NPU, KM_USED_MEM, KM_USED_DSK KM_USED_CPU, KM_USED_NPU, KM_USED_MEM, KM_USED_DSK
These represents the metrics of used resources at the start of the These represents the metrics of used resources at the start of the
...@@ -189,9 +191,8 @@ OK ...@@ -189,9 +191,8 @@ OK
that the computation failed and any values present should not be that the computation failed and any values present should not be
relied upon. relied upon.
If the tiered allocation mode is enabled, then many of the INI_/FIN_ Many of the INI_/FIN_ metrics will be also displayed with a TRL_ prefix,
metrics will be also displayed with a TRL_ prefix, and denote the and denote the cluster status at the end of the tiered allocation run.
cluster status at the end of the tiered allocation run.
The human output format should be self-explanatory, so it is not The human output format should be self-explanatory, so it is not
described further. described further.
...@@ -274,9 +275,9 @@ The options that can be passed to the program are as follows: ...@@ -274,9 +275,9 @@ The options that can be passed to the program are as follows:
about the description, see the man page **htools**(1). about the description, see the man page **htools**(1).
--standard-alloc *disk,ram,cpu* --standard-alloc *disk,ram,cpu*
This option specifies the instance size for the *standard* allocation This option overrides the instance size read from the cluster for the
mode, where we simply allocate instances of the same, fixed size until *standard* allocation mode, where we simply allocate instances of the
the cluster runs out of space. same, fixed size until the cluster runs out of space.
The specification given is similar to the *--simulate* option and it The specification given is similar to the *--simulate* option and it
holds: holds:
...@@ -289,19 +290,19 @@ The options that can be passed to the program are as follows: ...@@ -289,19 +290,19 @@ The options that can be passed to the program are as follows:
specification of 100GB of disk space, 4GiB of memory and 2 VCPUs. specification of 100GB of disk space, 4GiB of memory and 2 VCPUs.
--tiered-alloc *disk,ram,cpu* --tiered-alloc *disk,ram,cpu*
Besides the standard, fixed-size allocation, also do a tiered This option overrides the instance size for the *tiered* allocation
allocation scheme where the algorithm starts from the given mode. In this mode, the algorithm starts from the given specification
specification and allocates until there is no more space; then it and allocates until there is no more space; then it decreases the
decreases the specification and tries the allocation again. The specification and tries the allocation again. The decrease is done on
decrease is done on the metric that last failed during allocation. The the metric that last failed during allocation. The argument should
argument should have the same format as for ``-standard-alloc``. have the same format as for ``--standard-alloc``.
Also note that the normal allocation and the tiered allocation are Also note that the normal allocation and the tiered allocation are
independent, and both start from the initial cluster state; as such, independent, and both start from the initial cluster state; as such,
the instance count for these two modes are not related one to the instance count for these two modes are not related one to
another. another.
--machines-readable[=*choice*] --machine-readable[=*choice*]
By default, the output of the program is in "human-readable" format, By default, the output of the program is in "human-readable" format,
i.e. text descriptions. By passing this flag you can either enable i.e. text descriptions. By passing this flag you can either enable
(``--machine-readable`` or ``--machine-readable=yes``) or explicitly (``--machine-readable`` or ``--machine-readable=yes``) or explicitly
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment