GitLab

In "Ganeti.Types", rename constructor "ExportModeRemove" of
"ExportMode" to "ExportModeRemote".  This fixes issue 577.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix, in 'Ganeti.Types', data type 'NICMode', which is missing a
constructor which is the counterpart of the Python constant
'NIC_IP_POOL'. This fixes issue 576.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Haskell types should be the source of information related to
constants.  Therefore, instead of building the Haskell types from the
constants, the constants should be taken from the Haskell types.  This
patch updates all the types defined in 'Ganeti.Types' to follow this
approach.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add missing exports in 'Ganeti.Types' which are necessary for
'Ganeti.HsConstants'.  These exports concern the '*ToRaw' functions
which convert a Haskell type into JSON.  These functions are needed by
the constants which we want to generate from Haskell to Python based
on Haskell types.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add xen related constants, such as, 'xenBootloader' and 'xenCmdXm' to
the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'devConsole' and syslog related constants to the Haskell to Python
constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'declareLADT' in Template Haskell module to declare Haskell
datatypes using 'String's directly as values for the JSON
serialization, as opposed to 'Name's which is what the current
'declareADT' allows.  To achieve this, 'genFromRaw' must be
generalized, similarly to 'genToRaw'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add exit code constants to Haskell to Python constant generation
infrastructure.  The module 'Ganeti.ConstantUtils' must be imported
qualified in order to avoid a clash between
'Ganeti.ConstantUtils.exitFailure' and
'Ganeti.HsConstants.exitFailure'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Move constant 'DAEMONS_LOGBASE' from Python to Haskell to be
automatically generated.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Before this patch, Haskell types, such as, 'GanetiDaemon' and
'GanetiGroup', and related functions were taking their values from
Haskell constants.  However, given that the role of Haskell to Python
constants is to leverage Haskell and its typesystem, it makes sense to
first define the Haskell types and then have the constants depend on
these types.  In other words, this patch series inverts the dependency
between (some) Haskell types and constants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Given that Haskell has its own generated 'AutoConf' module, it makes
sense that other Haskell modules import 'configure' variables directly
from 'AutoConf' instead of importing the Python to Haskell generated
constants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Haskell constants 'daemonsExtraLogfilesGanetiMondAccess' and
'daemonsExtraLogfilesGanetiMondError' cannot be constants because
their Python counterparts are calculated through
'pathutils.GetLogFilename', which indirectly depends on the
environment variable 'GANETI_ROOTDIR', as part of the virtual cluster
configuration.  Instead, these paths must be computed at runtime, as
opposed to being computed at compile time through the Python to
Haskell constant generation, and must also depend on the same
environment variable.  Fixes issue 575.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Modify CPUload data collector to get the number of clock ticks
per second from Posix.Unistd. Since CPUload collector converts
clock ticks (jiffies) to seconds and the clock ticks to seconds
ratio varies between linux distributions, it must be read from
the corresponding library to be consistent.
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This will allow to do balancing based only on static information.
In particular, the effect of balancing towards same number of
instances per node that the dynamic usage consideration has
if no usage file is provided, can be turned off, if this is
desired.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Make the external data loader correctly honor the --ignore-dynu option
by clearing the dynamic utilisation data from all instances if the option
is given.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Using this utility function, the dynamic utilisation of all instances
can be set to 0. This will have the effect of all utilisation being ignored.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This option, if set, will tell htools to ignore any dynamic
use data.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

* add 'daemonsPorts' and related constants used in the construction of
  the 'Map' that maps each daemon to its corresponding protocol and
  port.  This breaks constant 'daemonsPortsGanetiNoded' which is
  automatically generated from the Python dictionary 'DAEMONS_PORTS'

* replace reference from constant 'daemonsPortsGanetiNoded' to
  'defaultNodedPort' because constant 'daemonsPortsGanetiNoded' was a
  tuple containing the protocol and the port for node daemon, but only
  the port was needed.  And because this tuple was making Haskell to
  Python constant generation more difficult, it is preferable to
  replace this tuple with the actual port and use it directly, in
  module 'Ganeti.Rpc'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'Protocol' datatype which is necessary for the 'daemonsPorts'
constant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix instances of 'PyValue', namely, for tuples and maps.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* move several groups of constants, from Python to Haskell, which are
  currently being used in Haskell, including SSH and daemon related
  constants, policy allocation, blockdev driver manual (this is a
  single constant), auto repair related constants, and data collector
  version (also a single constant)
* constant 'AUTO_REPAIR_ALL_RESULTS' is moved to Python and its type
  changes from '[String]' to 'FrozenSet String' in order to be
  consistent with the Python type
* fix tests related to the constant 'autoRepairAllResults' (i.e., the
  Haskell version of 'AUTO_REPAIR_ALL_RESULTS') to reflect the type
  change
* update instances of 'FrozenSet'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add user separation constants, such as, 'daemonsGroup', 'rapiGroup',
and 'rapiUser', to the Haskell to Python constant generation
infrastructure.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add constant 'adminstAll' to Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add type 'FrozenSet' that wrap a Haskell 'Set', and a guarded
constructor for this type and a 'PyValue' instance.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'Ganeti.Hs2Py.ListContants.hs.in' template module documentation to
warn about automatic module generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'HsConstants' module documentation to warn about proper module
usage.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In what Haskell to Python opcodes are concerned, a Haskell 'Set' is
translated into a Python 'list'.  In other words, currently, opcodes
that handle sets of parameters are actually handling lists because
this is how sets are currently encoded.  This patch introduces a new
type called 'ListSet' that wraps a Haskell 'Set' and it is used to
represent on the Haskell side a Python 'list' without duplicate
elements.  This patch also updates the respective opcode parameters
and updates the opcode tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'List' instance of 'PyValue' to properly convert Haskell lists to
Python lists.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit a9542a4f

 introduced support for DSA SSH keys. However, the dsahostkeypub
field added to the config is not marked as optional in the Haskell components.
As a result, luxid thinks the config file is corrupt and refuses to start. We
fix this by marking the dsahostkeypub as an optionalField.

This fixes issue 560.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix how Haskell to Python opcode generation handles unit. It seems
that, in 'template-haskell-2.7.0', unit is represented by Template
Haskell as a constructor of the form '(ConT name)', where 'name' is
the constructor name for 'Unit'.  However, in 'template-haskell-2.8.0'
unit is represented as a tuple with no elements (i.e., 'TupleT 0').
This patch fixes the opcode generation to handle both cases, not just
the first.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Haskell to Python constant generation introduced duplicated constants
in Haskell.  This patch eliminates the duplication and fixes Haskell
to import the correct constants, instead of the generated ones.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Setup a transitional Haskell to Python constant generation
infrastructure.  This infrastructure is transitional and Python
constants will not be migrated to Haskell all at once.  Instead, some
constants will be incrementally pulled from Python and moved to
Haskell.  This is a list of the relevant modules in this
infrastructure:

* Ganeti.HsConstants: contains the Haskell constants that are used
  both in Haskell (once re-exported through Ganeti.Constants) and
  generated in Python

* Ganety.Hs2Py.ListConstants: contains the list of Haskell constants
  to be generated in Python; these constants are defined in the
  previous module and the list is automatically generated by the
  Makefile

* hs2py-constants: Haskell program that outputs the actual constant
  definitions in Python; this task cannot be performed by 'hs2py' just
  yet because this programs depends on the 'Constants' module. But
  once all the constants have migrated to Haskell, then
  'hs2py-constants' and 'hs2py' can be merged.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Rename the generated 'Constants.hs' module, which contains the Haskell
constants generated from Python, to 'PyConstants.hs' in order to
eliminate duplicated constants in Haskell, a problem introduced by the
Haskell to Python constant generation, and to circumvent a problem
with import/export of Haskell modules.  Add a new module named
'Constants.hs' which imports 'PyConstants.hs' and exports its names.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add Template Haskell functions that generate a list in Haskell
containing all the constants to be generated in Python.  This list is
an association list mapping the constant name to its Python value,
which is stored as a string.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Move some 'PyValue' instances to a separate file so they can be
available to other modules which need these instances but do not want
to depend on the constants module, which led to compilation
constraints.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LD_* constants are basically like DT_* constants, except
for that both file and shared file were mapped to file.
In order to not having to maintain three slightly different
sets of disk-related constants (DT, LD and ST), we merge
DT and LD here. This patch does the renaming of the
various constants and then fixes the handling of shared
file disks.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In commit 807d8853

, new node parameters were introduced in the python world.
Add them to the haskell world as well to restore consistency.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

If information about instances is not available, just log the error and
continue without it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add logWarningIfBad, a utility function similar to exitIfBad, that logs a
warning and returns a default value instead of just crashing the program if
the unpacked value is Bad.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The disk_template parameter is optional during disk create (the first
enabled disk template is taken in this case), so don't require it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>