GitLab

In the example, we have two nodes and 6 instances, all using external
storage. There are four small instances on one node, and two big
instances, requiring twice the amount of resources of a small instance,
on the other node. So, with respect to static data, the cluster
is already perfectly balanced. Using the default assumption of
dynamic usage, however, the cluster can be improved by moving
one small instance to the other node. The reason is, that by default,
all instances are assumed to use the same resources, and this measure
dominates, as all static resources are greatly underutilized.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This will allow to do balancing based only on static information.
In particular, the effect of balancing towards same number of
instances per node that the dynamic usage consideration has
if no usage file is provided, can be turned off, if this is
desired.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Make the external data loader correctly honor the --ignore-dynu option
by clearing the dynamic utilisation data from all instances if the option
is given.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Using this utility function, the dynamic utilisation of all instances
can be set to 0. This will have the effect of all utilisation being ignored.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This option, if set, will tell htools to ignore any dynamic
use data.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch provides rudimentary unit test coverage for LUInstanceQuery
and LUInstanceQueryData.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds rudimentary unit test coverage for LUInstanceMigrate and
LUInstanceFailover.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch provides unit test coverage for LUInstanceChangeGroup.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Those two occurrences of warnings can be replaced the RpcResult.Warn
method, which is done in this patch.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The validation of the opcodes already checks the size parameter for the
right type, so this except block is not required.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Rename the "node" variable to "node_uuid" as that's the content it
actually receives.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds unit tests for LUInstanceSetParams. It does not provide
100% coverage, but covers a significant portion.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The netmask was previously wrongly given as a bitmask, now it's given as
the net prefix length.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix target 'src/hs2py-constants' because it must depend on
'src/AutoConf.hs'. Fixes issue 573.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.9
  Fix bridging in net-common
  Sync build_chroot with buildbot slack role
  Auto-upgrade of disks' config wrt LD-renaming
  Fix tests regarding DISK_LD_DEFAULTS
  Fixing renaming of DISK_LD_DEFAULTS
  Replace LD_* constants with DT_* constants
  QA: fix file storage QA wrt ipolicy
  Bump versions to release 2.9~beta1
  Add design-2.9 to docinput
  NEWS update and version bump for 2.8 rc2
  Lint improvements to regexps
  Add unit test for GetLinuxNodeInfo
  Prepare GetLinuxNodeInfo for testing
  Check right disk template in inst set params
  Make the DRBD collector more failure-resilient
  Add function to unwrap Results logging failures
  Fix bug in NodeD and RapiD usage strings

* stable-2.8
  Make tools/users-setup explain its actions first
  Generate tools/users-setup from descriptions
  Provide ganeti user and group data in easy-to-parse form
  Remove dsahostkeypub during config downgrade
  Mark the DSA host pubkey as optional
  Fix documentation for gnt-node evacuate -p option

Conflicts:
	Makefile.am (trivial)
	devel/build_chroot (took changes from 2.9 but newer regex-pcre
                        version)
	lib/cmdlib/cluster.py (followed LD_* -> DT_* change)
	lib/objects.py (followed LD_* -> DT_* change)
	test/py/ganeti.cmdlib_unittest.py (followed LD_* -> DT_* change)

The LD_* -> DT_* change required some more changes in the cmdlib unit
tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update reference to 'DAEMONS_PORTS' and related constants that are
used on the Haskell side to construct the 'daemonsPorts' constant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* add 'daemonsPorts' and related constants used in the construction of
  the 'Map' that maps each daemon to its corresponding protocol and
  port.  This breaks constant 'daemonsPortsGanetiNoded' which is
  automatically generated from the Python dictionary 'DAEMONS_PORTS'

* replace reference from constant 'daemonsPortsGanetiNoded' to
  'defaultNodedPort' because constant 'daemonsPortsGanetiNoded' was a
  tuple containing the protocol and the port for node daemon, but only
  the port was needed.  And because this tuple was making Haskell to
  Python constant generation more difficult, it is preferable to
  replace this tuple with the actual port and use it directly, in
  module 'Ganeti.Rpc'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'Protocol' datatype which is necessary for the 'daemonsPorts'
constant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix instances of 'PyValue', namely, for tuples and maps.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update references to the Python constants introduced by the previous
patch.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* move several groups of constants, from Python to Haskell, which are
  currently being used in Haskell, including SSH and daemon related
  constants, policy allocation, blockdev driver manual (this is a
  single constant), auto repair related constants, and data collector
  version (also a single constant)
* constant 'AUTO_REPAIR_ALL_RESULTS' is moved to Python and its type
  changes from '[String]' to 'FrozenSet String' in order to be
  consistent with the Python type
* fix tests related to the constant 'autoRepairAllResults' (i.e., the
  Haskell version of 'AUTO_REPAIR_ALL_RESULTS') to reflect the type
  change
* update instances of 'FrozenSet'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.8
  Make tools/users-setup explain its actions first
  Generate tools/users-setup from descriptions
  Provide ganeti user and group data in easy-to-parse form
  Remove dsahostkeypub during config downgrade
  Mark the DSA host pubkey as optional
  Fix documentation for gnt-node evacuate -p option

Conflicts:
	Makefile.am (trivial)
	tools/cfgupgrade (removed downgrade code to 2.7)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Before doing any changes, make tools/users-setup explain its
actions first, and ask the user for confirmation. If the first
argument is --yes-do-it, this step will be skipped.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Now that we have a machine readable description of the users
to be added to the system, use it to generate the tools/users-setup
script, so that we again have only one authoritative source of
the user and group requirements. This also has the advantage of
of avoiding duplicate additions and attemps add a user root.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Upon build time, also generate, in the directory doc/users, files
containing the groups, the users with their primary group, and the
additional groupmemberships to be added. In this way, packages can use
their own way of adding users to the system, instead of telling the
adiministrator to run tools/users-setup.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In 'lib/constants.py', fix inconsistency in the value of
'AUTO_REPAIR_TAG_SUSPENDED'. Fixes issue 569.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update Python user separation constants to reference the generated
module 'lib/_constants.py' instead of 'lib/_autoconf.py'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add user separation constants, such as, 'daemonsGroup', 'rapiGroup',
and 'rapiUser', to the Haskell to Python constant generation
infrastructure.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add Makefile target to generate 'AutoConf.hs' and add this file to
'.gitignore'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'AutoConf.hs.in' which is the template for generating the file
'AutoConf.hs'.  This file mimics the Python equivalent
'lib/_autoconf.py' and it is necessary for Haskell constants to be
able to retrieve their values from 'configure'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Update constant 'ADMINST_ALL' to take its value from the generated
'lib/_constants.py' module.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add constant 'adminstAll' to Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add type 'FrozenSet' that wrap a Haskell 'Set', and a guarded
constructor for this type and a 'PyValue' instance.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'Ganeti.Hs2Py.ListContants.hs.in' template module documentation to
warn about automatic module generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'lib/_constants.py.in' documentation to be consistent with other
modules.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'HsConstants' module documentation to warn about proper module
usage.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In what Haskell to Python opcodes are concerned, a Haskell 'Set' is
translated into a Python 'list'.  In other words, currently, opcodes
that handle sets of parameters are actually handling lists because
this is how sets are currently encoded.  This patch introduces a new
type called 'ListSet' that wraps a Haskell 'Set' and it is used to
represent on the Haskell side a Python 'list' without duplicate
elements.  This patch also updates the respective opcode parameters
and updates the opcode tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix 'List' instance of 'PyValue' to properly convert Haskell lists to
Python lists.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

In "lib/cmdlib/node.py", in lines 1315-1316, 1354, and 1359-1370,
replace string literals with constants.  Fixes issue 558.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>