GitLab

Add missing exports in 'Ganeti.Types' which are necessary for
'Ganeti.HsConstants'.  These exports concern the '*ToRaw' functions
which convert a Haskell type into JSON.  These functions are needed by
the constants which we want to generate from Haskell to Python based
on Haskell types.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add xen related constants, such as, 'xenBootloader' and 'xenCmdXm' to
the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'devConsole' and syslog related constants to the Haskell to Python
constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'declareLADT' in Template Haskell module to declare Haskell
datatypes using 'String's directly as values for the JSON
serialization, as opposed to 'Name's which is what the current
'declareADT' allows.  To achieve this, 'genFromRaw' must be
generalized, similarly to 'genToRaw'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update Python references to exit code constants to take their values
from the generated 'lib/_constants.py'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add exit code constants to Haskell to Python constant generation
infrastructure.  The module 'Ganeti.ConstantUtils' must be imported
qualified in order to avoid a clash between
'Ganeti.ConstantUtils.exitFailure' and
'Ganeti.HsConstants.exitFailure'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update Python reference of 'DAEMONS_LOGBASE' to fetch its value from
the generated 'lib/_constants.py' module.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Move constant 'DAEMONS_LOGBASE' from Python to Haskell to be
automatically generated.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Before this patch, Haskell types, such as, 'GanetiDaemon' and
'GanetiGroup', and related functions were taking their values from
Haskell constants.  However, given that the role of Haskell to Python
constants is to leverage Haskell and its typesystem, it makes sense to
first define the Haskell types and then have the constants depend on
these types.  In other words, this patch series inverts the dependency
between (some) Haskell types and constants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Given that Haskell has its own generated 'AutoConf' module, it makes
sense that other Haskell modules import 'configure' variables directly
from 'AutoConf' instead of importing the Python to Haskell generated
constants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix missing dependency on target 'src/hs2py-constants' in
'Ganeti.ConstantUtils'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds tests for the remaining functionality of LUNodeAdd.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds node_unittest.py containing the framework for unit
testing LUNodeAdd.
At this point, only test setup and tests for OpenvSwitch are
implemented.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Recently, some IP addresses were changed to TEST-NETs from RFC 5737.
This patch changes some more occurences of wrong IPs and replaces them.

TEST-NET-1 is used for primary_ips, TEST-NET-2 for networks and TEST-NET-3 for
secondary_ips.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch enables patching the rpc module to create a mocked version
which can be used to mock a rpc.DnsOnlyRunner(). This is needed for
unit testing LUNodeAdd, as it need to run RPCs against nodes not yet
present in the configuration.

Also, the default return IP of _GetHostnameMock needs to be changed, since
x.x.x.1 is the default IP of the first mocked node and will cause problems.

Parts of this patch were written by Thomas Thrainer.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Haskell constants 'daemonsExtraLogfilesGanetiMondAccess' and
'daemonsExtraLogfilesGanetiMondError' cannot be constants because
their Python counterparts are calculated through
'pathutils.GetLogFilename', which indirectly depends on the
environment variable 'GANETI_ROOTDIR', as part of the virtual cluster
configuration.  Instead, these paths must be computed at runtime, as
opposed to being computed at compile time through the Python to
Haskell constant generation, and must also depend on the same
environment variable.  Fixes issue 575.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch fixes one lint error introduced by my recent patch to
fix keyerrors in lib/cmdlib/node.py.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Our tests are a form of documentation, and hence should use
IPs from the TEST-NET ranges according to RFC 5737. They definitely
should not use real public IP addresses.

Similarly, example host names should be from one of the reserved
domains according to RFC 2606.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch fixes potential key errors in the OpenvSwitch implementation
in cmdlib/node.py. The checks in that file expected to have a ndparams dict
with certain keys to be present. This should not be the case as it will
result in KeyErrors when those keys are not present.

Also: Don't assume that ndparams is given at all.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Update the upgrade design document according to the outcome
of the design discussion at GanetiCon 2013. There are two
changes in the design.

* As revision upgrades are binary compatible at all levels,
  one can accept a rough upgrade, just replacing the binaries.
  Therefore, it will become a configure option whether version
  includes revision and suffix, or not.

* The install/uninstall scripts are replaced in favor of another
  level of indirection. In this way, at run time, only two symlinks
  in ${sysconfdir}/ganeti, will be changed, whereas links under
  ${PREFIX}/bin, ${PREFIX}/sbin, and similar, are not touched by
  ganeti itself.

* The layout for the ganeti python libraries, which do not provide
  stable interface, are changed to be used as private modules.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Modify CPUload data collector to get the number of clock ticks
per second from Posix.Unistd. Since CPUload collector converts
clock ticks (jiffies) to seconds and the clock ticks to seconds
ratio varies between linux distributions, it must be read from
the corresponding library to be consistent.
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In the example, we have two nodes and 6 instances, all using external
storage. There are four small instances on one node, and two big
instances, requiring twice the amount of resources of a small instance,
on the other node. So, with respect to static data, the cluster
is already perfectly balanced. Using the default assumption of
dynamic usage, however, the cluster can be improved by moving
one small instance to the other node. The reason is, that by default,
all instances are assumed to use the same resources, and this measure
dominates, as all static resources are greatly underutilized.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This will allow to do balancing based only on static information.
In particular, the effect of balancing towards same number of
instances per node that the dynamic usage consideration has
if no usage file is provided, can be turned off, if this is
desired.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Make the external data loader correctly honor the --ignore-dynu option
by clearing the dynamic utilisation data from all instances if the option
is given.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Using this utility function, the dynamic utilisation of all instances
can be set to 0. This will have the effect of all utilisation being ignored.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This option, if set, will tell htools to ignore any dynamic
use data.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This patch provides rudimentary unit test coverage for LUInstanceQuery
and LUInstanceQueryData.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds rudimentary unit test coverage for LUInstanceMigrate and
LUInstanceFailover.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch provides unit test coverage for LUInstanceChangeGroup.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Those two occurrences of warnings can be replaced the RpcResult.Warn
method, which is done in this patch.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The validation of the opcodes already checks the size parameter for the
right type, so this except block is not required.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Rename the "node" variable to "node_uuid" as that's the content it
actually receives.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds unit tests for LUInstanceSetParams. It does not provide
100% coverage, but covers a significant portion.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The netmask was previously wrongly given as a bitmask, now it's given as
the net prefix length.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix target 'src/hs2py-constants' because it must depend on
'src/AutoConf.hs'. Fixes issue 573.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.9
  Fix bridging in net-common
  Sync build_chroot with buildbot slack role
  Auto-upgrade of disks' config wrt LD-renaming
  Fix tests regarding DISK_LD_DEFAULTS
  Fixing renaming of DISK_LD_DEFAULTS
  Replace LD_* constants with DT_* constants
  QA: fix file storage QA wrt ipolicy
  Bump versions to release 2.9~beta1
  Add design-2.9 to docinput
  NEWS update and version bump for 2.8 rc2
  Lint improvements to regexps
  Add unit test for GetLinuxNodeInfo
  Prepare GetLinuxNodeInfo for testing
  Check right disk template in inst set params
  Make the DRBD collector more failure-resilient
  Add function to unwrap Results logging failures
  Fix bug in NodeD and RapiD usage strings

* stable-2.8
  Make tools/users-setup explain its actions first
  Generate tools/users-setup from descriptions
  Provide ganeti user and group data in easy-to-parse form
  Remove dsahostkeypub during config downgrade
  Mark the DSA host pubkey as optional
  Fix documentation for gnt-node evacuate -p option

Conflicts:
	Makefile.am (trivial)
	devel/build_chroot (took changes from 2.9 but newer regex-pcre
                        version)
	lib/cmdlib/cluster.py (followed LD_* -> DT_* change)
	lib/objects.py (followed LD_* -> DT_* change)
	test/py/ganeti.cmdlib_unittest.py (followed LD_* -> DT_* change)

The LD_* -> DT_* change required some more changes in the cmdlib unit
tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update reference to 'DAEMONS_PORTS' and related constants that are
used on the Haskell side to construct the 'daemonsPorts' constant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* add 'daemonsPorts' and related constants used in the construction of
  the 'Map' that maps each daemon to its corresponding protocol and
  port.  This breaks constant 'daemonsPortsGanetiNoded' which is
  automatically generated from the Python dictionary 'DAEMONS_PORTS'

* replace reference from constant 'daemonsPortsGanetiNoded' to
  'defaultNodedPort' because constant 'daemonsPortsGanetiNoded' was a
  tuple containing the protocol and the port for node daemon, but only
  the port was needed.  And because this tuple was making Haskell to
  Python constant generation more difficult, it is preferable to
  replace this tuple with the actual port and use it directly, in
  module 'Ganeti.Rpc'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add 'Protocol' datatype which is necessary for the 'daemonsPorts'
constant.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix instances of 'PyValue', namely, for tuples and maps.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>