GitLab

Switching from python to haskell queries, this patch
removes the option to dis/enable the haskell queries
at configure time.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The QA test 'TestClusterBurnin' pauses instances and the Xen state
domain 'paused' was not being handled properly by Ganeti because it
was not being classified as either a running or shutdown state.  This
patch adds the 'paused' state to the set of running states by
extending the function 'ganeti.hypervisor.hv_xen._IsInstanceRunning'
with the corresponding Xen state string '--p---'.

This fixes the QA test 'TestClusterBurnin'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The shelltests expected the wrong position of the -f option
as well. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

hsqueeze can produce a shell script with the commands to squeeze
the cluster; in the script, fix the position of the '-f' option
in the 'gnt-node power' command.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

The newest version does not build on debian squeeze, so avoid
it beeing pulled in as a dependency.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

This file is used for development to point to code definitions. Apparently this
has been broken since Februrary [cc40185c]. The improved version fixes the
problem and uses Exuberant Ctags <http://ctags.sourceforge.net/

>. It uses
"etags" instead of "ctags", so Emacs compatibility should be maintained.

The extra suffixes "-osuf" and "hisuf" are used to avoid problems with already
compiled object files.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

In order to work, it is necessary:
* Write a JSON Patch to file "qa/qa-patch.json" in a source repository.
  (And be careful not to commit it!) The file must conform to RFC6902
  and describes what changes should be done to the QA JSON
  configuration. This is simple as
   [
     { "op": "add",
       "path": "default",
       "value": false },
     { "op": "add",
       "path": "/tests/instance-add-file",
       "value": true }
   ]

* Install "jsonpatch" and "jsonpointer" modules on the machines QA is
  running on. Either using "easy_install" or as Debian packages using
  "pypi-install".

QA only loads the modules if the patch file is present and non-empty.
Therefore no changes to buildbots are needed, if the feature is not
used.

It's advisable to add a git pre-commit hook for the main repository to
prevent changes to "qa/patch.json", keeping it to "[]". Suggestion for
such a hook:

  # Exit code 1 if the patch file isn't just "[]":
  python -c 'import json, sys; \
    exit(0 if json.load(open("qa/qa-patch.json", "r")) == [] else 1)'

Locally it's possible to tell git not to track changes to the file at
all (see https://help.github.com/articles/ignoring-files

):

  git update-index --assume-unchanged qa/qa-patch.json
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When spindles where added to Ganeti, apparently it was forgotten
to add the parameter to the Haskell data type as well. Do this now.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

The disk parameter "spindles", if present, has to be an integer.
Enforce this already when parsing the command line, instead of
passing around the string representation of that integer.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

If any of the nodes involved would be reached through IPv6, the test is
skipped. This is because the required testing setup using "iptables" is
suiteble only for IPv4 and doesn't work for IPv6 connections.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The option -C was recently added to hsqueeze, but not documented in
the man page. Add the documentation now.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The NIC VLAN has previously not been modified via Haskell, causing the
INicParams class not to be used. With the recent job queue
refactorings, a modification definition is recorded, and for an empty
string (which is a legal default value) a crash happens. This patch
fixes this by allowing VLANS to be empty strings on the Haskell side.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Allow the retrieval of the VLANs of all the NICs through nic.vlans.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

The field was added to Python queries in an earlier version, and now
has to be added to the Haskell queries as well.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Similarly to simple tests, test blocks are now demarcated (in boldface)
and measured.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Allow combination of colors, background colors, bold, reverse video,
etc.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

It uses `tput` to check if the current terminal has enough colors.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

In the examples, pick a node that certainly has to be powered on or off,
respectively, and verify that the power command is there, as well
an appropriate migration command in the right position.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add an option to hsqueeze to show, or save in a file, the commands
that have to be carried out.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In hsqueeze, when computing the balancing sequence, also
remember the sequence of moves that lead there.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add a function that, given two adjacent cluster configurations of
a balancing sequence, computes the moves that led from the first
to the second configuration.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

When grouping moves into jobs, a new job set is started, if the new
move involves a node also touched by a previous move. When computing
the list of involved nodes, the new primary and secondary nodes of the
instance are included; if an instance, however, has only one node, the
"no secondary" index -1 should be left out to avoid spurious conflicts
between unrelated moves of single-homed instances.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Move the function that saves a list of a command in a file
to CLI.hs. In this way, it is reusable by other htools.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Merge branch 'stable-2.10' into master

* stable-2.10
  Don't allow optional node parameters
  Move OVS node parameters to the right place
  Make NIC VLAN queryable
  Pass VLAN parameter correctly in moves
  Use constant instead of raw string
  Merge 'hs2py-constants' into 'hs2py'
  Add flag '--opcodes' to 'hs2py'
  Sort design docs alphabetically
  Missing design docs from distribution
  When loading configuration fails, include the reason
  Fixing rapi documentation wrt to storage types
  Move Haskell constants to proper module
  Tear down Py2Hs constant infrastructure
  Hs2Py constants: additional module jstore
  Hs2Py constants: additional module errors
  Hs2Py constants: additional module qlang
  Hs2Py constants: additional module luxi
  Remove module 'ganeti.constants' from 'convert-constants'
  Fix reference to vcs version in query server
  Eliminate unused constants
  Use configure constants instead of generated
  Hs2Py constants: add 'UUID_REGEX'
  Add 'AF_INET*' constants to Haskell's 'AutoConf'
  Python autotool to print socket constants

* stable-2.9
  Bump revision for 2.9.1
  Update NEWS and schedule release for 2.9.1
  Fix retrieval of xen command in class method
  Fix docstring for ganeti.storage.filestorage_unittest.py
  Undo revision bump
  Postpone release of 2.9.1
  Revision bump for 2.9.1
  Update NEWS for 2.9.1 release
  Readd nodes as online
  When verifying BRBD version, ignore missing values

* stable-2.8
  Version bump for 2.8.2
  Update NEWS file for 2.8.2 release
  DRBD: ensure peers are UpToDate for dual-primary

Conflicts:
	Makefile.am: Add design docs from both sides (sorted); Remove
                 rules for hs2py-constants
	lib/hypervisor/hv_xen.py: Merge parameter list of
                              GetInstanceConsole
	lib/luxi.py: Use generated constant for REQ_PICKUP_JOB
	src/Ganeti/ConfigReader.hs: trivial
	src/Ganeti/HsConstants.hs: Manually add UserDown and ndSshPort
                               related constants to Constants.hs
	src/Ganeti/Query/Server.hs: trivial
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  Bump revision for 2.9.1
  Update NEWS and schedule release for 2.9.1
  Fix retrieval of xen command in class method
  Fix docstring for ganeti.storage.filestorage_unittest.py
  Undo revision bump
  Postpone release of 2.9.1
  Revision bump for 2.9.1
  Update NEWS for 2.9.1 release
  Readd nodes as online
  When verifying BRBD version, ignore missing values

* stable-2.8
  Version bump for 2.8.2
  Update NEWS file for 2.8.2 release
  DRBD: ensure peers are UpToDate for dual-primary

Conflicts:
    NEWS: trivial
	configure.ac: ignore version bump
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Before, calls to `gnt-instance list -o console` with an instance on a
node with a custom SSH port failed because of missing group
configuration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The test is turned on by setting `"group-custom-ssh-port": true` in
the config.

Unlike `"ssh-port": ...` in "options", this is a single test that
doesn't affect the rest of the tests.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This is needed for testing SSH ports so that we can remove a node from
a group, add it to a group with a custom SSH port, and then undo the
change.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

For acquiring nodes use `with AcquireManyNodes(num): ...`. The nodes
will be released automatically.
For creating a new group use `with NewGroupCtx() as group: ...`. The
group will be removed automatically.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

If SSH ports other than 22 are configured, abort the config downgrade,
as the operation would certainly fail. Otherwise just remove
_ndparams/ssh_port_ from the configuration.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Added to the other node parameters in ganeti.rst.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This patch adds the `ssh-port` option. If set to a non-standard port,
the QA script sets up the default node group with this port, and before
running tests it adds `iptable` rules to all nodes so that the nodes
see each other's SSH servers as running on this port.
Their SSH configuration is _not_ changed and other machines see the
nodes on 22 as before.

The `iptable` rules are reset on each QA run, trying to preserve any
existing rules (not created by the script) that might be present.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In particular _copyfile_ and _renew-crypto_.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This is accomplished by passing the corresponding node group to
hv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The unit tests now include the configurations of SSH ports for node
groups/nodes.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The parameter is added to Haskell sources, from which the corresponding
Python code is generated.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Also add a method for the retrieval of "ndparams" (node parameters)
of groups, not just nodes.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

In an effort to get rid of the python queries soon, this
patch fixes a FIXME of the group queries regarding the
missing testing of niceSort in this context. Due to the
lack of actually weirdly named hostnames, this patch
adds an extra unit test for hostnames instead.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>