devnotes.rst 8.32 KB
Newer Older
1 2 3
Developer notes
===============

4 5
.. highlight:: shell-example

6 7 8
Build dependencies
------------------

9 10
Most dependencies from :doc:`install-quick`, including ``qemu-img``
(marked there as optional) plus (for Python):
11

12 13 14
- `GNU make <http://www.gnu.org/software/make/>`_
- `GNU tar <http://www.gnu.org/software/tar/>`_
- `Gzip <http://www.gnu.org/software/gzip/>`_
15
- `pandoc <http://johnmacfarlane.net/pandoc/>`_
16
- `python-epydoc <http://epydoc.sourceforge.net/>`_
17
- `python-sphinx <http://sphinx.pocoo.org/>`_
18
  (tested with version 1.1.3)
19
- `python-mock <http://www.voidspace.org.uk/python/mock/>`_
20
  (tested with version 1.0.1)
21
- `graphviz <http://www.graphviz.org/>`_
22
- the `en_US.UTF-8` locale must be enabled on the system
23 24
- `pylint <http://www.logilab.org/857>`_ and its associated
  dependencies
25
- `pep8 <https://github.com/jcrocholl/pep8/>`_
26
- `PyYAML <http://pyyaml.org/>`_
27

28
For older developement (Ganeti < 2.4) ``docbook`` was used instead of
29 30
``pandoc``.

31
Note that for pylint, at the current moment the following versions
32
must be used::
33 34

    $ pylint --version
35 36
    pylint 0.26.0,
    astng 0.24.1, common 0.58.3
37

38 39 40
The same with pep8, other versions may give you errors::

     $ pep8 --version
41
     1.3.3
42

43
Both these versions are the ones shipped with Ubuntu 13.04.
44

45 46 47
To generate unittest coverage reports (``make coverage``), `coverage
<http://pypi.python.org/pypi/coverage>`_ needs to be installed.

48 49
Installation of all dependencies listed here::

50
     $ apt-get install python-setuptools automake git fakeroot
51
     $ apt-get install pandoc python-epydoc graphviz python-sphinx
52
     $ apt-get install python-yaml
53
     $ cd / && easy_install \
54 55 56 57
               logilab-astng==0.24.1 \
               logilab-common==0.58.3 \
               pylint==0.26.0 \
               pep8==1.3.3 \
58
               mock==1.0.1 \
59 60
               coverage

61 62 63 64 65 66 67 68
For Haskell development, again all things from the quick install
document, plus:

- `haddock <http://www.haskell.org/haddock/>`_, documentation
  generator (equivalent to epydoc for Python)
- `HsColour <http://hackage.haskell.org/package/hscolour>`_, again
  used for documentation (it's source-code pretty-printing)
- `hlint <http://community.haskell.org/~ndm/hlint/>`_, a source code
Iustin Pop's avatar
Iustin Pop committed
69
  linter (equivalent to pylint for Python), recommended version 1.8 or
70
  above (tested with 1.8.43)
71
- the `QuickCheck <http://hackage.haskell.org/package/QuickCheck>`_
72
  library, version 2.x
73 74 75 76 77
- the `HUnit <http://hunit.sourceforge.net/>`_ library (tested with
  1.2.x)
- the `test-framework
  <http://batterseapower.github.com/test-framework/>`_ libraries,
  tested versions: ``test-framework``: 0.6, ``test-framework-hunit``:
78
  0.2.7, ``test-framework-quickcheck2``: 0.2.12.1
79 80
- ``hpc``, which comes with the compiler, so you should already have
  it
81
- `shelltestrunner <http://joyful.com/shelltestrunner>`_, used for
82
  running shell-based unit-tests
83 84
- `temporary <https://github.com/batterseapower/temporary/>`_ library,
  tested with version 1.1.2.3
85 86 87

Under Debian Wheezy or later, these can be installed (on top of the
required ones from the quick install document) via::
88

89 90 91 92
  $ apt-get install libghc-quickcheck2-dev libghc-hunit-dev \
        libghc-test-framework-dev \
        libghc-test-framework-quickcheck2-dev \
        libghc-test-framework-hunit-dev \
93
        libghc-temporary-dev shelltestrunner \
94
        hscolour hlint
95

96 97
Or alternatively via ``cabal``::

98 99
  $ cabal install QuickCheck HUnit \
          test-framework test-framework-quickcheck2 test-framework-hunit \
100
          temporary hscolour hlint shelltestrunner
101

102

103 104 105
Configuring for development
---------------------------

106 107 108
Run the following command (only use ``PYTHON=...`` if you need to use a
different python version)::

109 110
  $ ./autogen.sh && \
    ./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var
111

112 113 114 115 116 117 118
Note that doing development on a machine which already has Ganeti
installed is problematic, as ``PYTHONPATH`` behaviour can be confusing
(see Issue 170 for a bit of history/details; in general it works if
the installed and developed versions are very similar, and/or if
PYTHONPATH is customised correctly). As such, in general it's
recommended to use a "clean" machine for ganeti development.

119 120 121 122 123
Style guide
-----------

Please adhere to the :doc:`dev-codestyle` while writing code for Ganeti.

124 125 126 127 128 129 130 131
Haskell development notes
-------------------------

There are a few things which can help writing or debugging the Haskell
code.

You can run the Haskell linter :command:`hlint` via::

132
  $ make hlint
133

Iustin Pop's avatar
Iustin Pop committed
134 135 136
This is not enabled by default (as the htools component is
optional). The above command will generate both output on the terminal
and, if any warnings are found, also an HTML report at
137 138 139 140 141
``doc/hs-lint.html``.

When writing or debugging TemplateHaskell code, it's useful to see
what the splices are converted to. This can be done via::

142
  $ make HEXTRA="-ddump-splices"
143

144 145 146 147
Or, more interactively::

  $ ghci
  λ> :set -ddump-splices
Iustin Pop's avatar
Iustin Pop committed
148
  λ> :l src/Ganeti/Objects.hs
149 150 151

And you will get the spliced code as the module is loaded.

152 153 154 155 156 157 158
To build profiling code you must install the ``ghc-prof`` (or
``gch6-prof``) package, and all the relevant libraries with their
``-prof`` counterparts. If installing libraries through cabal the config
file should include ``library-profiling: True`` or the ``-p`` flag
should be used. Any library already installed can be updated by passing
``--reinstall`` as well.

159
Due to the way TemplateHaskell works, it's not straightforward to
160 161
build profiling code. The recommended way is to run ``make hs-prof``,
or alternatively the manual sequence is::
162

163
  $ make clean
Iustin Pop's avatar
Iustin Pop committed
164 165 166
  $ make src/htools HEXTRA="-osuf .o"
  $ rm src/htools
  $ make src/htools HEXTRA="-osuf .prof_o -prof -auto-all"
167 168 169 170

This will build the binary twice, per the TemplateHaskell
documentation, the second one with profiling enabled.

171 172 173 174 175 176 177 178
The binary files generated by compilation and the profiling/coverage
files can "break" tab-completion in the sources; they can be ignored,
for example, in bash via ``.bashrc``::

  FIGNORE='.o:.hi:.prof_o:.tix'

or in emacs via ``completion-ignored-extensions`` (run ``M-x
customize-var completion-ignored-extensions``).
179

180 181 182 183
Running individual tests
~~~~~~~~~~~~~~~~~~~~~~~~

When developing code, running the entire test suite can be
184 185
slow. Running individual tests is possible. There are different
Makefile targets for running individual Python and Haskell tests.
186 187 188 189

For Python tests::

  $ export PYTHONPATH=$PWD
190
  $ python ./test/py/ganeti.%mytest%
191 192 193

For Haskell tests::

194
  $ make hs-test-%pattern%
195 196 197 198

Where ``pattern`` can be a simple test pattern (e.g. ``comma``,
matching any test whose name contains ``comma``), a test pattern
denoting a group (ending with a slash, e.g. ``Utils/``), or more
199 200
complex glob pattern. For more details, search for glob patterns in
the documentation of `test-framework
201 202
<http://batterseapower.github.com/test-framework/>`_).

203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220
For individual Haskell shelltests::

  $ make hs-shell-%name%

which runs the test ``test/hs/shelltests/htools-%name%.test``. For
example, to run the test ``test/hs/shelltests/htools-balancing.test``,
use::

  $ make hs-shell-balancing

For combined Haskell shelltests::

  $ make hs-shell-{%name1%,%name2%,...}

for example::

  $ make hs-shell-{balancing,basic}

221 222 223 224
Checking for the correct style of the NEWS file is also possible, by running::

  $ make check-news

225 226 227
Packaging notes
===============

228
Ganeti is mostly developed and tested on `Debian
229
<http://www.debian.org/>`_-based distributions, while still keeping
230
adaptability to other Linux distributions in mind.
231 232 233 234 235 236 237 238 239 240

The ``doc/examples/`` directory contains a number of potentially useful
scripts and configuration files. Some of them might need adjustment
before use.

``daemon-util``
---------------

This script, in the source code as ``daemons/daemon-util.in``, is used
to start/stop Ganeti and do a few other things related to system
241
daemons. It is recommended to use ``daemon-util`` also from the system's
242 243 244 245 246 247 248 249 250 251 252 253
init scripts. That way the code starting and stopping daemons is shared
and future changes have to be made in only one place.

``daemon-util`` reads extra arguments from variables (``*_ARGS``) in
``/etc/default/ganeti``. When modifying ``daemon-util``, keep in mind to
not remove support for the ``EXTRA_*_ARGS`` variables for starting
daemons. Some parts of Ganeti use them to pass additional arguments when
starting a daemon.

The ``reload_ssh_keys`` function can be adjusted to use another command
for reloading the OpenSSH daemon's host keys.

254
.. vim: set textwidth=72 :