install.rst 23.6 KB
Newer Older
1
2
3
Ganeti installation tutorial
============================

Iustin Pop's avatar
Iustin Pop committed
4
Documents Ganeti version |version|
5
6
7

.. contents::

Iustin Pop's avatar
Iustin Pop committed
8
9
.. highlight:: text

10
11
12
13
Introduction
------------

Ganeti is a cluster virtualization management system based on Xen or
Iustin Pop's avatar
Iustin Pop committed
14
15
16
17
18
19
KVM. This document explains how to bootstrap a Ganeti node (Xen *dom0*,
the host Linux system for KVM), create a running cluster and install
virtual instances (Xen *domUs*, KVM guests).  You need to repeat most of
the steps in this document for every node you want to install, but of
course we recommend creating some semi-automatic procedure if you plan
to deploy Ganeti on a medium/large scale.
20
21

A basic Ganeti terminology glossary is provided in the introductory
Iustin Pop's avatar
Iustin Pop committed
22
23
section of the :doc:`admin`. Please refer to that document if you are
uncertain about the terms we are using.
24

Iustin Pop's avatar
Iustin Pop committed
25
Ganeti has been developed for Linux and should be distribution-agnostic.
26
This documentation will use Debian Lenny as an example system but the
Iustin Pop's avatar
Iustin Pop committed
27
28
29
examples can be translated to any other distribution. You are expected
to be familiar with your distribution, its package management system,
and Xen or KVM before trying to use Ganeti.
30
31
32
33
34
35
36
37

This document is divided into two main sections:

- Installation of the base system and base components

- Configuration of the environment for Ganeti

Each of these is divided into sub-sections. While a full Ganeti system
Iustin Pop's avatar
Iustin Pop committed
38
39
40
will need all of the steps specified, some are not strictly required for
every environment. Which ones they are, and why, is specified in the
corresponding sections.
41
42
43
44
45
46
47

Installing the base system and base components
----------------------------------------------

Hardware requirements
+++++++++++++++++++++

Iustin Pop's avatar
Iustin Pop committed
48
49
Any system supported by your Linux distribution is fine. 64-bit systems
are better as they can support more memory.
50

Iustin Pop's avatar
Iustin Pop committed
51
52
53
54
55
Any disk drive recognized by Linux (``IDE``/``SCSI``/``SATA``/etc.) is
supported in Ganeti. Note that no shared storage (e.g. ``SAN``) is
needed to get high-availability features (but of course, one can be used
to store the images). It is highly recommended to use more than one disk
drive to improve speed. But Ganeti also works with one disk per machine.
56
57
58
59
60
61
62

Installing the base system
++++++++++++++++++++++++++

**Mandatory** on all nodes.

It is advised to start with a clean, minimal install of the operating
Iustin Pop's avatar
Iustin Pop committed
63
64
65
66
67
system. The only requirement you need to be aware of at this stage is to
partition leaving enough space for a big (**minimum** 20GiB) LVM volume
group which will then host your instance filesystems, if you want to use
all Ganeti features. The volume group name Ganeti uses (by default) is
``xenvg``.
68

Iustin Pop's avatar
Iustin Pop committed
69
70
You can also use file-based storage only, without LVM, but this setup is
not detailed in this document.
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108

While you can use an existing system, please note that the Ganeti
installation is intrusive in terms of changes to the system
configuration, and it's best to use a newly-installed system without
important data on it.

Also, for best results, it's advised that the nodes have as much as
possible the same hardware and software configuration. This will make
administration much easier.

Hostname issues
~~~~~~~~~~~~~~~

Note that Ganeti requires the hostnames of the systems (i.e. what the
``hostname`` command outputs to be a fully-qualified name, not a short
name. In other words, you should use *node1.example.com* as a hostname
and not just *node1*.

.. admonition:: Debian

   Debian Lenny and Etch configures the hostname differently than you
   need it for Ganeti. For example, this is what Etch puts in
   ``/etc/hosts`` in certain situations::

     127.0.0.1       localhost
     127.0.1.1       node1.example.com node1

   but for Ganeti you need to have::

     127.0.0.1       localhost
     192.168.1.1     node1.example.com node1

   replacing ``192.168.1.1`` with your node's address. Also, the file
   ``/etc/hostname`` which configures the hostname of the system
   should contain ``node1.example.com`` and not just ``node1`` (you
   need to run the command ``/etc/init.d/hostname.sh start`` after
   changing the file).

109
110
.. admonition:: Why a fully qualified host name

111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
   Although most distributions use only the short name in the
   /etc/hostname file, we still think Ganeti nodes should use the full
   name. The reason for this is that calling 'hostname --fqdn' requires
   the resolver library to work and is a 'guess' via heuristics at what
   is your domain name. Since Ganeti can be used among other things to
   host DNS servers, we don't want to depend on them as much as
   possible, and we'd rather have the uname() syscall return the full
   node name.

   We haven't ever found any breakage in using a full hostname on a
   Linux system, and anyway we recommend to have only a minimal
   installation on Ganeti nodes, and to use instances (or other
   dedicated machines) to run the rest of your network services. By
   doing this you can change the /etc/hostname file to contain an FQDN
   without the fear of breaking anything unrelated.
126
127


128
129
Installing The Hypervisor
+++++++++++++++++++++++++
130
131
132

**Mandatory** on all nodes.

133
While Ganeti is developed with the ability to modularly run on different
134
135
136
virtualization environments in mind the only two currently useable on a
live system are Xen and KVM. Supported Xen versions are: 3.0.3, 3.0.4
and 3.1.  Supported KVM version are 72 and above.
137

Iustin Pop's avatar
Iustin Pop committed
138
139
140
141
Please follow your distribution's recommended way to install and set up
Xen, or install Xen from the upstream source, if you wish, following
their manual. For KVM, make sure you have a KVM-enabled kernel and the
KVM tools.
142

143
After installing Xen, you need to reboot into your new system. On some
144
145
146
distributions this might involve configuring GRUB appropriately, whereas
others will configure it automatically when you install the respective
kernels. For KVM no reboot should be necessary.
147

148
.. admonition:: Xen on Debian
149

Iustin Pop's avatar
Iustin Pop committed
150
151
152
153
154
   Under Lenny or Etch you can install the relevant ``xen-linux-system``
   package, which will pull in both the hypervisor and the relevant
   kernel. Also, if you are installing a 32-bit Lenny/Etch, you should
   install the ``libc6-xen`` package (run ``apt-get install
   libc6-xen``).
155
156
157
158
159

Xen settings
~~~~~~~~~~~~

It's recommended that dom0 is restricted to a low amount of memory
Iustin Pop's avatar
Iustin Pop committed
160
161
162
(512MiB or 1GiB is reasonable) and that memory ballooning is disabled in
the file ``/etc/xen/xend-config.sxp`` by setting the value
``dom0-min-mem`` to 0, like this::
163
164
165
166

  (dom0-min-mem 0)

For optimum performance when running both CPU and I/O intensive
Iustin Pop's avatar
Iustin Pop committed
167
168
instances, it's also recommended that the dom0 is restricted to one CPU
only, for example by booting with the kernel parameter ``nosmp``.
169
170
171
172
173
174

It is recommended that you disable xen's automatic save of virtual
machines at system shutdown and subsequent restore of them at reboot.
To obtain this make sure the variable ``XENDOMAINS_SAVE`` in the file
``/etc/default/xendomains`` is set to an empty value.

Guido Trotter's avatar
Guido Trotter committed
175
176
177
If you want to use live migration make sure you have, in the xen config
file, something that allows the nodes to migrate instances between each
other. For example::
178
179
180
181
182
183

  (xend-relocation-server yes)
  (xend-relocation-port 8002)
  (xend-relocation-address '')
  (xend-relocation-hosts-allow '^192\\.168\\.3\\.[0-9]+$')

184
185
186
187

The second line assumess that the hypervisor parameter
``migration_port`` is set 8002, otherwise modify it to match. The last
line assumes that all your nodes have secondary IPs in the
188
189
192.168.3.0/24 network, adjust it accordingly to your setup.

190
191
192
193
194
195
196
197
198
199
.. admonition:: Debian

   Besides the ballooning change which you need to set in
   ``/etc/xen/xend-config.sxp``, you need to set the memory and nosmp
   parameters in the file ``/boot/grub/menu.lst``. You need to modify
   the variable ``xenhopt`` to add ``dom0_mem=1024M`` like this::

     ## Xen hypervisor options to use with the default Xen boot option
     # xenhopt=dom0_mem=1024M

Iustin Pop's avatar
Iustin Pop committed
200
   and the ``xenkopt`` needs to include the ``nosmp`` option like this::
201
202
203
204
205
206
207
208
209
210

     ## Xen Linux kernel options to use with the default Xen boot option
     # xenkopt=nosmp

   Any existing parameters can be left in place: it's ok to have
   ``xenkopt=console=tty0 nosmp``, for example. After modifying the
   files, you need to run::

     /sbin/update-grub

Iustin Pop's avatar
Iustin Pop committed
211
212
If you want to run HVM instances too with Ganeti and want VNC access to
the console of your instances, set the following two entries in
213
214
215
216
217
218
219
220
221
222
223
224
``/etc/xen/xend-config.sxp``::

  (vnc-listen '0.0.0.0') (vncpasswd '')

You need to restart the Xen daemon for these settings to take effect::

  /etc/init.d/xend restart

Selecting the instance kernel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

After you have installed Xen, you need to tell Ganeti exactly what
Iustin Pop's avatar
Iustin Pop committed
225
226
227
228
229
kernel to use for the instances it will create. This is done by creating
a symlink from your actual kernel to ``/boot/vmlinuz-2.6-xenU``, and one
from your initrd to ``/boot/initrd-2.6-xenU`` [#defkernel]_. Note that
if you don't use an initrd for the domU kernel, you don't need to create
the initrd symlink.
230
231
232
233
234
235
236
237
238
239
240
241
242

.. admonition:: Debian

   After installation of the ``xen-linux-system`` package, you need to
   run (replace the exact version number with the one you have)::

     cd /boot
     ln -s vmlinuz-2.6.26-1-xen-amd64 vmlinuz-2.6-xenU
     ln -s initrd.img-2.6.26-1-xen-amd64 initrd-2.6-xenU

Installing DRBD
+++++++++++++++

Iustin Pop's avatar
Iustin Pop committed
243
244
245
246
Recommended on all nodes: DRBD_ is required if you want to use the high
availability (HA) features of Ganeti, but optional if you don't require
them or only run Ganeti on single-node clusters. You can upgrade a
non-HA cluster to an HA one later, but you might need to export and
247
248
249
250
re-import all your instances to take advantage of the new features.

.. _DRBD: http://www.drbd.org/

Iustin Pop's avatar
Iustin Pop committed
251
252
253
254
Supported DRBD versions: 8.0+. It's recommended to have at least version
8.0.12. Note that for version 8.2 and newer it is needed to pass the
``usermode_helper=/bin/true`` parameter to the module, either by
configuring ``/etc/modules`` or when inserting it manually.
255
256

Now the bad news: unless your distribution already provides it
Iustin Pop's avatar
Iustin Pop committed
257
258
259
installing DRBD might involve recompiling your kernel or anyway fiddling
with it. Hopefully at least the Xen-ified kernel source to start from
will be provided (if you intend to use Xen).
260
261

The good news is that you don't need to configure DRBD at all. Ganeti
Iustin Pop's avatar
Iustin Pop committed
262
263
264
265
266
267
268
269
270
271
will do it for you for every instance you set up.  If you have the DRBD
utils installed and the module in your kernel you're fine. Please check
that your system is configured to load the module at every boot, and
that it passes the following option to the module:
``minor_count=NUMBER``. We recommend that you use 128 as the value of
the minor_count - this will allow you to use up to 64 instances in total
per node (both primary and secondary, when using only one disk per
instance). You can increase the number up to 255 if you need more
instances on a node.

272
273
274

.. admonition:: Debian

Iustin Pop's avatar
Iustin Pop committed
275
276
277
   On Debian, you can just install (build) the DRBD module with the
   following commands, making sure you are running the target (Xen or
   KVM) kernel::
278
279
280
281

     apt-get install drbd8-source drbd8-utils
     m-a update
     m-a a-i drbd8
Iustin Pop's avatar
Iustin Pop committed
282
     echo drbd minor_count=128 usermode_helper=/bin/true >> /etc/modules
283
     depmod -a
Iustin Pop's avatar
Iustin Pop committed
284
     modprobe drbd minor_count=128 usermode_helper=/bin/true
285

Iustin Pop's avatar
Iustin Pop committed
286
287
288
   It is also recommended that you comment out the default resources in
   the ``/etc/drbd.conf`` file, so that the init script doesn't try to
   configure any drbd devices. You can do this by prefixing all
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
   *resource* lines in the file with the keyword *skip*, like this::

     skip resource r0 {
       ...
     }

     skip resource "r1" {
       ...
     }

Other required software
+++++++++++++++++++++++

Besides Xen and DRBD, you will need to install the following (on all
nodes):

- LVM version 2, `<http://sourceware.org/lvm2/>`_

- OpenSSL, `<http://www.openssl.org/>`_

- OpenSSH, `<http://www.openssh.com/portable.html>`_

- bridge utilities, `<http://bridge.sourceforge.net/>`_

- iproute2, `<http://developer.osdl.org/dev/iproute2>`_

- arping (part of iputils package),
  `<ftp://ftp.inr.ac.ru/ip-routing/iputils-current.tar.gz>`_

- Python version 2.4 or 2.5, `<http://www.python.org>`_

- Python OpenSSL bindings, `<http://pyopenssl.sourceforge.net/>`_

- simplejson Python module, `<http://www.undefined.org/python/#simplejson>`_

- pyparsing Python module, `<http://pyparsing.wikispaces.com/>`_

326
327
- pyinotify Python module, `<http://trac.dbzteam.org/pyinotify>`_

328
329
330
331
332
333
334
335
336
337
These programs are supplied as part of most Linux distributions, so
usually they can be installed via apt or similar methods. Also many of
them will already be installed on a standard machine.


.. admonition:: Debian

   You can use this command line to install all needed packages::

     # apt-get install lvm2 ssh bridge-utils iproute iputils-arping \
338
339
     python python-pyopenssl openssl python-pyparsing \
     python-simplejson python-pyinotify
340
341
342
343
344
345
346
347
348

Setting up the environment for Ganeti
-------------------------------------

Configuring the network
+++++++++++++++++++++++

**Mandatory** on all nodes.

349
350
351
352
353
354
You can run Ganeti either in "bridge mode" or in "routed mode". In
bridge mode, the default, the instances network interfaces will be
attached to a software bridge running in dom0. Xen by default creates
such a bridge at startup, but your distribution might have a different
way to do things, and you'll definitely need to manually set it up under
KVM.
355

Iustin Pop's avatar
Iustin Pop committed
356
357
358
359
Beware that the default name Ganeti uses is ``xen-br0`` (which was used
in Xen 2.0) while Xen 3.0 uses ``xenbr0`` by default. The default bridge
your Ganeti cluster will use for new instances can be specified at
cluster initialization time.
360

361
362
363
364
If you want to run in "routing mode" you need to specify that at cluster
init time (using the --nicparam option), and then no bridge will be
needed. In this mode instance traffic will be routed by dom0, instead of
bridged.
365

366
367
368
369
In order to use "routing mode" under Xen, you'll need to change the
relevant parameters in the Xen config file. Under KVM instead, no config
change is necessary, but you still need to set up your network
interfaces correctly.
370
371

By default, under KVM, the "link" parameter you specify per-nic will
372
373
374
375
represent, if non-empty, a different routing table name or number to use
for your instances. This allows insulation between different instance
groups, and different routing policies between node traffic and instance
traffic.
376

377
378
379
380
You will need to configure your routing table basic routes and rules
outside of ganeti. The vif scripts will only add /32 routes to your
instances, through their interface, in the table you specified (under
KVM, and in the main table under Xen).
381
382

.. admonition:: Bridging under Debian
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417

   The recommended way to configure the Xen bridge is to edit your
   ``/etc/network/interfaces`` file and substitute your normal
   Ethernet stanza with the following snippet::

     auto xen-br0
     iface xen-br0 inet static
        address YOUR_IP_ADDRESS
        netmask YOUR_NETMASK
        network YOUR_NETWORK
        broadcast YOUR_BROADCAST_ADDRESS
        gateway YOUR_GATEWAY
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

The following commands need to be executed on the local console:

  ifdown eth0
  ifup xen-br0

To check if the bridge is setup, use the ``ip`` and ``brctl show``
commands::

  # ip a show xen-br0
  9: xen-br0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue
      link/ether 00:20:fc:1e:d5:5d brd ff:ff:ff:ff:ff:ff
      inet 10.1.1.200/24 brd 10.1.1.255 scope global xen-br0
      inet6 fe80::220:fcff:fe1e:d55d/64 scope link
         valid_lft forever preferred_lft forever

  # brctl show xen-br0
  bridge name     bridge id               STP enabled     interfaces
  xen-br0         8000.0020fc1ed55d       no              eth0

Iustin Pop's avatar
Iustin Pop committed
418
419
.. _configure-lvm-label:

420
421
422
423
424
425
426
Configuring LVM
+++++++++++++++

**Mandatory** on all nodes.

The volume group is required to be at least 20GiB.

Iustin Pop's avatar
Iustin Pop committed
427
428
429
430
If you haven't configured your LVM volume group at install time you need
to do it before trying to initialize the Ganeti cluster. This is done by
formatting the devices/partitions you want to use for it and then adding
them to the relevant volume group::
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448

  pvcreate /dev/sda3
  vgcreate xenvg /dev/sda3

or::

  pvcreate /dev/sdb1
  pvcreate /dev/sdc1
  vgcreate xenvg /dev/sdb1 /dev/sdc1

If you want to add a device later you can do so with the *vgextend*
command::

  pvcreate /dev/sdd1
  vgextend xenvg /dev/sdd1

Optional: it is recommended to configure LVM not to scan the DRBD
devices for physical volumes. This can be accomplished by editing
Iustin Pop's avatar
Iustin Pop committed
449
450
``/etc/lvm/lvm.conf`` and adding the ``/dev/drbd[0-9]+`` regular
expression to the ``filter`` variable, like this::
451
452
453

  filter = ["r|/dev/cdrom|", "r|/dev/drbd[0-9]+|" ]

Iustin Pop's avatar
Iustin Pop committed
454
455
456
457
458
Note that with Ganeti a helper script is provided - ``lvmstrap`` which
will erase and configure as LVM any not in-use disk on your system. This
is dangerous and it's recommended to read its ``--help`` output if you
want to use it.

459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
Installing Ganeti
+++++++++++++++++

**Mandatory** on all nodes.

It's now time to install the Ganeti software itself.  Download the
source from the project page at `<http://code.google.com/p/ganeti/>`_,
and install it (replace 2.0.0 with the latest version)::

  tar xvzf ganeti-2.0.0.tar.gz
  cd ganeti-2.0.0
  ./configure --localstatedir=/var --sysconfdir=/etc
  make
  make install
  mkdir /srv/ganeti/ /srv/ganeti/os /srv/ganeti/export

Iustin Pop's avatar
Iustin Pop committed
475
476
You also need to copy the file ``doc/examples/ganeti.initd`` from the
source archive to ``/etc/init.d/ganeti`` and register it with your
477
478
479
480
distribution's startup scripts, for example in Debian::

  update-rc.d ganeti defaults 20 80

Iustin Pop's avatar
Iustin Pop committed
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
In order to automatically restart failed instances, you need to setup a
cron job run the *ganeti-watcher* command. A sample cron file is
provided in the source at ``doc/examples/ganeti.cron`` and you can copy
that (eventually altering the path) to ``/etc/cron.d/ganeti``.

What gets installed
~~~~~~~~~~~~~~~~~~~

The above ``make install`` invocation, or installing via your
distribution mechanisms, will install on the system:

- a set of python libraries under the *ganeti* namespace (depending on
  the python version this can be located in either
  ``lib/python-$ver/site-packages`` or various other locations)
- a set of programs under ``/usr/local/sbin`` or ``/usr/sbin``
- man pages for the above programs
- a set of tools under the ``lib/ganeti/tools`` directory
- an example iallocator script (see the admin guide for details) under
  ``lib/ganeti/iallocators``
- a cron job that is needed for cluster maintenance
- an init script for automatic startup of Ganeti daemons
- provided but not installed automatically by ``make install`` is a bash
  completion script that hopefully will ease working with the many
  cluster commands
505
506
507
508
509
510
511
512
513

Installing the Operating System support packages
++++++++++++++++++++++++++++++++++++++++++++++++

**Mandatory** on all nodes.

To be able to install instances you need to have an Operating System
installation script. An example OS that works under Debian and can
install Debian and Ubuntu instace OSes is provided on the project web
Iustin Pop's avatar
Iustin Pop committed
514
515
516
site.  Download it from the project page and follow the instructions in
the ``README`` file.  Here is the installation procedure (replace 0.7
with the latest version that is compatible with your ganeti version)::
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539

  cd /usr/local/src/
  wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-0.7.tar.gz
  tar xzf ganeti-instance-debootstrap-0.7.tar.gz
  cd ganeti-instance-debootstrap-0.7
  ./configure
  make
  make install

In order to use this OS definition, you need to have internet access
from your nodes and have the *debootstrap*, *dump* and *restore*
commands installed on all nodes. Also, if the OS is configured to
partition the instance's disk in
``/etc/default/ganeti-instance-debootstrap``, you will need *kpartx*
installed.

.. admonition:: Debian

   Use this command on all nodes to install the required packages::

     apt-get install debootstrap dump kpartx

Alternatively, you can create your own OS definitions. See the manpage
Iustin Pop's avatar
Iustin Pop committed
540
:manpage:`ganeti-os-interface`.
541
542
543
544

Initializing the cluster
++++++++++++++++++++++++

Iustin Pop's avatar
Iustin Pop committed
545
**Mandatory** once per cluster, on the first node.
546

Iustin Pop's avatar
Iustin Pop committed
547
The last step is to initialize the cluster. After you have repeated the
548
549
550
551
552
above process on all of your nodes, choose one as the master, and
execute::

  gnt-cluster init <CLUSTERNAME>

Iustin Pop's avatar
Iustin Pop committed
553
554
555
556
557
558
559
The *CLUSTERNAME* is a hostname, which must be resolvable (e.g. it must
exist in DNS or in ``/etc/hosts``) by all the nodes in the cluster. You
must choose a name different from any of the nodes names for a
multi-node cluster. In general the best choice is to have a unique name
for a cluster, even if it consists of only one machine, as you will be
able to expand it later without any problems. Please note that the
hostname used for this must resolve to an IP address reserved
560
561
562
**exclusively** for this purpose, and cannot be the name of the first
(master) node.

563
564
If you want to use a bridge which is not ``xen-br0``, or no bridge at
all, use ``--nicparams``.
565

566
567
If the bridge name you are using is not ``xen-br0``, use the *-b
<BRIDGENAME>* option to specify the bridge name. In this case, you
Iustin Pop's avatar
Iustin Pop committed
568
569
should also use the *--master-netdev <BRIDGENAME>* option with the same
BRIDGENAME argument.
570
571
572

You can use a different name than ``xenvg`` for the volume group (but
note that the name must be identical on all nodes). In this case you
Iustin Pop's avatar
Iustin Pop committed
573
574
need to specify it by passing the *-g <VGNAME>* option to ``gnt-cluster
init``.
575

Iustin Pop's avatar
Iustin Pop committed
576
To set up the cluster as an Xen HVM cluster, use the
577
``--enabled-hypervisors=xen-hvm`` option to enable the HVM hypervisor
Iustin Pop's avatar
Iustin Pop committed
578
579
(you can also add ``,xen-pvm`` to enable the PVM one too). You will also
need to create the VNC cluster password file
580
581
582
583
584
585
586
587
588
589
590
591
592
593
``/etc/ganeti/vnc-cluster-password`` which contains one line with the
default VNC password for the cluster.

To setup the cluster for KVM-only usage (KVM and Xen cannot be mixed),
pass ``--enabled-hypervisors=kvm`` to the init command.

You can also invoke the command with the ``--help`` option in order to
see all the possibilities.

Joining the nodes to the cluster
++++++++++++++++++++++++++++++++

**Mandatory** for all the other nodes.

Iustin Pop's avatar
Iustin Pop committed
594
595
596
After you have initialized your cluster you need to join the other nodes
to it. You can do so by executing the following command on the master
node::
597
598
599
600
601
602
603
604
605
606
607
608
609
610

  gnt-node add <NODENAME>

Separate replication network
++++++++++++++++++++++++++++

**Optional**

Ganeti uses DRBD to mirror the disk of the virtual instances between
nodes. To use a dedicated network interface for this (in order to
improve performance or to enhance security) you need to configure an
additional interface for each node.  Use the *-s* option with
``gnt-cluster init`` and ``gnt-node add`` to specify the IP address of
this secondary interface to use for each node. Note that if you
Iustin Pop's avatar
Iustin Pop committed
611
612
specified this option at cluster setup time, you must afterwards use it
for every node add operation.
613
614
615
616

Testing the setup
+++++++++++++++++

Iustin Pop's avatar
Iustin Pop committed
617
Execute the ``gnt-node list`` command to see all nodes in the cluster::
618
619
620
621
622

  # gnt-node list
  Node              DTotal  DFree MTotal MNode MFree Pinst Sinst
  node1.example.com 197404 197404   2047  1896   125     0     0

Iustin Pop's avatar
Iustin Pop committed
623
The above shows a couple of things:
624

Iustin Pop's avatar
Iustin Pop committed
625
626
627
- The various Ganeti daemons can talk to each other
- Ganeti can examine the storage of the node (DTotal/DFree)
- Ganeti can talk to the selected hypervisor (MTotal/MNode/MFree)
628

Iustin Pop's avatar
Iustin Pop committed
629
630
Cluster burnin
~~~~~~~~~~~~~~
631

Iustin Pop's avatar
Iustin Pop committed
632
633
634
635
636
With Ganeti a tool called :command:`burnin` is provided that can test
most of the Ganeti functionality. The tool is installed under the
``lib/ganeti/tools`` directory (either under ``/usr`` or ``/usr/local``
based on the installation method). See more details under
:ref:`burnin-label`.
637

Iustin Pop's avatar
Iustin Pop committed
638
639
Further steps
-------------
640

Iustin Pop's avatar
Iustin Pop committed
641
642
643
644
You can now proceed either to the :doc:`admin`, or read the manpages of
the various commands (:manpage:`ganeti(7)`, :manpage:`gnt-cluster(8)`,
:manpage:`gnt-node(8)`, :manpage:`gnt-instance(8)`,
:manpage:`gnt-job(8)`).
645

Iustin Pop's avatar
Iustin Pop committed
646
.. rubric:: Footnotes
647

Iustin Pop's avatar
Iustin Pop committed
648
649
650
.. [#defkernel] The kernel and initrd paths can be changed at either
   cluster level (which changes the default for all instances) or at
   instance level.
651
652

.. vim: set textwidth=72 :
Iustin Pop's avatar
Iustin Pop committed
653
654
655
656
.. Local Variables:
.. mode: rst
.. fill-column: 72
.. End: