gnt-cluster.rst 40.3 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
gnt-cluster(8) Ganeti | Version @GANETI_VERSION@
================================================

Name
----

gnt-cluster - Ganeti administration, cluster-wide

Synopsis
--------

**gnt-cluster** {command} [arguments...]

DESCRIPTION
-----------

The **gnt-cluster** is used for cluster-wide administration in the
Ganeti system.

COMMANDS
--------

23 24 25 26 27 28 29
ACTIVATE-MASTER-IP
~~~~~~~~~~~~~~~~~~

**activate-master-ip**

Activates the master IP on the master node.

30 31 32
COMMAND
~~~~~~~

33
**command** [-n *node*] [-g *group*] [-M] {*command*}
34

35
Executes a command on all nodes. This command is designed for simple
36
usage. For more complex use cases the commands **dsh**\(1) or **cssh**\(1)
37 38 39 40 41
should be used instead.

If the option ``-n`` is not given, the command will be executed on all
nodes, otherwise it will be executed only on the node(s) specified. Use
the option multiple times for running it on multiple nodes, like::
42 43 44

    # gnt-cluster command -n node1.example.com -n node2.example.com date

45 46 47 48 49
The ``-g`` option can be used to run a command only on a specific node
group, e.g.::

    # gnt-cluster command -g default date

50
The ``-M`` option can be used to prepend the node name to all output
51 52
lines. The ``--failure-only`` option hides successful commands, making
it easier to see failures.
53

54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
The command is executed serially on the selected nodes. If the
master node is present in the list, the command will be executed
last on the master. Regarding the other nodes, the execution order
is somewhat alphabetic, so that node2.example.com will be earlier
than node10.example.com but after node1.example.com.

So given the node names node1, node2, node3, node10, node11, with
node3 being the master, the order will be: node1, node2, node10,
node11, node3.

The command is constructed by concatenating all other command line
arguments. For example, to list the contents of the /etc directory
on all nodes, run::

    # gnt-cluster command ls -l /etc

and the command which will be executed will be ``ls -l /etc``.

COPYFILE
~~~~~~~~

Iustin Pop's avatar
Iustin Pop committed
75
| **copyfile** [\--use-replication-network] [-n *node*] [-g *group*]
76
| {*file*}
77 78 79 80

Copies a file to all or to some nodes. The argument specifies the
source file (on the current system), the ``-n`` argument specifies
the target node, or nodes if the option is given multiple times. If
81 82
``-n`` is not given at all, the file will be copied to all nodes. The
``-g`` option can be used to only select nodes in a specific node group.
83 84 85 86
Passing the ``--use-replication-network`` option will cause the
copy to be done over the replication network (only matters if the
primary/secondary IPs are different). Example::

87
    # gnt-cluster copyfile -n node1.example.com -n node2.example.com /tmp/test
88 89 90 91

This will copy the file /tmp/test from the current node to the two
named nodes.

92 93 94
DEACTIVATE-MASTER-IP
~~~~~~~~~~~~~~~~~~~~

Iustin Pop's avatar
Iustin Pop committed
95
**deactivate-master-ip** [\--yes]
96 97 98 99 100 101 102 103

Deactivates the master IP on the master node.

This should be run only locally or on a connection to the node ip
directly, as a connection to the master ip will be broken by this
operation. Because of this risk it will require user confirmation
unless the ``--yes`` option is passed.

104 105 106
DESTROY
~~~~~~~

Iustin Pop's avatar
Iustin Pop committed
107
**destroy** {\--yes-do-it}
108 109 110 111 112

Remove all configuration files related to the cluster, so that a
**gnt-cluster init** can be done again afterwards.

Since this is a dangerous command, you are required to pass the
Iustin Pop's avatar
Iustin Pop committed
113
argument *\--yes-do-it.*
114

René Nussbaumer's avatar
René Nussbaumer committed
115 116 117
EPO
~~~

Iustin Pop's avatar
Iustin Pop committed
118
**epo** [\--on] [\--groups|\--all] [\--power-delay] *arguments*
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134

Performs an emergency power-off on nodes given as arguments. If
``--groups`` is given, arguments are node groups. If ``--all`` is
provided, the whole cluster will be shut down.

The ``--on`` flag recovers the cluster after an emergency power-off.
When powering on the cluster you can use ``--power-delay`` to define the
time in seconds (fractions allowed) waited between powering on
individual nodes.

Please note that the master node will not be turned down or up
automatically.  It will just be left in a state, where you can manully
perform the shutdown of that one node. If the master is in the list of
affected nodes and this is not a complete cluster emergency power-off
(e.g. using ``--all``), you're required to do a master failover to
another node not affected.
René Nussbaumer's avatar
René Nussbaumer committed
135

136 137 138 139 140 141 142 143 144 145
GETMASTER
~~~~~~~~~

**getmaster**

Displays the current master node.

INFO
~~~~

Iustin Pop's avatar
Iustin Pop committed
146
**info** [\--roman]
147 148 149 150 151 152 153 154

Shows runtime cluster information: cluster name, architecture (32
or 64 bit), master node, node list and instance list.

Passing the ``--roman`` option gnt-cluster info will try to print
its integer fields in a latin friendly way. This allows further
diffusion of Ganeti among ancient cultures.

155 156 157 158 159 160 161 162
SHOW-ISPECS-CMD
~~~~~~~~~~~~~~~

**show-ispecs-cmd**

Shows the command line that can be used to recreate the cluster with the
same options relative to specs in the instance policies.

163 164 165 166
INIT
~~~~

| **init**
Iustin Pop's avatar
Iustin Pop committed
167 168 169 170 171 172 173 174 175
| [{-s|\--secondary-ip} *secondary\_ip*]
| [\--vg-name *vg-name*]
| [\--master-netdev *interface-name*]
| [\--master-netmask *netmask*]
| [\--use-external-mip-script {yes \| no}]
| [{-m|\--mac-prefix} *mac-prefix*]
| [\--no-etc-hosts]
| [\--no-ssh-init]
| [\--file-storage-dir *dir*]
176
| [\--shared-file-storage-dir *dir*]
177
| [\--gluster-storage-dir *dir*]
Iustin Pop's avatar
Iustin Pop committed
178 179 180 181 182 183 184 185
| [\--enabled-hypervisors *hypervisors*]
| [{-H|\--hypervisor-parameters} *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
| [{-B|\--backend-parameters} *be-param*=*value*[,*be-param*=*value*...]]
| [{-N|\--nic-parameters} *nic-param*=*value*[,*nic-param*=*value*...]]
| [{-D|\--disk-parameters} *disk-template*:*disk-param*=*value*[,*disk-param*=*value*...]]
| [\--maintain-node-health {yes \| no}]
| [\--uid-pool *user-id pool definition*]
| [{-I|\--default-iallocator} *default instance allocator*]
186
| [\--default-iallocator-params *ial-param*=*value*,*ial-param*=*value*]
Iustin Pop's avatar
Iustin Pop committed
187 188 189 190 191 192 193 194 195
| [\--primary-ip-version *version*]
| [\--prealloc-wipe-disks {yes \| no}]
| [\--node-parameters *ndparams*]
| [{-C|\--candidate-pool-size} *candidate\_pool\_size*]
| [\--specs-cpu-count *spec-param*=*value* [,*spec-param*=*value*...]]
| [\--specs-disk-count *spec-param*=*value* [,*spec-param*=*value*...]]
| [\--specs-disk-size *spec-param*=*value* [,*spec-param*=*value*...]]
| [\--specs-mem-size *spec-param*=*value* [,*spec-param*=*value*...]]
| [\--specs-nic-count *spec-param*=*value* [,*spec-param*=*value*...]]
196 197
| [\--ipolicy-std-specs *spec*=*value* [,*spec*=*value*...]]
| [\--ipolicy-bounds-specs *bounds_ispecs*]
198
| [\--ipolicy-disk-templates *template* [,*template*...]]
199 200
| [\--ipolicy-spindle-ratio *ratio*]
| [\--ipolicy-vcpu-ratio *ratio*]
Iustin Pop's avatar
Iustin Pop committed
201 202
| [\--disk-state *diskstate*]
| [\--hypervisor-state *hvstate*]
203
| [\--drbd-usermode-helper *helper*]
204
| [\--enabled-disk-templates *template* [,*template*...]]
205
| [\--install-image *image*]
206
| [\--zeroing-image *image*]
Hrvoje Ribicic's avatar
Hrvoje Ribicic committed
207
| [\--compression-tools [*tool*, [*tool*]]]
208
| [\--user-shutdown {yes \| no}]
209 210 211 212 213 214 215 216 217 218 219 220 221 222
| {*clustername*}

This commands is only run once initially on the first node of the
cluster. It will initialize the cluster configuration, setup the
ssh-keys, start the daemons on the master node, etc. in order to have
a working one-node cluster.

Note that the *clustername* is not any random name. It has to be
resolvable to an IP address using DNS, and it is best if you give the
fully-qualified domain name. This hostname must resolve to an IP
address reserved exclusively for this purpose, i.e. not already in
use.

The cluster can run in two modes: single-home or dual-homed. In the
Iustin Pop's avatar
Iustin Pop committed
223 224
first case, all traffic (both public traffic, inter-node traffic and
data replication traffic) goes over the same interface. In the
225
dual-homed case, the data replication traffic goes over the second
Iustin Pop's avatar
Iustin Pop committed
226 227 228 229
network. The ``-s (--secondary-ip)`` option here marks the cluster as
dual-homed and its parameter represents this node's address on the
second network.  If you initialise the cluster with ``-s``, all nodes
added must have a secondary IP as well.
230 231 232 233 234 235 236 237 238 239

Note that for Ganeti it doesn't matter if the secondary network is
actually a separate physical network, or is done using tunneling,
etc. For performance reasons, it's recommended to use a separate
network, of course.

The ``--vg-name`` option will let you specify a volume group
different than "xenvg" for Ganeti to use when creating instance
disks. This volume group must have the same name on all nodes. Once
the cluster is initialized this can be altered by using the
240 241 242
**modify** command. Note that if the volume group name is modified after
the cluster creation and DRBD support is enabled you might have to
manually modify the metavg as well.
243 244

If you don't want to use lvm storage at all use
245
the ``--enabled-disk-templates`` option to restrict the set of enabled
246
disk templates. Once the cluster is initialized
247 248 249 250 251 252 253
you can change this setup with the **modify** command.

The ``--master-netdev`` option is useful for specifying a different
interface on which the master will activate its IP address. It's
important that all nodes have this interface because you'll need it
for a master failover.

254 255 256 257 258
The ``--master-netmask`` option allows to specify a netmask for the
master IP. The netmask must be specified as an integer, and will be
interpreted as a CIDR netmask. The default value is 32 for an IPv4
address and 128 for an IPv6 address.

259 260 261 262 263
The ``--use-external-mip-script`` option allows to specify whether to
use an user-supplied master IP address setup script, whose location is
``@SYSCONFDIR@/ganeti/scripts/master-ip-setup``. If the option value is
set to False, the default script (located at
``@PKGLIBDIR@/tools/master-ip-setup``) will be executed.
264

Iustin Pop's avatar
Iustin Pop committed
265 266 267 268
The ``-m (--mac-prefix)`` option will let you specify a three byte
prefix under which the virtual MAC addresses of your instances will be
generated. The prefix must be specified in the format ``XX:XX:XX`` and
the default is ``aa:00:00``.
269 270 271 272 273 274 275

The ``--no-etc-hosts`` option allows you to initialize the cluster
without modifying the /etc/hosts file.

The ``--no-ssh-init`` option allows you to initialize the cluster
without creating or distributing SSH key pairs.

276 277 278 279 280 281 282
The ``--file-storage-dir``, ``--shared-file-storage-dir`` and
``--gluster-storage-dir`` options allow you set the directory to use for
storing the instance disk files when using respectively the file storage
backend, the shared file storage backend and the gluster storage
backend. Note that these directories must be an allowed directory for
file storage. Those directories are specified in the
``@SYSCONFDIR@/ganeti/file-storage-paths`` file.
283 284
The file storage directory can also be a subdirectory of an allowed one.
The file storage directory should be present on all nodes.
285

286 287 288 289
The ``--prealloc-wipe-disks`` sets a cluster wide configuration value
for wiping disks prior to allocation and size changes (``gnt-instance
grow-disk``). This increases security on instance level as the instance
can't access untouched data from its underlying storage.
290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316

The ``--enabled-hypervisors`` option allows you to set the list of
hypervisors that will be enabled for this cluster. Instance
hypervisors can only be chosen from the list of enabled
hypervisors, and the first entry of this list will be used by
default. Currently, the following hypervisors are available:

xen-pvm
    Xen PVM hypervisor

xen-hvm
    Xen HVM hypervisor

kvm
    Linux KVM hypervisor

chroot
    a simple chroot manager that starts chroot based on a script at the
    root of the filesystem holding the chroot

fake
    fake hypervisor for development/testing

Either a single hypervisor name or a comma-separated list of
hypervisor names can be specified. If this option is not specified,
only the xen-pvm hypervisor is enabled by default.

317 318 319 320 321
The ``--user-shutdown`` option enables or disables user shutdown
detection at the cluster level.  User shutdown detection allows users to
initiate instance poweroff from inside the instance, and Ganeti will
report the instance status as 'USER_down' (as opposed, to 'ERROR_down')
and the watcher will not restart these instances, thus preserving their
322
instance status.  This option is disabled by default.  For KVM,
323 324
the hypervisor parameter ``user_shutdown`` must also be set, either at
the cluster level or on a per-instance basis (see **gnt-instance**\(8)).
325

Iustin Pop's avatar
Iustin Pop committed
326
The ``-H (--hypervisor-parameters)`` option allows you to set default
327 328
hypervisor specific parameters for the cluster. The format of this
option is the name of the hypervisor, followed by a colon and a
Iustin Pop's avatar
Iustin Pop committed
329
comma-separated list of key=value pairs. The keys available for each
330
hypervisors are detailed in the **gnt-instance**\(8) man page, in the
Iustin Pop's avatar
Iustin Pop committed
331
**add** command plus the following parameters which are only
332 333 334 335 336
configurable globally (at cluster level):

migration\_port
    Valid for the Xen PVM and KVM hypervisors.

337 338 339 340
    This options specifies the TCP port to use for live-migration when
    using the xm toolstack. The same port should be configured on all
    nodes in the ``@XEN_CONFIG_DIR@/xend-config.sxp`` file, under the
    key "xend-relocation-port".
341 342 343 344 345 346 347 348 349 350

migration\_bandwidth
    Valid for the KVM hypervisor.

    This option specifies the maximum bandwidth that KVM will use for
    instance live migrations. The value is in MiB/s.

    This option is only effective with kvm versions >= 78 and qemu-kvm
    versions >= 0.10.0.

Iustin Pop's avatar
Iustin Pop committed
351
The ``-B (--backend-parameters)`` option allows you to set the default
352
backend parameters for the cluster. The parameter format is a
Iustin Pop's avatar
Iustin Pop committed
353 354
comma-separated list of key=value pairs with the following supported
keys:
355 356 357 358 359

vcpus
    Number of VCPUs to set for an instance by default, must be an
    integer, will be set to 1 if no specified.

360 361 362 363 364 365 366 367 368 369 370
maxmem
    Maximum amount of memory to allocate for an instance by default, can
    be either an integer or an integer followed by a unit (M for
    mebibytes and G for gibibytes are supported), will be set to 128M if
    not specified.

minmem
    Minimum amount of memory to allocate for an instance by default, can
    be either an integer or an integer followed by a unit (M for
    mebibytes and G for gibibytes are supported), will be set to 128M if
    not specified.
371 372 373 374 375

auto\_balance
    Value of the auto\_balance flag for instances to use by default,
    will be set to true if not specified.

376
always\_failover
377
    Default value for the ``always_failover`` flag for instances; if
378 379
    not set, ``False`` is used.

380

381 382 383 384
The ``-N (--nic-parameters)`` option allows you to set the default
network interface parameters for the cluster. The parameter format is a
comma-separated list of key=value pairs with the following supported
keys:
385 386

mode
387 388
    The default NIC mode, one of ``routed``, ``bridged`` or
    ``openvswitch``.
389 390

link
391 392 393 394 395 396 397
    In ``bridged`` or ``openvswitch`` mode the default interface where
    to attach NICs. In ``routed`` mode it represents an
    hypervisor-vif-script dependent value to allow different instance
    groups. For example under the KVM default network script it is
    interpreted as a routing table number or name. Openvswitch support
    is also hypervisor dependent and currently works for the default KVM
    network script. Under Xen a custom network script must be provided.
398

399 400 401 402 403 404 405
The ``-D (--disk-parameters)`` option allows you to set the default disk
template parameters at cluster level. The format used for this option is
similar to the one use by the  ``-H`` option: the disk template name
must be specified first, followed by a colon and by a comma-separated
list of key-value pairs. These parameters can only be specified at
cluster and node group level; the cluster-level parameter are inherited
by the node group at the moment of its creation, and can be further
406
modified at node group level using the **gnt-group**\(8) command.
407

408 409 410
The following is the list of disk parameters available for the **drbd**
template, with measurement units specified in square brackets at the end
of the description (when applicable):
411 412

resync-rate
413
    Static re-synchronization rate. [KiB/s]
414 415

data-stripes
416
    Number of stripes to use for data LVs.
417 418

meta-stripes
419
    Number of stripes to use for meta LVs.
420 421 422 423 424

disk-barriers
    What kind of barriers to **disable** for disks. It can either assume
    the value "n", meaning no barrier disabled, or a non-empty string
    containing a subset of the characters "bfd". "b" means disable disk
425
    barriers, "f" means disable disk flushes, "d" disables disk drains.
426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444

meta-barriers
    Boolean value indicating whether the meta barriers should be
    disabled (True) or not (False).

metavg
    String containing the name of the default LVM volume group for DRBD
    metadata. By default, it is set to ``xenvg``. It can be overridden
    during the instance creation process by using the ``metavg`` key of
    the ``--disk`` parameter.

disk-custom
    String containing additional parameters to be appended to the
    arguments list of ``drbdsetup disk``.

net-custom
    String containing additional parameters to be appended to the
    arguments list of ``drbdsetup net``.

445 446 447 448 449
protocol
    Replication protocol for the DRBD device. Has to be either "A", "B"
    or "C". Refer to the DRBD documentation for further information
    about the differences between the protocols.

450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475
dynamic-resync
    Boolean indicating whether to use the dynamic resync speed
    controller or not. If enabled, c-plan-ahead must be non-zero and all
    the c-* parameters will be used by DRBD. Otherwise, the value of
    resync-rate will be used as a static resync speed.

c-plan-ahead
    Agility factor of the dynamic resync speed controller. (the higher,
    the slower the algorithm will adapt the resync speed). A value of 0
    (that is the default) disables the controller. [ds]

c-fill-target
    Maximum amount of in-flight resync data for the dynamic resync speed
    controller. [sectors]

c-delay-target
    Maximum estimated peer response latency for the dynamic resync speed
    controller. [ds]

c-min-rate
    Minimum resync speed for the dynamic resync speed controller. [KiB/s]

c-max-rate
    Upper bound on resync speed for the dynamic resync speed controller.
    [KiB/s]

476 477 478
List of parameters available for the **plain** template:

stripes
479
    Number of stripes to use for new LVs.
480

481 482 483 484 485 486 487
List of parameters available for the **rbd** template:

pool
    The RADOS cluster pool, inside which all rbd volumes will reside.
    When a new RADOS cluster is deployed, the default pool to put rbd
    volumes (Images in RADOS terminology) is 'rbd'.

488 489 490 491 492 493 494 495 496 497 498 499 500 501 502
access
    If 'userspace', instances will access their disks directly without
    going through a block device, avoiding expensive context switches
    with kernel space and the potential for deadlocks_ in low memory
    scenarios.

    The default value is 'kernelspace' and it disables this behaviour.
    This setting may only be changed to 'userspace' if all instance
    disks in the affected group or cluster can be accessed in userspace.

    Attempts to use this feature without rbd support compiled in KVM
    result in a "no such file or directory" error messages.

.. _deadlocks: http://tracker.ceph.com/issues/3076

Iustin Pop's avatar
Iustin Pop committed
503
The option ``--maintain-node-health`` allows one to enable/disable
504 505 506
automatic maintenance actions on nodes. Currently these include
automatic shutdown of instances and deactivation of DRBD devices on
offline nodes; in the future it might be extended to automatic
507 508
removal of unknown LVM volumes, etc. Note that this option is only
useful if the use of ``ganeti-confd`` was enabled at compilation.
509 510 511 512 513 514 515 516 517 518 519

The ``--uid-pool`` option initializes the user-id pool. The
*user-id pool definition* can contain a list of user-ids and/or a
list of user-id ranges. The parameter format is a comma-separated
list of numeric user-ids or user-id ranges. The ranges are defined
by a lower and higher boundary, separated by a dash. The boundaries
are inclusive. If the ``--uid-pool`` option is not supplied, the
user-id pool is initialized to an empty list. An empty list means
that the user-id pool feature is disabled.

The ``-I (--default-iallocator)`` option specifies the default
520 521 522 523 524 525 526
instance allocator. The instance allocator will be used for operations
like instance creation, instance and node migration, etc. when no
manual override is specified. If this option is not specified and
htools was not enabled at build time, the default instance allocator
will be blank, which means that relevant operations will require the
administrator to manually specify either an instance allocator, or a
set of nodes. If the option is not specified but htools was enabled,
527
the default iallocator will be **hail**\(1) (assuming it can be found
528 529
on disk). The default iallocator can be changed later using the
**modify** command.
530

531 532 533 534
The option ``--default-iallocator-params`` sets the cluster-wide
iallocator parameters used by the default iallocator only on instance
allocations.

535 536 537 538 539
The ``--primary-ip-version`` option specifies the IP version used
for the primary address. Possible values are 4 and 6 for IPv4 and
IPv6, respectively. This option is used when resolving node names
and the cluster name.

540
The ``--node-parameters`` option allows you to set default node
541
parameters for the cluster. Please see **ganeti**\(7) for more
542 543
information about supported key=value pairs.

Iustin Pop's avatar
Iustin Pop committed
544 545 546
The ``-C (--candidate-pool-size)`` option specifies the
``candidate_pool_size`` cluster parameter. This is the number of nodes
that the master will try to keep as master\_candidates. For more
547
details about this role and other node roles, see the **ganeti**\(7).
Iustin Pop's avatar
Iustin Pop committed
548

549
The ``--specs-...`` and ``--ipolicy-...`` options specify the instance
550 551
policy on the cluster. The ``--ipolicy-bounds-specs`` option sets the
minimum and maximum specifications for instances. The format is:
552 553
min:*param*=*value*,.../max:*param*=*value*,... and further
specifications pairs can be added by using ``//`` as a separator. The
554 555 556 557 558 559 560 561 562 563 564 565 566 567 568
``--ipolicy-std-specs`` option takes a list of parameter/value pairs.
For both options, *param* can be:

- ``cpu-count``: number of VCPUs for an instance
- ``disk-count``: number of disk for an instance
- ``disk-size``: size of each disk
- ``memory-size``: instance memory
- ``nic-count``: number of network interface
- ``spindle-use``: spindle usage for an instance

For the ``--specs-...`` options, each option can have three values:
``min``, ``max`` and ``std``, which can also be modified on group level
(except for ``std``, which is defined once for the entire cluster).
Please note, that ``std`` values are not the same as defaults set by
``--beparams``, but they are used for the capacity calculations.
569 570 571 572 573 574 575

- ``--specs-cpu-count`` limits the number of VCPUs that can be used by an
  instance.
- ``--specs-disk-count`` limits the number of disks
- ``--specs-disk-size`` limits the disk size for every disk used
- ``--specs-mem-size`` limits the amount of memory available
- ``--specs-nic-count`` sets limits on the number of NICs used
576

577 578 579 580 581
The ``--ipolicy-spindle-ratio`` option takes a decimal number. The
``--ipolicy-disk-templates`` option takes a comma-separated list of disk
templates. This list of disk templates must be a subset of the list
of cluster-wide enabled disk templates (which can be set with
``--enabled-disk-templates``).
582

583 584
- ``--ipolicy-spindle-ratio`` limits the instances-spindles ratio
- ``--ipolicy-vcpu-ratio`` limits the vcpu-cpu ratio
Agata Murawska's avatar
Agata Murawska committed
585

586 587 588
All the instance policy elements can be overridden at group level. Group
level overrides can be removed by specifying ``default`` as the value of
an item.
Agata Murawska's avatar
Agata Murawska committed
589

590 591 592
The ``--drbd-usermode-helper`` option can be used to specify a usermode
helper. Check that this string is the one used by the DRBD kernel.

593
For details about how to use ``--hypervisor-state`` and ``--disk-state``
594
have a look at **ganeti**\(7).
595

596 597 598 599 600 601
The ``--enabled-disk-templates`` option specifies a list of disk templates
that can be used by instances of the cluster. For the possible values in
this list, see **gnt-instance**\(8). Note that in contrast to the list of
disk templates in the ipolicy, this list is a hard restriction. It is not
possible to create instances with disk templates that are not enabled in
the cluster. It is also not possible to disable a disk template when there
602 603 604
are still instances using it. The first disk template in the list of
enabled disk template is the default disk template. It will be used for
instance creation, if no disk template is requested explicitely.
605

606 607 608 609 610 611 612
The ``--install-image`` option specifies the location of the OS image to
use to run the OS scripts inside a virtualized environment. This can be
a file path or a URL. In the case that a file path is used, nodes are
expected to have the install image located at the given path, although
that is enforced during a instance create with unsafe OS scripts
operation only.

613 614 615 616 617 618
The ``--zeroing-image`` option specifies the location of the OS image to
use to zero out the free space of an instance. This can be a file path
or a URL. In the case that a file path is used, nodes are expected to
have the zeroing image located at the given path, although that is
enforced during a zeroing operation only.

Hrvoje Ribicic's avatar
Hrvoje Ribicic committed
619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634
The ``--compression-tools`` option specifies the tools that can be used
to compress the disk data of instances in transfer. The default tools
are: 'gzip', 'gzip-slow', and 'gzip-fast'. For compatibility reasons,
the 'gzip' tool cannot be excluded from the list of compression tools.
Ganeti knows how to use certain tools, but does not provide them as a
default as they are not commonly present: currently only 'lzop'. The
user should indicate their presence by specifying them through this
option.
Any other custom tool specified must have a simple executable name
('[-_a-zA-Z0-9]+'), accept input on stdin, and produce output on
stdout. The '-d' flag specifies that decompression rather than
compression is taking place. The '-h' flag must be supported as a means
of testing whether the executable exists. These requirements are
compatible with the gzip command line options, allowing many tools to
be easily wrapped and used.

635 636 637
MASTER-FAILOVER
~~~~~~~~~~~~~~~

638
**master-failover** [\--no-voting] [\--yes-do-it]
639 640 641 642 643 644 645 646 647 648 649 650 651

Failover the master role to the current node.

The ``--no-voting`` option skips the remote node agreement checks.
This is dangerous, but necessary in some cases (for example failing
over the master role in a 2 node cluster with the original master
down). If the original master then comes up, it won't be able to
start its master daemon because it won't have enough votes, but so
won't the new master, if the master daemon ever needs a restart.
You can pass ``--no-voting`` to **ganeti-masterd** on the new
master to solve this problem, and run **gnt-cluster redist-conf**
to make sure the cluster is consistent again.

652 653 654 655 656
The option ``--yes-do-it`` is used together with ``--no-voting``, for
skipping the interactive checks. This is even more dangerous, and should
only be used in conjunction with other means (e.g. a HA suite) to
confirm that the operation is indeed safe.

657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672
MASTER-PING
~~~~~~~~~~~

**master-ping**

Checks if the master daemon is alive.

If the master daemon is alive and can respond to a basic query (the
equivalent of **gnt-cluster info**), then the exit code of the
command will be 0. If the master daemon is not alive (either due to
a crash or because this is not the master node), the exit code will
be 1.

MODIFY
~~~~~~

673
| **modify** [\--submit] [\--print-job-id]
674
| [\--force]
Iustin Pop's avatar
Iustin Pop committed
675 676 677 678 679 680 681 682 683 684
| [\--vg-name *vg-name*]
| [\--enabled-hypervisors *hypervisors*]
| [{-H|\--hypervisor-parameters} *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
| [{-B|\--backend-parameters} *be-param*=*value*[,*be-param*=*value*...]]
| [{-N|\--nic-parameters} *nic-param*=*value*[,*nic-param*=*value*...]]
| [{-D|\--disk-parameters} *disk-template*:*disk-param*=*value*[,*disk-param*=*value*...]]
| [\--uid-pool *user-id pool definition*]
| [\--add-uids *user-id pool definition*]
| [\--remove-uids *user-id pool definition*]
| [{-C|\--candidate-pool-size} *candidate\_pool\_size*]
685
| [--max-running-jobs *count* ]
686
| [--max-tracked-jobs *count* ]
Iustin Pop's avatar
Iustin Pop committed
687 688 689
| [\--maintain-node-health {yes \| no}]
| [\--prealloc-wipe-disks {yes \| no}]
| [{-I|\--default-iallocator} *default instance allocator*]
690
| [\--default-iallocator-params *ial-param*=*value*,*ial-param*=*value*]
Iustin Pop's avatar
Iustin Pop committed
691 692
| [\--reserved-lvs=*NAMES*]
| [\--node-parameters *ndparams*]
693
| [{-m|\--mac-prefix} *mac-prefix*]
Iustin Pop's avatar
Iustin Pop committed
694 695 696 697 698
| [\--master-netdev *interface-name*]
| [\--master-netmask *netmask*]
| [\--use-external-mip-script {yes \| no}]
| [\--hypervisor-state *hvstate*]
| [\--disk-state *diskstate*]
699 700
| [\--ipolicy-std-specs *spec*=*value* [,*spec*=*value*...]]
| [\--ipolicy-bounds-specs *bounds_ispecs*]
701
| [\--ipolicy-disk-templates *template* [,*template*...]]
702 703
| [\--ipolicy-spindle-ratio *ratio*]
| [\--ipolicy-vcpu-ratio *ratio*]
704
| [\--enabled-disk-templates *template* [,*template*...]]
705
| [\--drbd-usermode-helper *helper*]
706
| [\--file-storage-dir *dir*]
707
| [\--shared-file-storage-dir *dir*]
Hrvoje Ribicic's avatar
Hrvoje Ribicic committed
708
| [\--compression-tools [*tool*, [*tool*]]]
709
| [\--instance-communication-network *network*]
710 711
| [\--install-image *image*]
| [\--zeroing-image *image*]
712
| [\--user-shutdown {yes \| no}]
Agata Murawska's avatar
Agata Murawska committed
713

714 715 716

Modify the options for the cluster.

717 718 719
The ``--vg-name``, ``--enabled-hypervisors``, ``-H (--hypervisor-parameters)``,
``-B (--backend-parameters)``, ``-D (--disk-parameters)``, ``--nic-parameters``,
``-C (--candidate-pool-size)``, ``--maintain-node-health``,
720
``--prealloc-wipe-disks``, ``--uid-pool``, ``--node-parameters``,
721 722
``--mac-prefix``, ``--master-netdev``, ``--master-netmask``,
``--use-external-mip-script``, ``--drbd-usermode-helper``,
Hrvoje Ribicic's avatar
Hrvoje Ribicic committed
723 724
``--file-storage-dir``, ``--shared-file-storage-dir``,
``--compression-tools``, and ``--enabled-disk-templates`` options are described in the **init** command.
725
``--master-netdev``, ``--master-netmask``, ``--use-external-mip-script``,
726
``--drbd-usermode-helper``, ``--file-storage-dir``,
727 728
``--shared-file-storage-dir``, ``--enabled-disk-templates``, and
``--user-shutdown`` options are
729
described in the **init** command.
730

731
The ``--hypervisor-state`` and ``--disk-state`` options are described in
732
detail in **ganeti**\(7).
733

734 735 736
The ``--max-running-jobs`` options allows to set limit on the
number of jobs in non-finished jobs that are not queued, i.e.,
the number of jobs that are in waiting or running state.
737 738 739 740 741
The ``--max-tracked-jobs`` options allows to set the limit on
the tracked jobs. Normally, Ganeti will watch waiting and running
jobs by tracking their job file with inotify. If this limit is
exceeded, however, Ganeti will back off and only periodically
pull for updates.
742

743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758
The ``--add-uids`` and ``--remove-uids`` options can be used to
modify the user-id pool by adding/removing a list of user-ids or
user-id ranges.

The option ``--reserved-lvs`` specifies a list (comma-separated) of
logical volume group names (regular expressions) that will be
ignored by the cluster verify operation. This is useful if the
volume group used for Ganeti is shared with the system for other
uses. Note that it's not recommended to create and mark as ignored
logical volume names which match Ganeti's own name format (starting
with UUID and then .diskN), as this option only skips the
verification, but not the actual use of the names given.

To remove all reserved logical volumes, pass in an empty argument
to the option, as in ``--reserved-lvs=`` or ``--reserved-lvs ''``.

Iustin Pop's avatar
Iustin Pop committed
759 760 761
The ``-I (--default-iallocator)`` is described in the **init**
command. To clear the default iallocator, just pass an empty string
('').
762

763 764 765 766
The option ``--default-iallocator-params`` is described in the **init**
command. To clear the default iallocator parameters, just pass an empty
string ('').

767
The ``--ipolicy-...`` options are described in the **init** command.
Agata Murawska's avatar
Agata Murawska committed
768

769 770 771 772 773 774 775 776
The ``--instance-communication-network`` enables instance communication
by specifying the name of the Ganeti network that should be used for
instance communication.  If the supplied network does not exist, Ganeti
will create a new network with the supplied name with the default
parameters for instance communication.  If the supplied network exists,
Ganeti will check its parameters and warn about unusual configurations,
but it will still use that network for instance communication.

777 778 779
See **gnt-cluster init** for a description of ``--install-image`` and
``--zeroing-image``.

780
See **ganeti**\(7) for a description of ``--submit`` and other common
781 782
options.

783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812
QUEUE
~~~~~

**queue** {drain | undrain | info}

Change job queue properties.

The ``drain`` option sets the drain flag on the job queue. No new
jobs will be accepted, but jobs already in the queue will be
processed.

The ``undrain`` will unset the drain flag on the job queue. New
jobs will be accepted.

The ``info`` option shows the properties of the job queue.

WATCHER
~~~~~~~

**watcher** {pause *duration* | continue | info}

Make the watcher pause or let it continue.

The ``pause`` option causes the watcher to pause for *duration*
seconds.

The ``continue`` option will let the watcher continue.

The ``info`` option shows whether the watcher is currently paused.

813
REDIST-CONF
814 815
~~~~~~~~~~~

816
**redist-conf** [\--submit] [\--print-job-id]
817 818 819 820 821 822

This command forces a full push of configuration files from the
master node to the other nodes in the cluster. This is normally not
needed, but can be run if the **verify** complains about
configuration mismatches.

823
See **ganeti**\(7) for a description of ``--submit`` and other common
824
options.
825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843

RENAME
~~~~~~

**rename** [-f] {*name*}

Renames the cluster and in the process updates the master IP
address to the one the new name resolves to. At least one of either
the name or the IP address must be different, otherwise the
operation will be aborted.

Note that since this command can be dangerous (especially when run
over SSH), the command will require confirmation unless run with
the ``-f`` option.

RENEW-CRYPTO
~~~~~~~~~~~~

| **renew-crypto** [-f]
844
| [\--new-cluster-certificate] | [\--new-node-certificates]
845
| [\--new-confd-hmac-key]
Iustin Pop's avatar
Iustin Pop committed
846 847 848 849
| [\--new-rapi-certificate] [\--rapi-certificate *rapi-cert*]
| [\--new-spice-certificate | \--spice-certificate *spice-cert*
| \--spice-ca-certificate *spice-ca-cert*]
| [\--new-cluster-domain-secret] [\--cluster-domain-secret *filename*]
850 851 852

This command will stop all Ganeti daemons in the cluster and start
them again once the new certificates and keys are replicated. The
853 854 855 856 857 858 859 860 861 862 863 864 865 866
option ``--new-confd-hmac-key`` can be used to regenerate
the HMAC key used by **ganeti-confd**\(8).

The option ``--new-cluster-certificate`` will regenerate the
cluster-internal server SSL certificate. The option
``--new-node-certificates`` will generate new node SSL
certificates for all nodes. Note that for the regeneration of
of the server SSL certficate will invoke a regeneration of the
node certificates as well, because node certificates are signed
by the server certificate and thus have to be recreated and
signed by the new server certificate. Nodes which are offline
during a renewal of the server or the node certificates are not
accessible anymore once they are marked as online again. To
fix this, please readd the node instead.
867

868
To generate a new self-signed RAPI certificate (used by
869
**ganeti-rapi**\(8)) specify ``--new-rapi-certificate``. If you want to
870 871 872
use your own certificate, e.g. one signed by a certificate
authority (CA), pass its filename to ``--rapi-certificate``.

873
To generate a new self-signed SPICE certificate, used for SPICE
874 875 876 877 878
connections to the KVM hypervisor, specify the
``--new-spice-certificate`` option. If you want to provide a
certificate, pass its filename to ``--spice-certificate`` and pass the
signing CA certificate to ``--spice-ca-certificate``.

879 880 881 882
Finally ``--new-cluster-domain-secret`` generates a new, random
cluster domain secret, and ``--cluster-domain-secret`` reads the
secret from a file. The cluster domain secret is used to sign
information exchanged between separate clusters via a third party.
883 884 885 886 887 888 889 890 891 892

REPAIR-DISK-SIZES
~~~~~~~~~~~~~~~~~

**repair-disk-sizes** [instance...]

This command checks that the recorded size of the given instance's
disks matches the actual size and updates any mismatches found.
This is needed if the Ganeti configuration is no longer consistent
with reality, as it will impact some disk operations. If no
893 894
arguments are given, all instances will be checked. When exclusive
storage is active, also spindles are updated.
895 896 897

Note that only active disks can be checked by this command; in case
a disk cannot be activated it's advised to use
Iustin Pop's avatar
Iustin Pop committed
898
**gnt-instance activate-disks \--ignore-size ...** to force
899 900
activation without regard to the current size.

901
When all the disk sizes are consistent, the command will return no
902 903 904
output. Otherwise it will log details about the inconsistencies in
the configuration.

905 906 907 908 909 910 911 912 913 914 915 916
UPGRADE
~~~~~~~

**upgrade** {--to *version* | --resume}

This command safely switches all nodes of the cluster to a new Ganeti
version. It is a prerequisite that the new version is already installed,
albeit not activated, on all nodes; this requisite is checked before any
actions are done.

If called with the ``--resume`` option, any pending upgrade is
continued, that was interrupted by a power failure or similar on
917 918
master. It will do nothing, if not run on the master node, or if no
upgrade was in progress.
919 920


921 922 923
VERIFY
~~~~~~

Iustin Pop's avatar
Iustin Pop committed
924 925 926
| **verify** [\--no-nplus1-mem] [\--node-group *nodegroup*]
| [\--error-codes] [{-I|\--ignore-errors} *errorcode*]
| [{-I|\--ignore-errors} *errorcode*...]
927 928 929 930 931 932 933 934 935

Verify correctness of cluster configuration. This is safe with
respect to running instances, and incurs no downtime of the
instances.

If the ``--no-nplus1-mem`` option is given, Ganeti won't check
whether if it loses a node it can restart all the instances on
their secondaries (and report an error otherwise).

936 937 938 939 940
With ``--node-group``, restrict the verification to those nodes and
instances that live in the named group. This will not verify global
settings, but will allow to perform verification of a group while other
operations are ongoing in other groups.

941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969
The ``--error-codes`` option outputs each error in the following
parseable format: *ftype*:*ecode*:*edomain*:*name*:*msg*.
These fields have the following meaning:

ftype
    Failure type. Can be *WARNING* or *ERROR*.

ecode
    Error code of the failure. See below for a list of error codes.

edomain
    Can be *cluster*, *node* or *instance*.

name
    Contains the name of the item that is affected from the failure.

msg
    Contains a descriptive error message about the error

``gnt-cluster verify`` will have a non-zero exit code if at least one of
the failures that are found are of type *ERROR*.

The ``--ignore-errors`` option can be used to change this behaviour,
because it demotes the error represented by the error code received as a
parameter to a warning. The option must be repeated for each error that
should be ignored (e.g.: ``-I ENODEVERSION -I ENODEORPHANLV``). The
``--error-codes`` option can be used to determine the error code of a
given error.

970 971 972 973
Note that the verification of the configuration file consistency across
master candidates can fail if there are other concurrently running
operations that modify the configuration.

974 975 976 977
List of error codes:

@CONSTANTS_ECODES@

978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996
VERIFY-DISKS
~~~~~~~~~~~~

**verify-disks**

The command checks which instances have degraded DRBD disks and
activates the disks of those instances.

This command is run from the **ganeti-watcher** tool, which also
has a different, complementary algorithm for doing this check.
Together, these two should ensure that DRBD disks are kept
consistent.

VERSION
~~~~~~~

**version**

Show the cluster version.
997

998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059
Tags
~~~~

ADD-TAGS
^^^^^^^^

**add-tags** [\--from *file*] {*tag*...}

Add tags to the cluster. If any of the tags contains invalid
characters, the entire operation will abort.

If the ``--from`` option is given, the list of tags will be
extended with the contents of that file (each line becomes a tag).
In this case, there is not need to pass tags on the command line
(if you do, both sources will be used). A file name of - will be
interpreted as stdin.

LIST-TAGS
^^^^^^^^^

**list-tags**

List the tags of the cluster.

REMOVE-TAGS
^^^^^^^^^^^

**remove-tags** [\--from *file*] {*tag*...}

Remove tags from the cluster. If any of the tags are not existing
on the cluster, the entire operation will abort.

If the ``--from`` option is given, the list of tags to be removed will
be extended with the contents of that file (each line becomes a tag).
In this case, there is not need to pass tags on the command line (if
you do, tags from both sources will be removed). A file name of - will
be interpreted as stdin.

SEARCH-TAGS
^^^^^^^^^^^

**search-tags** {*pattern*}

Searches the tags on all objects in the cluster (the cluster
itself, the nodes and the instances) for a given pattern. The
pattern is interpreted as a regular expression and a search will be
done on it (i.e. the given pattern is not anchored to the beggining
of the string; if you want that, prefix the pattern with ^).

If no tags are matching the pattern, the exit code of the command
will be one. If there is at least one match, the exit code will be
zero. Each match is listed on one line, the object and the tag
separated by a space. The cluster will be listed as /cluster, a
node will be listed as /nodes/*name*, and an instance as
/instances/*name*. Example:

::

    # gnt-cluster search-tags time
    /cluster ctime:2007-09-01
    /nodes/node1.example.com mtime:2007-10-04

1060 1061 1062 1063 1064
.. vim: set textwidth=72 :
.. Local Variables:
.. mode: rst
.. fill-column: 72
.. End: