ganeti.rst 15.1 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
ganeti(7) Ganeti | Version @GANETI_VERSION@
===========================================

Name
----

ganeti - cluster-based virtualization management

Synopsis
--------

::

    # gnt-cluster init cluster1.example.com
    # gnt-node add node2.example.com
    # gnt-instance add -n node2.example.com \
    > -o debootstrap --disk 0:size=30g \
    > -t plain instance1.example.com


DESCRIPTION
-----------

The Ganeti software manages physical nodes and virtual instances of a
cluster based on a virtualization software. The current version (2.3)
supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an
experimental hypervisor.

Quick start
-----------

First you must install the software on all the cluster nodes, either
from sources or (if available) from a package. The next step is to
create the initial cluster configuration, using **gnt-cluster init**.

Then you can add other nodes, or start creating instances.

Cluster architecture
--------------------

In Ganeti 2.0, the architecture of the cluster is a little more
complicated than in 1.2. The cluster is coordinated by a master daemon
43
(**ganeti-masterd**\(8)), running on the master node. Each node runs
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98
(as before) a node daemon, and the master has the RAPI daemon running
too.

Node roles
~~~~~~~~~~

Each node can be in one of the following states:

master
    Only one node per cluster can be in this role, and this node is the
    one holding the authoritative copy of the cluster configuration and
    the one that can actually execute commands on the cluster and
    modify the cluster state. See more details under
    *Cluster configuration*.

master_candidate
    The node receives the full cluster configuration (configuration
    file and jobs) and can become a master via the
    **gnt-cluster master-failover** command. Nodes that are not in this
    state cannot transition into the master role due to missing state.

regular
    This the normal state of a node.

drained
    Nodes in this state are functioning normally but cannot receive
    new instances, because the intention is to set them to *offline*
    or remove them from the cluster.

offline
    These nodes are still recorded in the Ganeti configuration, but
    except for the master daemon startup voting procedure, they are not
    actually contacted by the master. This state was added in order to
    allow broken machines (that are being repaired) to remain in the
    cluster but without creating problems.


Node flags
~~~~~~~~~~

Nodes have two flags which govern which roles they can take:

master_capable
    The node can become a master candidate, and furthermore the master
    node. When this flag is disabled, the node cannot become a
    candidate; this can be useful for special networking cases, or less
    reliable hardware.

vm_capable
    The node can host instances. When enabled (the default state), the
    node will participate in instance allocation, capacity calculation,
    etc. When disabled, the node will be skipped in many cluster checks
    and operations.


99 100 101
Node Parameters
~~~~~~~~~~~~~~~

102 103
The ``ndparams`` refer to node parameters. These can be set as defaults
on cluster and node group levels, but they take effect for nodes only.
104 105 106 107 108 109 110 111

Currently we support the following node parameters:

oob_program
    Path to an executable used as the out-of-band helper as described in
    the `Ganeti Node OOB Management Framework <design-oob.rst>`_ design
    document.

112 113 114 115 116 117 118
spindle_count
    This should reflect the I/O performance of local attached storage
    (e.g. for "file", "plain" and "drbd" disk templates). It doesn't
    have to match the actual spindle count of (any eventual) mechanical
    hard-drives, its meaning is site-local and just the relative values
    matter.

119 120 121 122
exclusive_storage
    When this Boolean flag is enabled, physical disks on the node are
    assigned to instance disks in an exclusive manner, so as to lower I/O
    interference between instances. See the `Partitioned Ganeti
123 124 125
    <design-partitioned.rst>`_ design document for more details. This
    parameter cannot be set on individual nodes, as its value must be
    the same within each node group.
126

127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
ovs
    When this Boolean flag is enabled, OpenvSwitch will be used as the
    network layer. This will cause the initialization of OpenvSwitch on
    the nodes when added to the cluster. Per default this is not enabled.

ovs_name
    When ovs is enabled, this parameter will represent the name of the
    OpenvSwitch to generate and use. This will default to `switch1`.

ovs_link
    When ovs is enabled, a OpenvSwitch will be initialized on new nodes
    and will have this as its connection to the outside. This parameter
    is not set per default, as it depends very much on the specific
    setup.

142 143 144 145 146 147 148 149
ssh_port
    The port used for SSH connections to nodes belonging to a group. The user
    is responsible for properly configuring the ports of SSH daemons on
    machines prior to adding them as Ganeti nodes or when modifying the
    parameter value of an existing group. Note that using non-standard SSH
    ports and downgrading to an older Ganeti version that doesn't support
    ``ssh_port`` will break the cluster.

150

151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177
Hypervisor State Parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using ``--hypervisor-state`` you can set hypervisor specific states as
pointed out in ``Ganeti Resource Model <design-resource-model.rst>``.

The format is: ``hypervisor:option=value``.

Currently we support the following hypervisor state values:

mem_total
  Total node memory, as discovered by this hypervisor
mem_node
  Memory used by, or reserved for, the node itself; note that some
  hypervisors can report this in an authoritative way, other not
mem_hv
  Memory used either by the hypervisor itself or lost due to instance
  allocation rounding; usually this cannot be precisely computed, but
  only roughly estimated
cpu_total
  Total node cpu (core) count; usually this can be discovered
  automatically
cpu_node
  Number of cores reserved for the node itself; this can either be
  discovered or set manually. Only used for estimating how many VCPUs
  are left for instances

178 179 180
Note that currently this option is unused by Ganeti; values will be
recorded but will not influence the Ganeti operation.

181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202

Disk State Parameters
~~~~~~~~~~~~~~~~~~~~~

Using ``--disk-state`` you can set disk specific states as pointed out
in ``Ganeti Resource Model <design-resource-model.rst>``.

The format is: ``storage_type/identifier:option=value``. Where we
currently just support ``lvm`` as storage type. The identifier in this
case is the LVM volume group. By default this is ``xenvg``.

Currently we support the following hypervisor state values:

disk_total
  Total disk size (usually discovered automatically)
disk_reserved
  Reserved disk size; this is a lower limit on the free space, if such a
  limit is desired
disk_overhead
  Disk that is expected to be used by other volumes (set via
  ``reserved_lvs``); usually should be zero

203 204 205
Note that currently this option is unused by Ganeti; values will be
recorded but will not influence the Ganeti operation.

206

207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254
Cluster configuration
~~~~~~~~~~~~~~~~~~~~~

The master node keeps and is responsible for the cluster
configuration. On the filesystem, this is stored under the
``@LOCALSTATEDIR@/ganeti/lib`` directory, and if the master daemon is
stopped it can be backed up normally.

The master daemon will replicate the configuration database called
``config.data`` and the job files to all the nodes in the master
candidate role. It will also distribute a copy of some configuration
values via the *ssconf* files, which are stored in the same directory
and start with a ``ssconf_`` prefix, to all nodes.

Jobs
~~~~

All cluster modification are done via jobs. A job consists of one
or more opcodes, and the list of opcodes is processed serially. If
an opcode fails, the entire job is failed and later opcodes are no
longer processed. A job can be in one of the following states:

queued
    The job has been submitted but not yet processed by the master
    daemon.

waiting
    The job is waiting for for locks before the first of its opcodes.

canceling
    The job is waiting for locks, but is has been marked for
    cancellation. It will not transition to *running*, but to
    *canceled*.

running
    The job is currently being executed.

canceled
    The job has been canceled before starting execution.

success
    The job has finished successfully.

error
    The job has failed during runtime, or the master daemon has been
    stopped during the job execution.


255 256 257 258 259
Common command line features
----------------------------

Options
~~~~~~~
260 261 262 263 264

Many Ganeti commands provide the following options. The
availability for a certain command can be checked by calling the
command using the ``--help`` option.

265
| **gnt-...** *command* [\--dry-run] [\--priority {low | normal | high}]
266
| [\--submit] [\--print-job-id]
267 268 269 270 271 272

The ``--dry-run`` option can be used to check whether an operation
would succeed.

The option ``--priority`` sets the priority for opcodes submitted
by the command.
273

274 275 276 277
The ``--submit`` option is used to send the job to the master daemon but
not wait for its completion. The job ID will be shown so that it can be
examined using **gnt-job info**.

278 279 280
The ``--print-job-id`` option makes the command print the job id as first
line on stdout, so that it is easy to parse by other programs.

281 282 283 284 285 286 287 288 289 290 291 292 293 294 295
Defaults
~~~~~~~~

For certain commands you can use environment variables to provide
default command line arguments. Just assign the arguments as a string to
the corresponding environment variable. The format of that variable
name is **binary**_*command*. **binary** is the name of the ``gnt-*``
script all upper case and dashes replaced by underscores, and *command*
is the command invoked on that script.

Currently supported commands are ``gnt-node list``, ``gnt-group list``
and ``gnt-instance list``. So you can configure default command line
flags by setting ``GNT_NODE_LIST``, ``GNT_GROUP_LIST`` and
``GNT_INSTANCE_LIST``.

296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311
Debug options
~~~~~~~~~~~~~

If the variable ``FORCE_LUXI_SOCKET`` is set, it will override the
socket used for LUXI connections by command-line tools
(``gnt-*``). This is useful mostly for debugging, and some operations
won't work at all if, for example, you point this variable to the
confd-supplied query socket and try to submit a job.

If the variable is set to the value ``master``, it will connect to the
correct path for the master daemon (even if, for example, split
queries are enabled and this is a query operation). If set to
``query``, it will always (try to) connect to the query socket, even
if split queries are disabled. Otherwise, the value is taken to
represent a filesystem path to the socket to use.

312 313
Field formatting
----------------
314 315 316 317 318 319 320

Multiple ganeti commands use the same framework for tabular listing of
resources (e.g. **gnt-instance list**, **gnt-node list**, **gnt-group
list**, **gnt-debug locks**, etc.). For these commands, special states
are denoted via a special symbol (in terse mode) or a string (in
verbose mode):

321
\*, (offline)
322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345
    The node in question is marked offline, and thus it cannot be
    queried for data. This result is persistent until the node is
    de-offlined.

?, (nodata)
    Ganeti expected to receive an answer from this entity, but the
    cluster RPC call failed and/or we didn't receive a valid answer;
    usually more information is available in the node daemon log (if
    the node is alive) or the master daemon log. This result is
    transient, and re-running command might return a different result.

-, (unavail)
    The respective field doesn't make sense for this entity;
    e.g. querying a down instance for its current memory 'live' usage,
    or querying a non-vm_capable node for disk/memory data. This
    result is persistent, and until the entity state is changed via
    ganeti commands, the result won't change.

??, (unknown)
    This field is not known (note that this is different from entity
    being unknown). Either you have mis-typed the field name, or you
    are using a field that the running Ganeti master daemon doesn't
    know. This result is persistent, re-running the command won't
    change it.
346 347 348 349 350 351 352 353 354 355 356 357 358 359

Key-value parameters
~~~~~~~~~~~~~~~~~~~~

Multiple options take parameters that are of the form
``key=value,key=value,...`` or ``category:key=value,...``. Examples
are the hypervisor parameters, backend parameters, etc. For these,
it's possible to use values that contain commas by escaping with via a
backslash (which needs two if not single-quoted, due to shell
behaviour)::

  # gnt-instance modify -H kernel_path=an\\,example instance1
  # gnt-instance modify -H kernel_path='an\,example' instance1

360 361 362 363 364 365 366 367 368
Query filters
~~~~~~~~~~~~~

Most commands listing resources (e.g. instances or nodes) support filtering.
The filter language is similar to Python expressions with some elements from
Perl. The language is not generic. Each condition must consist of a field name
and a value (except for boolean checks), a field can not be compared to another
field. Keywords are case-sensitive.

369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392
Examples (see below for syntax details):

- List webservers::

    gnt-instance list --filter 'name =* "web*.example.com"'

- List instances with three or six virtual CPUs and whose primary
  nodes reside in groups starting with the string "rack"::

    gnt-instance list --filter
      '(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/'

- Nodes hosting primary instances::

    gnt-node list --filter 'pinst_cnt != 0'

- Nodes which aren't master candidates::

    gnt-node list --filter 'not master_candidate'

- Short version for globbing patterns::

    gnt-instance list '*.site1' '*.site2'

393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414
Syntax in pseudo-BNF::

  <quoted-string> ::= /* String quoted with single or double quotes,
                         backslash for escaping */

  <integer> ::= /* Number in base-10 positional notation */

  <re> ::= /* Regular expression */

  /*
    Modifier "i": Case-insensitive matching, see
    http://docs.python.org/library/re#re.IGNORECASE

    Modifier "s": Make the "." special character match any character,
    including newline, see http://docs.python.org/library/re#re.DOTALL
  */
  <re-modifiers> ::= /* empty */ | i | s

  <value> ::= <quoted-string> | <integer>

  <condition> ::=
    { /* Value comparison */
415
      <field> { == | != | < | <= | >= | > } <value>
416 417 418 419 420 421 422 423 424

      /* Collection membership */
      | <value> [ not ] in <field>

      /* Regular expressions (recognized delimiters
         are "/", "#", "^", and "|"; backslash for escaping)
      */
      | <field> { =~ | !~ } m/<re>/<re-modifiers>

425 426 427
      /* Globbing */
      | <field> { =* | !* } <quoted-string>

428 429 430 431 432 433 434 435 436 437 438 439 440 441
      /* Boolean */
      | <field>
    }

  <filter> ::=
    { [ not ] <condition> | ( <filter> ) }
    [ { and | or } <filter> ]

Operators:

*==*
  Equality
*!=*
  Inequality
442 443 444 445 446 447 448 449
*<*
  Less than
*<=*
  Less than or equal
*>*
  Greater than
*>=*
  Greater than or equal
450 451 452 453
*=~*
  Pattern match using regular expression
*!~*
  Logically negated from *=~*
454
*=\**
455
  Globbing, see **glob**\(7), though only * and ? are supported
456 457
*!\**
  Logically negated from *=\**
458 459 460 461
*in*, *not in*
  Collection membership and negation


462 463 464 465
Common daemon functionality
---------------------------

All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal.
466
**logrotate**\(8) can be used to rotate Ganeti's log files.
467 468 469 470 471 472

.. vim: set textwidth=72 :
.. Local Variables:
.. mode: rst
.. fill-column: 72
.. End: