hooks.rst 14 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
Ganeti customisation using hooks
================================

Documents ganeti version 2.0

.. contents::

Introduction
------------


In order to allow customisation of operations, ganeti runs scripts
under ``/etc/ganeti/hooks`` based on certain rules.


This is similar to the ``/etc/network/`` structure present in Debian
for network interface handling.

Organisation
------------

For every operation, two sets of scripts are run:

- pre phase (for authorization/checking)
- post phase (for logging)

Also, for each operation, the scripts are run on one or more nodes,
depending on the operation type.

Note that, even though we call them scripts, we are actually talking
about any executable.

*pre* scripts
~~~~~~~~~~~~~

The *pre* scripts have a definite target: to check that the operation
is allowed given the site-specific constraints. You could have, for
example, a rule that says every new instance is required to exists in
a database; to implement this, you could write a script that checks
the new instance parameters against your database.

The objective of these scripts should be their return code (zero or
non-zero for success and failure). However, if they modify the
environment in any way, they should be idempotent, as failed
executions could be restarted and thus the script(s) run again with
exactly the same parameters.

Note that if a node is unreachable at the time a hooks is run, this
will not be interpreted as a deny for the execution. In other words,
only an actual error returned from a script will cause abort, and not
an unreachable node.

Therefore, if you want to guarantee that a hook script is run and
denies an action, it's best to put it on the master node.

*post* scripts
~~~~~~~~~~~~~~

These scripts should do whatever you need as a reaction to the
completion of an operation. Their return code is not checked (but
logged), and they should not depend on the fact that the *pre* scripts
have been run.

Naming
~~~~~~

The allowed names for the scripts consist of (similar to *run-parts* )
upper and lower case, digits, underscores and hyphens. In other words,
the regexp ``^[a-zA-Z0-9_-]+$``. Also, non-executable scripts will be
ignored.


Order of execution
~~~~~~~~~~~~~~~~~~

On a single node, the scripts in a directory are run in lexicographic
order (more exactly, the python string comparison order). It is
advisable to implement the usual *NN-name* convention where *NN* is a
two digit number.

For an operation whose hooks are run on multiple nodes, there is no
specific ordering of nodes with regard to hooks execution; you should
assume that the scripts are run in parallel on the target nodes
(keeping on each node the above specified ordering).  If you need any
kind of inter-node synchronisation, you have to implement it yourself
in the scripts.

Execution environment
~~~~~~~~~~~~~~~~~~~~~

The scripts will be run as follows:

- no command line arguments

- no controlling *tty*

- stdin is actually */dev/null*

- stdout and stderr are directed to files

- PATH is reset to ``/sbin:/bin:/usr/sbin:/usr/bin``

- the environment is cleared, and only ganeti-specific variables will
  be left


Michael Hanselmann's avatar
Michael Hanselmann committed
107
All information about the cluster is passed using environment
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
variables. Different operations will have sligthly different
environments, but most of the variables are common.

Operation list
--------------

Node operations
~~~~~~~~~~~~~~~

OP_ADD_NODE
+++++++++++

Adds a node to the cluster.

:directory: node-add
:env. vars: NODE_NAME, NODE_PIP, NODE_SIP
:pre-execution: all existing nodes
:post-execution: all nodes plus the new node


OP_REMOVE_NODE
++++++++++++++

131
132
Removes a node from the cluster. On the removed node the hooks are called
during the execution of the operation and not after its completion.
133
134
135
136

:directory: node-remove
:env. vars: NODE_NAME
:pre-execution: all existing nodes except the removed node
137
:post-execution: all existing nodes
138
139
140
141
142
143
144
145
146
147
148

OP_NODE_SET_PARAMS
++++++++++++++++++

Changes a node's parameters.

:directory: node-modify
:env. vars: MASTER_CANDIDATE, OFFLINE, DRAINED
:pre-execution: master node, the target node
:post-execution: master node, the target node

149
150
151
152
153
154
155
156
157
158
OP_NODE_EVACUATE
++++++++++++++++

Relocate secondary instances from a node.

:directory: node-evacuate
:env. vars: NEW_SECONDARY, NODE_NAME
:pre-execution: master node, target node
:post-execution: master node, target node

159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308

Instance operations
~~~~~~~~~~~~~~~~~~~

All instance operations take at least the following variables:
INSTANCE_NAME, INSTANCE_PRIMARY, INSTANCE_SECONDARIES,
INSTANCE_OS_TYPE, INSTANCE_DISK_TEMPLATE, INSTANCE_MEMORY,
INSTANCE_DISK_SIZES, INSTANCE_VCPUS, INSTANCE_NIC_COUNT,
INSTANCE_NICn_IP, INSTANCE_NICn_BRIDGE, INSTANCE_NICn_MAC,
INSTANCE_DISK_COUNT, INSTANCE_DISKn_SIZE, INSTANCE_DISKn_MODE.

The INSTANCE_NICn_* and INSTANCE_DISKn_* variables represent the
properties of the *n* -th NIC and disk, and are zero-indexed.


OP_INSTANCE_ADD
+++++++++++++++

Creates a new instance.

:directory: instance-add
:env. vars: ADD_MODE, SRC_NODE, SRC_PATH, SRC_IMAGES
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_REINSTALL
+++++++++++++++++++++

Reinstalls an instance.

:directory: instance-reinstall
:env. vars: only the standard instance vars
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_BACKUP_EXPORT
++++++++++++++++

Exports the instance.


:directory: instance-export
:env. vars: EXPORT_NODE, EXPORT_DO_SHUTDOWN
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_START
+++++++++++++++++

Starts an instance.

:directory: instance-start
:env. vars: INSTANCE_NAME, INSTANCE_PRIMARY, INSTANCE_SECONDARIES, FORCE
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_SHUTDOWN
++++++++++++++++++++

Stops an instance.

:directory: instance-shutdown
:env. vars: INSTANCE_NAME, INSTANCE_PRIMARY, INSTANCE_SECONDARIES
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_REBOOT
++++++++++++++++++

Reboots an instance.

:directory: instance-reboot
:env. vars: IGNORE_SECONDARIES, REBOOT_TYPE
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_MODIFY
++++++++++++++++++

Modifies the instance parameters.

:directory: instance-modify
:env. vars: INSTANCE_NAME, MEM_SIZE, VCPUS, INSTANCE_IP
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_FAILOVER
++++++++++++++++++++

Failovers an instance.

:directory: instance-failover
:env. vars: IGNORE_CONSISTENCY
:pre-execution: master node, secondary node
:post-execution: master node, secondary node

OP_INSTANCE_MIGRATE
++++++++++++++++++++

Migrates an instance.

:directory: instance-failover
:env. vars: INSTANCE_MIGRATE_LIVE, INSTANCE_MIGRATE_CLEANUP
:pre-execution: master node, secondary node
:post-execution: master node, secondary node


OP_INSTANCE_REMOVE
++++++++++++++++++

Remove an instance.

:directory: instance-remove
:env. vars: INSTANCE_NAME, INSTANCE_PRIMARY, INSTANCE_SECONDARIES
:pre-execution: master node
:post-execution: master node

OP_INSTANCE_REPLACE_DISKS
+++++++++++++++++++++++++

Replace an instance's disks.

:directory: mirror-replace
:env. vars: MODE, NEW_SECONDARY, OLD_SECONDARY
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

OP_INSTANCE_GROW_DISK
+++++++++++++++++++++

Grows the disk of an instance.

:directory: disk-grow
:env. vars: DISK, AMOUNT
:pre-execution: master node, primary node
:post-execution: master node, primary node

OP_INSTANCE_RENAME
++++++++++++++++++

Renames an instance.

:directory: instance-rename
:env. vars: INSTANCE_NEW_NAME
:pre-execution: master node, primary and secondary nodes
:post-execution: master node, primary and secondary nodes

Cluster operations
~~~~~~~~~~~~~~~~~~

309
310
311
312
313
314
315
316
317
318
OP_POST_INIT_CLUSTER
++++++++++++++++++++

This hook is called via a special "empty" LU right after cluster initialization.

:directory: cluster-init
:env. vars: none
:pre-execution: none
:post-execution: master node

Luca Bigliardi's avatar
Luca Bigliardi committed
319
320
321
322
323
324
325
326
327
328
329
OP_DESTROY_CLUSTER
++++++++++++++++++

The post phase of this hook is called during the execution of destroy operation
and not after its completion.

:directory: cluster-destroy
:env. vars: none
:pre-execution: none
:post-execution: master node

330
331
332
333
334
335
336
337
338
OP_CLUSTER_VERIFY
+++++++++++++++++

Verifies the cluster status. This is a special LU with regard to
hooks, as the result of the opcode will be combined with the result of
post-execution hooks, in order to allow administrators to enhance the
cluster verification procedure.

:directory: cluster-verify
339
:env. vars: CLUSTER, MASTER, CLUSTER_TAGS, NODE_TAGS_<name>
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
:pre-execution: none
:post-execution: all nodes

OP_CLUSTER_RENAME
+++++++++++++++++

Renames the cluster.

:directory: cluster-rename
:env. vars: NEW_NAME
:pre-execution: master-node
:post-execution: master-node

OP_CLUSTER_SET_PARAMS
+++++++++++++++++++++

Modifies the cluster parameters.

:directory: cluster-modify
:env. vars: NEW_VG_NAME
:pre-execution: master node
:post-execution: master node


Obsolete operations
~~~~~~~~~~~~~~~~~~~

The following operations are no longer present or don't execute hooks
anymore in Ganeti 2.0:

- OP_INIT_CLUSTER
- OP_MASTER_FAILOVER
- OP_INSTANCE_ADD_MDDRBD
- OP_INSTANCE_REMOVE_MDDRBD


Environment variables
---------------------

Note that all variables listed here are actually prefixed with
*GANETI_* in order to provide a clear namespace.

Common variables
~~~~~~~~~~~~~~~~

This is the list of environment variables supported by all operations:

HOOKS_VERSION
  Documents the hooks interface version. In case this doesnt match
  what the script expects, it should not run. The documents conforms
  to the version 2.

HOOKS_PHASE
  One of *PRE* or *POST* denoting which phase are we in.

CLUSTER
  The cluster name.

MASTER
  The master node.

OP_CODE
  One of the *OP_* values from the list of operations.

OBJECT_TYPE
  One of ``INSTANCE``, ``NODE``, ``CLUSTER``.

DATA_DIR
  The path to the Ganeti configuration directory (to read, for
  example, the *ssconf* files).


Specialised variables
~~~~~~~~~~~~~~~~~~~~~

This is the list of variables which are specific to one or more
operations.

INSTANCE_NAME
  The name of the instance which is the target of the operation.

INSTANCE_DISK_TEMPLATE
  The disk type for the instance.

INSTANCE_DISK_COUNT
  The number of disks for the instance.

INSTANCE_DISKn_SIZE
  The size of disk *n* for the instance.

INSTANCE_DISKn_MODE
  Either *rw* for a read-write disk or *ro* for a read-only one.

INSTANCE_NIC_COUNT
  The number of NICs for the instance.

INSTANCE_NICn_BRIDGE
  The bridge to which the *n* -th NIC of the instance is attached.

INSTANCE_NICn_IP
  The IP (if any) of the *n* -th NIC of the instance.

INSTANCE_NICn_MAC
  The MAC address of the *n* -th NIC of the instance.

INSTANCE_OS_TYPE
  The name of the instance OS.

INSTANCE_PRIMARY
  The name of the node which is the primary for the instance.

INSTANCE_SECONDARIES
  Space-separated list of secondary nodes for the instance.

INSTANCE_MEMORY
  The memory size (in MiBs) of the instance.

INSTANCE_VCPUS
  The number of virtual CPUs for the instance.

INSTANCE_STATUS
  The run status of the instance.

NODE_NAME
  The target node of this operation (not the node on which the hook
  runs).

NODE_PIP
  The primary IP of the target node (the one over which inter-node
  communication is done).

NODE_SIP
  The secondary IP of the target node (the one over which drbd
  replication is done). This can be equal to the primary ip, in case
  the cluster is not dual-homed.

FORCE
  This is provided by some operations when the user gave this flag.

IGNORE_CONSISTENCY
  The user has specified this flag. It is used when failing over
  instances in case the primary node is down.

ADD_MODE
  The mode of the instance create: either *create* for create from
  scratch or *import* for restoring from an exported image.

SRC_NODE, SRC_PATH, SRC_IMAGE
  In case the instance has been added by import, these variables are
  defined and point to the source node, source path (the directory
  containing the image and the config file) and the source disk image
  file.

NEW_SECONDARY
  The name of the node on which the new mirror component is being
  added. This can be the name of the current secondary, if the new
  mirror is on the same secondary.

OLD_SECONDARY
  The name of the old secondary in the replace-disks command Note that
  this can be equal to the new secondary if the secondary node hasn't
  actually changed.

EXPORT_NODE
  The node on which the exported image of the instance was done.

EXPORT_DO_SHUTDOWN
  This variable tells if the instance has been shutdown or not while
  doing the export. In the "was shutdown" case, it's likely that the
  filesystem is consistent, whereas in the "did not shutdown" case,
  the filesystem would need a check (journal replay or full fsck) in
  order to guarantee consistency.

513
514
515
516
517
518
CLUSTER_TAGS
  The list of cluster tags, space separated.

NODE_TAGS_<name>
  The list of tags for node *<name>*, space separated.

519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
Examples
--------

The startup of an instance will pass this environment to the hook
script::

  GANETI_CLUSTER=cluster1.example.com
  GANETI_DATA_DIR=/var/lib/ganeti
  GANETI_FORCE=False
  GANETI_HOOKS_PATH=instance-start
  GANETI_HOOKS_PHASE=post
  GANETI_HOOKS_VERSION=2
  GANETI_INSTANCE_DISK0_MODE=rw
  GANETI_INSTANCE_DISK0_SIZE=128
  GANETI_INSTANCE_DISK_COUNT=1
  GANETI_INSTANCE_DISK_TEMPLATE=drbd
  GANETI_INSTANCE_MEMORY=128
  GANETI_INSTANCE_NAME=instance2.example.com
  GANETI_INSTANCE_NIC0_BRIDGE=xen-br0
  GANETI_INSTANCE_NIC0_IP=
  GANETI_INSTANCE_NIC0_MAC=aa:00:00:a5:91:58
  GANETI_INSTANCE_NIC_COUNT=1
  GANETI_INSTANCE_OS_TYPE=debootstrap
  GANETI_INSTANCE_PRIMARY=node3.example.com
  GANETI_INSTANCE_SECONDARIES=node5.example.com
  GANETI_INSTANCE_STATUS=down
  GANETI_INSTANCE_VCPUS=1
  GANETI_MASTER=node1.example.com
  GANETI_OBJECT_TYPE=INSTANCE
  GANETI_OP_CODE=OP_INSTANCE_STARTUP
  GANETI_OP_TARGET=instance2.example.com