architecture.rst 14.7 KB
Newer Older
1
2
3
Architecture
============

4
5
6
7
8
9
10
11
12
13
14
15
16
17
Overview
^^^^^^^^

snf-image is a Ganeti OS definition. This means that Ganeti provisions a new
disk (block device) and passes it to snf-image. Then, snf-image is responsible
to deploy an Image on that disk. If snf-image returns successfully, Ganeti will
then spawn a VM with that disk as its primary disk.

Thus, snf-image is responsible for two (2) things, which are executed in two
separate steps:

| 1. Fill the newly provisioned disk with Image data
| 2. Customize the Image accordingly

18
For (1), snf-image can fetch the Image from a number of backends, as we
Nikos Skalkotos's avatar
Nikos Skalkotos committed
19
20
21
22
describe later. For (2) snf-image spawns a helper VM and runs a number of
configuration tasks inside the isolated environment. Once the last task returns
successfully, the helper VM ceases and snf-image returns the newly configured
disk to Ganeti.
23
24
25
26

The whole procedure is configurable via OS interface parameters, that can be
passed to snf-image from the Ganeti command line or RAPI.

27
snf-image is split in two components: The main program running on the Ganeti
28
host with full root privilege (*snf-image*, previously *snf-image-host*) and a
29
part running inside an unprivileged helper VM (*snf-image-helper*).
30

31
32
We describe each part in the following sections:

33
34
35
36
37
snf-image
^^^^^^^^^

This part implements the Ganeti OS interface. It extracts the Image onto the
Ganeti-provided block device, using streaming block I/O (dd with oflag=direct),
38
39
40
41
then spawns a helper VM, and passes control to snf-image-helper running inside
that helper VM. The helper VM is created using either KVM or XEN depending on
the supported hypervisor as dictated by Ganeti. It runs as an unprivileged
user.
42
43

There is no restriction on the distribution running inside the helper VM, as
44
long as it executes the snf-image-helper component automatically upon boot-up.
45
46
47
The snf-image-update-helper script is provided with snf-image to automate the
creation of a helper VM image based on Debian Stable, using multistrap.

48
49
The snf-image-helper component runs inside a specific environment, which is
created and ensured by snf-image:
50

Nikos Skalkotos's avatar
Nikos Skalkotos committed
51
 * The VM features a virtual floppy, containing an ext2 file system with all
52
   parameters needed for image customization.
53
 * The hard disk provided by Ganeti that we want to deploy and customize is
Nikos Skalkotos's avatar
Nikos Skalkotos committed
54
   accessible as the first VirtIO hard disk.
55
56
57
58
59
 * All kernel/console output is redirected to the first virtual serial console,
   and eventually finds its way into the OS definition log files that Ganeti
   maintains.
 * The helper VM is expected to output "SUCCESS" to its second serial port if
   image customization was successful inside the VM.
60
61
 * If "SUCCESS" is not returned, snf-image assumes that, execution of the
   helper VM or snf-image-helper has failed.
62
63
 * The helper VM is expected to shutdown automatically once it is done. Its
   execution is time-limited; if it has not terminated after a number of
64
65
   seconds, configurable via ``/etc/default/snf-image``, snf-image sends a
   SIGTERM and/or a SIGKILL to it.
66
67
68
69

snf-image-helper
^^^^^^^^^^^^^^^^

Nikos Skalkotos's avatar
Nikos Skalkotos committed
70
This part runs inside the helper VM during boot-up and undertakes customization
71
72
73
74
75
of the target disk. It does so, by running a number of :ref:`configuration
tasks <image-configuration-tasks>`. The exact tasks that should run, are
specified by rules found in the virtual floppy, placed there by *snf-image*,
before spawning the helper VM. *snf-image-helper* uses *runparts* to run the
tasks which are found under ``/usr/lib/snf-image-helper/tasks``.
76
77
78
79
80
81

Graphical Representation
^^^^^^^^^^^^^^^^^^^^^^^^

The architecture is presented below:

82
.. image:: images/arch.png
83

84
85
86

.. _storage-backends:

87
88
Storage backends
^^^^^^^^^^^^^^^^
89
90

As stated above, for step (1), *snf-image* is capable of fetching images that
91
92
are stored in a variety of different backends and then extracting them onto the
newly created block device. The following back-ends are supported:
Nikos Skalkotos's avatar
Nikos Skalkotos committed
93

94
95
 * **Local backend**:
   The local backend is used to retrieve images that are stored on the Ganeti
Nikos Skalkotos's avatar
Nikos Skalkotos committed
96
97
98
   node that the image deployment takes place. All local images are expected to
   be found under a predefined image directory. By default */var/lib/snf-image*
   is used, but the user may change this by overwriting the value of the
99
100
   *IMAGE_DIR* variable under ``/etc/default/snf-image``.

101
102
 * **Network backend**:
   The network backend is used to retrieve images that are accessible from the
Nikos Skalkotos's avatar
Nikos Skalkotos committed
103
104
   network. snf-image can fetch images via *http:*, *https:*, *ftp:* or
   *ftps:*, using `cURL <http://curl.haxx.se/>`_.
105

106
 * **Pithos backend**:
107
   *snf-image* contains a special command-line tool (*pithcat*) for retrieving
Nikos Skalkotos's avatar
Nikos Skalkotos committed
108
   images residing on a Pithos installation. To set up snf-image's Pithos
Chrysostomos Nanakos's avatar
Chrysostomos Nanakos committed
109
110
111
112
113
114
   backend the user needs to setup the ``PITHOS_BACKEND_STORAGE`` variable
   inside ``/etc/default/snf-image``.
   Possible values are ``nfs`` and ``rados``. If ``nfs`` is used the user needs
   to setup ``PITHOS_DATA`` variable, and when ``rados`` is used the user needs
   to setup ``PITHOS_RADOS_POOL_MAPS`` and ``PITHOS_RADOS_POOL_BLOCKS``
   accordingly.
115

116
117
 * **Null backend**:
   If the null backend is selected, no image copying is performed. This
Nikos Skalkotos's avatar
Nikos Skalkotos committed
118
119
120
121
   actually is meant for bypassing step (1) altogether. This is useful, if the
   disk provisioned by Ganeti already contains an OS installation before
   *snf-image* is executed (for example if the disk was created as a clone of
   an existing VM's hard disk).
122

123
124
.. _image-configuration-tasks:

125
126
Image Configuration Tasks
^^^^^^^^^^^^^^^^^^^^^^^^^
127

128
129
130
Configuration tasks are scripts called by snf-image-helper inside the helper VM
to accomplish various configuration steps on the newly created instance. See
below for a description of each one of them:
131
132
133

**FixPartitionTable**: Enlarges the last partition in the partition table of
the instance, to consume all the available space and optionally adds a swap
134
partition in the end. The task will fail if the environment variable
135
136
*SNF_IMAGE_DEV*, which specifies the device file of the instance's hard disk,
is missing.
137
138

**FilesystemResizeUnmounted**: Extends the file system of the last partition to
139
cover up the whole partition. This only works for ext{2,3,4}, FFS and UFS2 file
140
systems. Any other file system type is ignored and a warning is triggered. The
141
task will fail if *SNF_IMAGE_DEV* environment variable is missing.
142
143
144
145
146

**MountImage**: Mounts the root partition of the instance, specified by the
*SNF_IMAGE_PROPERTY_ROOT_PARTITION* variable. On Linux systems after the root
fs is mounted, the instance's ``/etc/fstab`` file is examined and the rest of
the disk file systems are mounted too, in a correct order. The script will fail
147
if any of the environment variables *SNF_IMAGE_DEV*,
148
*SNF_IMAGE_PROPERTY_ROOT_PARTITION* or *SNF_IMAGE_TARGET* is unset or has a
149
150
non-sane value.

Nikos Skalkotos's avatar
Nikos Skalkotos committed
151
**AddSwap**: Formats the swap partition added by *FixPartitionTable* task and
152
153
154
155
adds an appropriate swap entry in the system's ``/etc/fstab``. The script will
only run if *SNF_IMAGE_PROPERTY_SWAP* is present and will fail if
*SNF_IMAGE_TARGET* in not defined.

156
157
158
159
160
161
**DeleteSSHKeys**: On Linux and \*BSD instances, this script will clear out any
ssh keys found in the instance's disk. For Debian and Ubuntu systems, the keys
are also recreated. Besides removing files that comply to the
``/etc/ssh/ssh_*_key`` pattern, the script will also parses
``/etc/ssh/sshd_config`` file for custom keys. The only variable this script
depends on is *SNF_IMAGE_TARGET*.
162
163

**DisableRemoteDesktopConnections**: This script temporary disables RDP
164
connections on Windows instances by changing the value of *fDenyTSConnection*
165
166
167
registry key. RDP connections will be enabled back during the specialize pass
of the Windows setup. The task will fail if *SNF_IMAGE_TARGET* is not defined.

168
169
**InstallUnattend**: Installs the Unattend.xml files on Windows instances. This
is needed by Windows in order to perform an unattended setup. The
170
171
*SNF_IMAGE_TARGET* variables needs to be present for this task to run.

Nikos Skalkotos's avatar
Nikos Skalkotos committed
172
**SELinuxAutorelabel**: Creates *.autorelabel* file in Red Hat images. This is
173
needed if SELinux is enabled to enforce an automatic file system relabeling
174
during the first boot. The only environment variable required by this task is
175
176
*SNF_IMAGE_TARGET*.

177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
**AssignHostname**: Assigns or changes the hostname of the instance. The task
will fail if the Linux distribution is not supported and ``/etc/hostname`` is
not present on the file system. For now, we support Debian, Red Hat, Slackware,
SUSE and Gentoo derived distributions. The hostname is read from
*SNF_IMAGE_HOSTNAME* variable. In addition to the latter, *SNF_IMAGE_TARGET* is
also required.

**ChangePassword**: Changes the password for a list of existing users. On Linux 
systems this is accomplished by directly altering the instance's
``/etc/shadow`` file. On Windows systems a script is injected into the VM's
hard disk. This script will be executed during the specialize pass of the
Windows setup. On \*BSD systems ``/etc/master.passwd`` is altered,
``/etc/spwd.db`` is removed and a script is injected into the VM's hard disk
that will recreate the aforementioned file during the first boot. The list of
users whose passwords will changed is determined by the
Nikos Skalkotos's avatar
Nikos Skalkotos committed
192
193
194
*SNF_IMAGE_PROPERTY_USERS* variable (see :ref:`image-properties`). For this
task to run *SNF_IMAGE_TARGET* and *SNF_IMAGE_PASSWORD* variables need to be
present.
195
196
197
198
199
200
201
202

**FilesystemResizeMounted**: Injects a script into a Windows image file system
that will enlarge the last file system to cover up the whole partition. The
script will run during the specialize pass of the Windows setup. If the
*SNF_IMAGE_TARGET* variable is missing, the task will fail.

**EnforcePersonality**: Injects the files specified by the
*SNF_IMAGE_PROPERTY_OSFAMILY* variable into the file system. If the variable is
203
204
missing a warning is produced. Only *SNF_IMAGE_TARGET* is required for this
task to run.
205

206
**UmountImage**: Umounts the file systems previously mounted by MountImage. The
207
only environment variable required is *SNF_IMAGE_TARGET*.
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247


+-------------------------------+---+--------------------------------------------+--------------------------------------------------+
|                               |   |               Dependencies                 |               Enviromental Variables [#]_        |
+          Name                 |   +------------------+-------------------------+-------------------------+------------------------+
|                               |Pr.|        Run-After |        Run-Before       |        Required         |      Optional          |
+===============================+===+==================+=========================+=========================+========================+
|FixPartitionTable              |10 |                  |FilesystemResizeUnmounted|DEV                      |                        |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|FilesystemResizeUnmounted      |20 |FixPartitionTable |MountImage               |DEV                      |                        |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|MountImage                     |30 |                  |UmountImage              |DEV                      |                        |
|                               |   |                  |                         |TARGET                   |                        |
|                               |   |                  |                         |PROPERTY_ROOT_PARTITION  |                        |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|AddSwap                        |40 |MountImage        |EnforcePersonality       |TARGET                   |PROPERTY_OSFAMILY       |
|                               |   |                  |                         |                         |PROPERTY_SWAP           |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|DeleteSSHKeys                  |40 |MountImage        |EnforcePersonality       |TARGET                   |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|DisableRemoteDesktopConnections|40 |EnforcePersonality|UmountImage              |TARGET                   |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|InstallUnattend                |40 |MountImage        |EnforcePersonality       |TARGET                   |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|SELinuxAutorelabel             |40 |MountImage        |EnforcePersonality       |TARGET                   |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|AssignHostname                 |50 |InstallUnattend   |EnforcePersonality       |TARGET                   |                        |
|                               |   |                  |                         |HOSTNAME                 |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|ChangePassword                 |50 |InstallUnattend   |EnforcePersonality       |TARGET                   |PROPERTY_USERS          |
|                               |   |                  |                         |PASSWORD                 |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|FilesystemResizeMounted        |50 |InstallUnattend   |EnforcePersonality       |TARGET                   |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|EnforcePersonality             |60 |MountImage        |UmountImage              |TARGET                   |PERSONALITY             |
|                               |   |                  |                         |                         |PROPERTY_OSFAMILY       |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
|UmountImage                    |80 |MountImage        |                         |TARGET                   |                        |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+

248
.. [#] all environment variables are prefixed with *SNF_IMAGE_*