usage.rst 19.4 KB
Newer Older
1
Usage
2
^^^^^
3 4

snf-image-creator comes in 2 variants:
5

6 7
 * snf-mkimage: A non-interactive command line program
 * snf-image-creator: A user-friendly dialog-based program
8

9 10 11 12
Both expect the input media as first argument. The input media may be a local
file, a block device or *"/"* if you want to create an image out of the running
system (see `host bundling operation`_).

13
Non-interactive version
14
=======================
15

16
snf-mkimage receives the following options:
17 18 19

.. code-block:: console

20
  # snf-mkimage --help
21
  Usage: snf-mkimage [options] <input_media>
22 23 24 25 26 27 28

  Options:
    --version             show program's version number and exit
    -h, --help            show this help message and exit
    -a URL, --authentication-url=URL
                          use this authentication URL when uploading/registering
                          images
29 30
    --allow-unsupported   proceed with the image creation even if the media is
                          not supported
31 32 33 34 35 36
    -c CLOUD, --cloud=CLOUD
                          use this saved cloud account to authenticate against a
                          cloud when uploading/registering images
    --disable-sysprep=SYSPREP
                          prevent SYSPREP operation from running on the input
                          media
37 38 39 40 41 42 43 44 45
    --enable-sysprep=SYSPREP
                          run SYSPREP operation on the input media
    -f, --force           overwrite output files if they exist
    --host-run=SCRIPT     mount the media in the host and run a script against
                          the guest media. This option may be defined multiple
                          times. The script's working directory will be the
                          guest's root directory. BE CAREFUL! DO NOT USE
                          ABSOLUTE PATHS INSIDE THE SCRIPT! YOU MAY HARM YOUR
                          SYSTEM!
46
    --install-virtio=DIR  install VirtIO drivers hosted under DIR (Windows only)
47 48 49 50 51 52 53 54 55 56
    -m KEY=VALUE, --metadata=KEY=VALUE
                          add custom KEY=VALUE metadata to the image
    --no-snapshot         don't snapshot the input media. (THIS IS DANGEROUS AS
                          IT WILL ALTER THE ORIGINAL MEDIA!!!)
    --no-sysprep          don't perform any system preparation operation
    -o FILE, --outfile=FILE
                          dump image to FILE
    --print-metadata      print the detected image metadata
    --print-syspreps      print the enabled and disabled system preparation
                          operations for this input media
57
    --print-sysprep-params
58 59
                          print the defined system preparation parameters for
                          this input media
60 61 62 63
    --public              register image with the cloud as public
    -r IMAGENAME, --register=IMAGENAME
                          register the image with a cloud as IMAGENAME
    -s, --silent          output only errors
64
    --sysprep-param=SYSPREP_PARAMS
65
                          add KEY=VALUE system preparation parameter
66 67 68
    -t TOKEN, --token=TOKEN
                          use this authentication token when
                          uploading/registering images
69
    --tmpdir=DIR          create large temporary image files under DIR
70 71
    -u FILENAME, --upload=FILENAME
                        upload the image to the cloud with name FILENAME
72 73

Most input options are self-describing. If you want to save a local copy of
Nikos Skalkotos's avatar
Nikos Skalkotos committed
74
the image you create, provide a filename using the *-o* option. To upload the
75 76 77 78
image to the storage service of a cloud, provide valid cloud API access info
(by either using a token and a URL with *-t* and *-a* respectively, or a cloud
name with *-c*) and a remote filename using *-u*. If you also want to register
the image with the compute service of the cloud, in addition to *-u* provide a
79 80 81
registration name using *-r*. All images are registered as *private*. Only the
user that registers the image can create VM's out of it. If you want the image
to be visible by other user too, use the *--public* option.
Nikos Skalkotos's avatar
Nikos Skalkotos committed
82

83 84 85 86 87 88 89 90
By default, before extracting the image, snf-mkimage will perform a number of
system preparation operations on the snapshot of the media. You can disable
this by specifying *--no-sysprep*.

You may use the *--host-run* option multiple times to define scripts that will
run on the image's locally mounted root directory before the image
customization is performed. Be careful when using those. The scripts run on the
host system without any jail protection. Use only relative paths.
91

92
If *--print-sysprep* is defined, the program will exit after printing a
93
list of enabled and disabled system preparation operations applicable to this
Nikos Skalkotos's avatar
Nikos Skalkotos committed
94 95 96
input media. The user can enable or disable specific *syspreps*, using
*-{enable,disable}-sysprep* options. The user may specify those options
multiple times.
97

98 99
Running *snf-mkimage* with *--print-sysprep* on a raw file that hosts an
Ubuntu system, will print the following output:
100

101 102
.. _sysprep:

103 104
.. code-block:: console

105
  # snf-mkimage --print-syspreps ubuntu.raw
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165

  snf-image-creator 0.7
  ===========================
  Examining source media `ubuntu.raw' ... looks like an image file
  Snapshotting media source ... done
  Enabling recovery proc
  Launching helper VM (may take a while) ... done
  Inspecting Operating System ... ubuntu
  Collecting image metadata ... done

  Running OS inspection:
  Checking if the media contains logical volumes (LVM)... no

  Enabled system preparation operations:
      cleanup-tmp:
          Remove all files under /tmp and /var/tmp

      remove-swap-entry:
          Remove swap entry from /etc/fstab. If swap is the last partition
          then the partition will be removed when shrinking is performed. If the
          swap partition is not the last partition in the disk or if you are not
          going to shrink the image you should probably disable this.

      cleanup-cache:
          Remove all regular files under /var/cache

      cleanup-userdata:
          Delete sensitive user data

      cleanup-passwords:
          Remove all passwords and lock all user accounts

      cleanup-log:
          Empty all files under /var/log

      remove-persistent-net-rules:
          Remove udev rules that will keep network interface names persistent
          after hardware changes and reboots. Those rules will be created again
          the next time the image runs.

      use-persistent-block-device-names:
          Scan fstab & grub configuration files and replace all non-persistent
          device references with UUIDs.

      fix-acpid:
          Replace acpid powerdown action scripts to immediately shutdown the
          system without checking if a GUI is running.

      shrink:
          Shrink the last file system and update the partition table

  Disabled system preparation operations:
      remove-user-accounts:
          Remove all user accounts with id greater than 1000

      cleanup-mail:
          Remove all files under /var/mail and /var/spool/mail


  cleaning up ...
166

167 168
If you want the image to have all normal user accounts and all mail files
removed, you should use *--enable-sysprep* option like this:
169 170 171

.. code-block:: console

172
   $ snf-mkimage --enable-sysprep cleanup-mail --enable-sysprep remove-user-accounts ...
173

174 175 176 177 178 179 180 181 182
Sysprep parameters are parameters used by some sysprep tasks. In most cases you
don't need to change their value. You can see the available sysprep parameters
and the default values they have by using the *--print-sysprep-params* option.
You can update their values by using the *--sysprep-param* option.

If the media is a Windows image, you can install or update its VirtIO drivers
by using the *--install-virtio* option. With this option you can point to a
directory that hosts a set of extracted Windows VirtIO drivers.

183
Dialog-based version
184
====================
185

186
*snf-image-creator* receives the following options:
187 188 189

.. code-block:: console

190
 # snf-image-creator --help
191
 Usage: snf-image-creator [options] [<input_media>]
192

193 194 195 196 197 198
 Options:
   --version             show program's version number and exit
   -h, --help            show this help message and exit
   -l FILE, --logfile=FILE
                         log all messages to FILE
   --tmpdir=DIR          create large temporary image files under DIR
199

200
If the input media is not specified in the command line, in the first dialog
201 202 203 204 205 206 207 208 209
box the user will be asked to specify it:

.. image:: /snapshots/select_media.png

The user can select a file (regular or block device) or use the *Bundle Host*
button to create an image out of the running system (see
`Host bundling operation`_).

After the input media is examined and the program is initialized, the user will
210
be given the choice to run *snf-image-creator* in *wizard* or *expert* mode.
211 212 213 214

Wizard mode
-----------

215 216
When *snf-image-creator* runs in *wizard* mode, the user is just asked to
provide the following basic information:
217

Nikos Skalkotos's avatar
Nikos Skalkotos committed
218
 * Cloud: The cloud account to use to upload and register the resulting image
Alex Pyrgiotis's avatar
Alex Pyrgiotis committed
219
 * Name: A short name for the image (ex. "Slackware")
220 221
 * Description: An one-line description for the image
   (ex. "Slackware Linux 14.0 with KDE")
222
 * VirtIO: A directory that hosts VirtIO drivers (for Windows images only)
Nikos Skalkotos's avatar
Nikos Skalkotos committed
223
 * Registration Type: Private or Public
224

225 226 227
After confirming, the image will be extracted, uploaded to the storage service
and registered with the compute service of the selected cloud. The user will
also be given the choice to keep a local copy of it.
228 229

For most users the functionality this mode provides should be sufficient.
230 231 232 233 234

Expert mode
-----------

Expert mode allows the user to have better control on the image creation
Nikos Skalkotos's avatar
Nikos Skalkotos committed
235
process. The main menu can be seen in the picture below:
236 237 238

.. image:: /snapshots/main_menu.png

239
In the *Customize* sub-menu the user can control:
240

241
 * The installation of VirtIO drivers (only for Windows images)
242
 * The system preparation operations that will be applied on the media
243
 * The properties associated with the image
244
 * The configuration tasks that will run during image deployment
245

246
In the *Register* sub-menu the user can provide:
247

Nikos Skalkotos's avatar
Nikos Skalkotos committed
248
 * Which cloud account to use
249 250 251
 * A filename for the uploaded *diskdump* image
 * A name for the image to use when registering it with the storage service of
   the cloud, as well as the registration type (*private* or *public*)
252

253 254
By choosing the *Extract* menu entry, the user can dump the image to the local
file system. Finally, if the user selects *Reset*, the system will ignore
255
all changes made so far and will start the image creation process again.
256

257 258 259 260
Host bundling operation
=======================

As a new feature in *v0.2*, snf-image-creator can create images out of the host
261 262 263 264 265 266
system that runs the program. This is done either by specifying */* as input
media or by using the *Bundle Host* button in the media selection dialog.
During this operation, the files of the disk are copied into a temporary image
file, which means that the file system that will host the temporary image needs
to have a lot of free space (see `large temporary files`_ for more
information).
267

268 269
Creating a new image
====================
270

271 272
Suppose your host system is a Debian Wheezy and you want to create a new Ubuntu
server image. Download the installation disk from the Internet:
273

274
.. code-block:: console
275

Nikos Skalkotos's avatar
Nikos Skalkotos committed
276
   $ wget http://ubuntureleases.tsl.gr/12.04.2/ubuntu-12.04.2-server-amd64.iso
277

278
Verify that it has been downloaded correctly:
279

280
.. code-block:: console
281

Nikos Skalkotos's avatar
Nikos Skalkotos committed
282
   $ echo 'a8c667e871f48f3a662f3fbf1c3ddb17  ubuntu-12.04.2-server-amd64.iso' > check.md5
283 284
   $ md5sum -c check.md5

Alex Pyrgiotis's avatar
Alex Pyrgiotis committed
285
Create a 2G sparse file to host the new system:
286

287
.. code-block:: console
288

Nikos Skalkotos's avatar
Nikos Skalkotos committed
289
   $ truncate -s 2G ubuntu.raw
290

291
And install the Ubuntu system on this file:
292

293
.. code-block:: console
294

Nikos Skalkotos's avatar
Nikos Skalkotos committed
295
   $ sudo kvm -boot d -drive file=ubuntu.raw,format=raw,cache=none,if=virtio \
Nikos Skalkotos's avatar
Nikos Skalkotos committed
296
     -m 1G -cdrom ubuntu-12.04.2-server-amd64.iso
297

298
.. warning::
299

300
   During the installation, you will be asked about the partition scheme. Don't
301
   use LVM partitions. They are not supported by snf-image-creator.
302

Nikos Skalkotos's avatar
Nikos Skalkotos committed
303
You will be able to boot your installed OS and make any changes you want
304
(e.g. install OpenSSH Server) using the following command:
305 306

.. code-block:: console
307

Nikos Skalkotos's avatar
Nikos Skalkotos committed
308
   $ sudo kvm -m 1G -boot c -drive file=ubuntu.raw,format=raw,cache=none,if=virtio
309

310 311
After you're done, you may use *snf-image-creator* as root to create and upload
the image:
312

313
.. code-block:: console
314

315
   $ sudo -s
316
   # snf-image-creator ubuntu.raw
317

318 319
In the first screen you will be asked to choose if you want to run the program
in *Wizard* or *Expert* mode. Choose *Wizard*.
320

Nikos Skalkotos's avatar
Nikos Skalkotos committed
321
.. image:: /snapshots/wizard.png
322

323 324 325
Then you will be asked to select a cloud and provide a name, a description and
a registration type (*private* or *public*). Finally, you'll be asked to
confirm the provided data.
326

Nikos Skalkotos's avatar
Nikos Skalkotos committed
327
.. image:: /snapshots/confirm.png
328

329
Choosing *YES* will create and upload the image to your cloud account.
330

331 332 333
Working with different image formats
====================================

334 335
*snf-image-creator* is able to work with the most popular disk image formats.
It has been successfully tested with:
336

337 338
* Raw disk images
* VMDK (VMware)
339
* VHD (Microsoft Hyper-V) [#f1]_
340 341
* VDI (VirtualBox)
* qcow2 (QEMU)
342

343 344
It can support any image format QEMU supports as long as it represents a
bootable hard drive.
345

Nikos Skalkotos's avatar
Nikos Skalkotos committed
346 347 348 349 350 351
Limitations
===========

Supported operating systems
---------------------------

352 353 354
*snf-image-creator* can fully function on input media hosting *Linux*,
*FreeBSD*, *OpenBSD*, *NetBSD* and *Windows* (Server starting from 2008R2 and
Desktop starting from 7) systems.
Nikos Skalkotos's avatar
Nikos Skalkotos committed
355 356 357 358

Logical Volumes
---------------

359
The program cannot work on input media that contain LVM partitions inside
360
[#f2]_. The input media may only contain primary or logical partitions.
361

362 363
Para-virtualized drivers
------------------------
364

365
Most Synnefo deployments uses the *VirtIO* framework. The disk I/O controller
366 367 368 369 370 371
and the Ethernet cards on the VM instances are para-virtualized and need
special *VirtIO* drivers. Those drivers are included in the Linux Kernel
mainline since version 2.6.25 and are shipped with all the popular Linux
distributions. The problem is that if the driver for the para-virtualized disk
I/O controller is built as module, it needs to be preloaded using an initial
ramdisk, otherwise the VM won't be able to boot.
372

Nikos Skalkotos's avatar
Nikos Skalkotos committed
373
Many popular Linux distributions, like Ubuntu and Debian, will automatically
374 375
create a generic initial ramdisk file that contains many different modules,
including the VirtIO drivers. Others that target more experienced users, like
376
Slackware, won't do that [#f3]_. *snf-image-creator* cannot resolve this kind
377 378 379
of problems and it's left to the user to do so. Please refer to your
distribution's documentation for more information on this. You can always check
if a system can boot with para-virtualized disk controller by launching it with
Nikos Skalkotos's avatar
Nikos Skalkotos committed
380
kvm using the *if=virtio* option (see the kvm command in the
381 382
`Creating a new image`_ section).

383 384
For Windows the program supports installing VirtIO drivers. You may download
the latest drivers from the
385 386
`Fedora Project <http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/>`_.

Nikos Skalkotos's avatar
Nikos Skalkotos committed
387 388 389
Some caveats on image creation
==============================

390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406
Image partition schemes and shrinking
-------------------------------------

When image shrinking is enabled, *snf-image-creator* will shrink the last
partition on the disk. If this is a swap partition, it will remove it, save
enough information to recreate it during image deployment and shrink the
partition that lays just before that. This will make the image smaller which
speeds up the deployment process.

During image deployment, the last partition is enlarged to occupy the available
space in the VM's hard disk and a swap partition is added at the end if a SWAP
image property is present.

Keep this in mind when creating images. It's always better to have your swap
partition placed as the last partition on the disk and have your largest
partition (*/* or */home*) just before that.

Nikos Skalkotos's avatar
Nikos Skalkotos committed
407 408
Large temporary files
---------------------
409 410 411 412 413

*snf-image-creator* may create large temporary files when running:

 * During image shrinking, the input media snapshot file may reach the size of
   the original media.
414 415
 * When bundling the host system, the temporary image file may became 10%
   larger than rest of the disk files altogether.
416

Nikos Skalkotos's avatar
Nikos Skalkotos committed
417
*/tmp* directory is not a good place for hosting large files. In many systems
418 419 420 421 422
the contents of */tmp* are stored in volatile memory and the size they may
occupy is limited. By default, *snf-image-creator* will use a heuristic
approach to determine where to store large temporary files. It will examine the
free space under */var/tmp*, the user's home directory and */mnt* and will pick
the one with the most available space. The user may overwrite this behavior and
Nikos Skalkotos's avatar
Nikos Skalkotos committed
423 424
indicate a different directory using the *tmpdir* option. This option is
supported by both *snf-image-creator* and *snf-mkimage*.
425

426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496
Troubleshooting
===============

Failures in launching libguestfs's helper VM
--------------------------------------------

The most common error you may get when using *snf-image-creator* is a failure
when launching *libguestfs*'s helper VM. *libguestfs* [#f4]_ is a library
for manipulating disk images and *snf-image-creator* makes heavy use of it.
Most of the time those errors have to do with the installation of this
library and not with *snf-image-creator* itself.

The first thing you should do when troubleshooting this is to run the
``liguestfs-test-tool`` diagnostic tool. This tool gets shipped with the
library to test if *libguestfs* works as expected. If it runs to completion
successfully, you will see this near the end:

.. code-block:: console

    ===== TEST FINISHED OK =====

and the test tool will exit with code 0.

If you get errors like this:

.. code-block:: console

   libguestfs: launch: backend=libvirt
   libguestfs: launch: tmpdir=/tmp/libguestfseKwXgq
   libguestfs: launch: umask=0022
   libguestfs: launch: euid=0
   libguestfs: libvirt version = 1001001 (1.1.1)
   libguestfs: [00012ms] connect to libvirt
   libguestfs: opening libvirt handle: URI = NULL, auth = virConnectAuthPtrDefault, flags = 0
   libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
   libguestfs: error: could not connect to libvirt (URI = NULL): Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory [code=38 domain=7]
   libguestfs-test-tool: failed to launch appliance
   libguestfs: closing guestfs handle 0x7ff0d44f8bb0 (state 0)
   libguestfs: command: run: rm
   libguestfs: command: run: \ -rf /tmp/libguestfseKwXgq

it means that *libguestfs* is configured to use *libvirt* backend by default
but the libvirt deamon is not running. You can either start libvirt deamon
(providing instructions on how to do this is out of the scope of this
tutorial) or change the default backend to *direct* by defining the
**LIBGUESTFS_BACKEND** variable like this:

.. code-block:: console

   # export LIBGUESTFS_BACKEND=direct

If you run the ``libguestfs-test-tool``, the command should finish without
errors. Do the same every time before running *snf-image-creator*.

If you get errors on *febootstrap-supermin-helper* like this one:

.. code-block:: console

   febootstrap-supermin-helper: ext2: parent directory not found: /lib:
   File not found by ext2_lookup
   libguestfs: error: external command failed, see earlier error messages
   libguestfs-test-tool: failed to launch appliance
   libguestfs: closing guestfs handle 0x7b3160 (state 0)

you probably need to update the supermin appliance (just once). On Debian
and Ubuntu systems you can do it using the command below:

.. code-block:: console

   # update-guestfs-appliance

497
.. rubric:: Footnotes
498

499 500 501
.. [#f1] http://technet.microsoft.com/en-us/library/bb676673.aspx
.. [#f2] http://sourceware.org/lvm2/
.. [#f3] http://mirrors.slackware.com/slackware/slackware-14.0/README.initrd
502
.. [#f4] http://libguestfs.org/