diff --git a/ChangeLog b/ChangeLog index 7c298670973893126ff21f7360d9498fa397afc7..06889f01d83639c6e7287411d78b9fe0f643f4e2 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2016-09-29, v0.20rc1 + * Add support for NetworkManager in CofigureNetwork configuration task + * Add a new Linux configuration task for regenerating the machine ID + * Update the documentation + 2016-07-20, v0.19.1 * Add "auto" to the known Linux file systems diff --git a/docs/advanced.rst b/docs/advanced.rst index ab31f35bf79088bab1b0b786937e313db99e3d21..c2f06bce5324f57c039586dfd79c2271f6d69833 100644 --- a/docs/advanced.rst +++ b/docs/advanced.rst @@ -52,16 +52,16 @@ Although *diskdump* is a lot more flexible than the older formats, there are still some rules to follow: * For Linux: - * All block devices in ``/etc/fstab`` should be specified using persistent - names (UUID or LABEL) - * LVM partitions are not supported - * Only ext{2,3,4} file systems are supported + * All block devices in ``/etc/fstab`` should be specified using persistent + names (UUID or LABEL) + * LVM partitions are not supported + * Only ext{2,3,4} file systems are supported * For FreeBSD: - * GUID Partition Tables (GPT) should be used - * Only UFS2 file systems are supported - * Labels should be omitted in ``/etc/fstab`` entries + * GUID Partition Tables (GPT) should be used + * Only UFS2 file systems are supported + * Labels should be omitted in ``/etc/fstab`` entries * For {Open,Net}BSD: - * Only FFS file systems should be used + * Only FFS file systems should be used .. _windows-deployment: @@ -144,7 +144,7 @@ one: The ``@TIMEZONE_INDEX@`` container will be replaced during the installation of the file by the index value of the time zone which is specified through the *WINDOWS_TIMEZONE* configuration variable. For more info check `here -`_. +`__. This file is known to work with Windows XP and in order for the deployment to be completely unattended, the user must provide a Windows Product key using the *os_product_key* OS parameter *snf-image* offers. As with the newer Windows @@ -291,7 +291,7 @@ environment variables that are present when the configuration tasks run. |NIC_%N_* |The Ganeti provided environment variable for the Nth | | |network interface controller. Check | | |`here `_ | +| |-os-interface.html>`__ | +---------------------+-------------------------------------------------------+ |DHCP_TAGS |The value of the DHCP_TAGS configuration parameter (see| | |:ref:`Configuration Parameters | diff --git a/docs/architecture.rst b/docs/architecture.rst index f2ab96165fcf833c2702d42b880d4cce712610de..b07ed5be8a10d3763dca4ccd3c75ceecc008152b 100644 --- a/docs/architecture.rst +++ b/docs/architecture.rst @@ -149,31 +149,22 @@ if any of the environment variables *SNF_IMAGE_DEV*, *SNF_IMAGE_PROPERTY_ROOT_PARTITION* or *SNF_IMAGE_TARGET* is unset or has a non-sane value. -**AddSwap**: Formats the swap partition added by *FixPartitionTable* task and -adds an appropriate swap entry in the system's ``/etc/fstab``. The script will -only run if *SNF_IMAGE_PROPERTY_SWAP* is present and will fail if -*SNF_IMAGE_TARGET* in not defined. - -**DeleteSSHKeys**: On Linux and \*BSD instances, this script will clear out any -ssh keys found in the instance's disk. For Debian and Ubuntu systems, the keys -are also recreated. Besides removing files that comply to the -``/etc/ssh/ssh_*_key`` pattern, the script will also parses -``/etc/ssh/sshd_config`` file for custom keys. The only variable this script -depends on is *SNF_IMAGE_TARGET*. - -**DisableRemoteDesktopConnections**: This script temporary disables RDP -connections on Windows instances by changing the value of *fDenyTSConnection* -registry key. RDP connections will be enabled back during the specialize pass -of the Windows setup. The task will fail if *SNF_IMAGE_TARGET* is not defined. - **InstallUnattend**: Installs the Unattend.xml files on Windows instances. This is needed by Windows in order to perform an unattended setup. The *SNF_IMAGE_TARGET* variables needs to be present for this task to run. -**SELinuxAutorelabel**: Creates *.autorelabel* file in Red Hat images. This is -needed if SELinux is enabled to enforce an automatic file system relabeling -during the first boot. The only environment variable required by this task is -*SNF_IMAGE_TARGET*. +**FilesystemResizeMounted**: For Windows VMs this task injects a script into +the VM's file system that will enlarge the last file system to cover up the +whole partition. The script will run during the specialize pass of the Windows +setup. For Linux VMs this task is used to extend the last file system in case +its type is Btrfs or XFS, since those file systems require to be mounted in +order to resize them. If the *SNF_IMAGE_TARGET* variable is missing, the task +will fail. + +**AddSwap**: Formats the swap partition added by *FixPartitionTable* task and +adds an appropriate swap entry in the system's ``/etc/fstab``. The script will +only run if *SNF_IMAGE_PROPERTY_SWAP* is present and will fail if +*SNF_IMAGE_TARGET* in not defined. **AssignHostname**: Assigns or changes the hostname of the instance. The task will fail if the Linux distribution is not supported and ``/etc/hostname`` is @@ -182,6 +173,11 @@ SUSE and Gentoo derived distributions. The hostname is read from *SNF_IMAGE_HOSTNAME* variable. In addition to the latter, *SNF_IMAGE_TARGET* is also required. +**ChangeMachineId**: On Linux instances, this script will generate a new random +machine ID and will place it in ``/etc/machine-id``. For more info check +`here `_. The +task will fail if *SNF_IMAGE_TARGET* is missing. + **ChangePassword**: Changes the password for a list of existing users. On Linux systems this is accomplished by directly altering the instance's ``/etc/shadow`` file. On Windows systems a script is injected into the VM's @@ -202,16 +198,25 @@ variables are exported to the task. The only variable required by this task is adjust the *DHCP_TAGS* and the *\*_DHCPV6_TAGS* configuration parameters (see :doc:`/configuration`). -**FilesystemResizeMounted**: For Windows VMs this task injects a script into -the VM's file system that will enlarge the last file system to cover up the -whole partition. The script will run during the specialize pass of the Windows -setup. For Linux VMs this task is used to extend the last file system in case -its type is Btrfs or XFS, since those file systems require to be mounted in -order to resize them. If the *SNF_IMAGE_TARGET* variable is missing, the task -will fail. +**DeleteSSHKeys**: On Linux and \*BSD instances, this script will clear out any +ssh keys found in the instance's disk. For Debian and Ubuntu systems, the keys +are also recreated. Besides removing files that comply to the +``/etc/ssh/ssh_*_key`` pattern, the script will also parses +``/etc/ssh/sshd_config`` file for custom keys. The only variable this script +depends on is *SNF_IMAGE_TARGET*. + +**DisableRemoteDesktopConnections**: This script temporary disables RDP +connections on Windows instances by changing the value of *fDenyTSConnection* +registry key. RDP connections will be enabled back during the specialize pass +of the Windows setup. The task will fail if *SNF_IMAGE_TARGET* is not defined. + +**SELinuxAutorelabel**: Creates *.autorelabel* file in Red Hat images. This is +needed if SELinux is enabled to enforce an automatic file system relabeling +during the first boot. The only environment variable required by this task is +*SNF_IMAGE_TARGET*. **EnforcePersonality**: Injects the files specified by the -*SNF_IMAGE_PROPERTY_OSFAMILY* variable into the file system. If the variable is +*SNF_IMAGE_PERSONALITY* variable into the file system. If the variable is missing a warning is produced. Only *SNF_IMAGE_TARGET* is required for this task to run. @@ -222,48 +227,65 @@ a warning is produced. **UmountImage**: Umounts the file systems previously mounted by MountImage. The only environment variable required is *SNF_IMAGE_TARGET*. - -+-------------------------------+---+--------------------------------------------+----------------------------------------------+ -| | | Dependencies | Environment Variables [#]_ | -+ Name | +------------------+-------------------------+-------------------------+--------------------+ -| |Pr.| Run-After | Run-Before | Required | Optional | -+===============================+===+==================+=========================+=========================+====================+ -|FixPartitionTable |10 | |FilesystemResizeUnmounted|DEV | | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|FilesystemResizeUnmounted |20 |FixPartitionTable |MountImage |DEV | | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|MountImage |30 | |UmountImage |DEV | | -| | | | |TARGET | | -| | | | |PROPERTY_ROOT_PARTITION | | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|AddSwap |40 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | -| | | | | |PROPERTY_SWAP | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|DeleteSSHKeys |40 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|DisableRemoteDesktopConnections|40 |EnforcePersonality|UmountImage |TARGET |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|InstallUnattend |40 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|SELinuxAutorelabel |40 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|AssignHostname |50 |InstallUnattend |EnforcePersonality |TARGET | | -| | | | |HOSTNAME |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|ChangePassword |50 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_USERS | -| | | | | |PROPERTY_OSFAMILY | -| | | | | |PASSWD | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|ConfigureNetwork |50 |InstallUnattend |EnforcePersonality |TARGET |NIC_* | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|FilesystemResizeMounted |50 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|EnforcePersonality |60 |MountImage |UmountImage |TARGET |PERSONALITY | -| | | | | |PROPERTY_OSFAMILY | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|RunCustomTask |70 |MountImage |UmountImage |TARGET |PROPERTY_CUSTOM_TASK| -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ -|UmountImage |80 |MountImage | |TARGET | | -+-------------------------------+---+------------------+-------------------------+-------------------------+--------------------+ +**FilesystemResizeAfterUmount**: This is used for doing offline resize if the +file system in the last partition is NTFS. This is done after umount and not +before mounting the file system, because *ntfsresize* (which is used to perform +the actual resize) will mark the file system as dirty at the end and mounting +it afterwards is not recommended. This is done in order to force a chkdsk the +next time Windows boots. Offline NTFS resize is favored on windows-legacy and +non-windows OSes that do not support online resize. If you want to force +offline resize on newer Windows systems, the *OFFLINE_NTFSRESIZE* image +property must be defined. + ++-------------------------------+---+--------------------------------------------+-----------------------------------------------------+ +| | | Dependencies | Environment Variables [#]_ | ++ Name | +------------------+-------------------------+-------------------------+---------------------------+ +| |Pr.| Run-After | Run-Before | Required | Optional | ++===============================+===+==================+=========================+=========================+===========================+ +|FixPartitionTable |10 | |FilesystemResizeUnmounted|DEV | | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|FilesystemResizeUnmounted |20 |FixPartitionTable |MountImage |DEV |RESIZE_PART | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|MountImage |30 | |UmountImage |DEV | | +| | | | |TARGET | | +| | | | |PROPERTY_ROOT_PARTITION | | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|InstallUnattend |35 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|FilesystemResizeMounted |40 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | +| | | | | |RESIZE_PART | +| | | | | |PROPERTY_OFFLINE_NTFSRESIZE| ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|AddSwap |50 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | +| | | | | |PROPERTY_SWAP | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|AssignHostname |50 |InstallUnattend |EnforcePersonality |TARGET | | +| | | | |HOSTNAME |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|ChangeMachineId |50 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|ChangePassword |50 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_USERS | +| | | | | |PROPERTY_OSFAMILY | +| | | | | |PASSWD | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|ConfigureNetwork |50 |InstallUnattend |EnforcePersonality |TARGET |NIC_* | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|DeleteSSHKeys |50 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|DisableRemoteDesktopConnections|50 |EnforcePersonality|UmountImage |TARGET |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|SELinuxAutorelabel |50 |MountImage |EnforcePersonality |TARGET |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|EnforcePersonality |60 |MountImage |UmountImage |TARGET |PERSONALITY | +| | | | | |PROPERTY_OSFAMILY | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|RunCustomTask |70 |MountImage |UmountImage |TARGET |PROPERTY_CUSTOM_TASK | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|UmountImage |80 |MountImage | |TARGET | | ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ +|FilesystemResizeAfterUmount |81 |UmountImage | |DEV |RESIZE_PART | +| | | | | |PROPERTY_OSFAMILY | +| | | | | |PROPERTY_OFFLINE_NTFSRESIZE| ++-------------------------------+---+------------------+-------------------------+-------------------------+---------------------------+ .. [#] all environment variables are prefixed with *SNF_IMAGE_* diff --git a/docs/index.rst b/docs/index.rst index 8aa5487f6e513c3d0e8da5a965f7194f5a3777b7..4414464951f12c9120908f53c82c82dc274bb3eb 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -27,7 +27,7 @@ snf-image also supports Image customization via hooks. Hooks allow for: snf-image is being used in large scale production environments with Ganeti to successfully deploy many major Linux distributions (Debian, Ubuntu/Kubuntu, CentOS, Fedora, OpenSUSE, Slackware, Arch Linux, CoreOS), Windows Server -flavors (2008 R2, 2012, 20012 R2), as well as BSD systems (FreeBSD, OpenBSD, +flavors (2008 R2, 2012, 2012 R2), as well as BSD systems (FreeBSD, OpenBSD, NetBSD). It is also known to work well with current Desktop versions of Windows (7, 8, 8.1) as well as Windows XP. diff --git a/docs/installation.rst b/docs/installation.rst index 3100801ad9d63504ede14f1f68a7b13fff5be56b..7c90bc6314315b87f0584d1a884cb9cea95d4f07 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -13,13 +13,19 @@ Installing snf-image using packages Debian GNU/Linux ^^^^^^^^^^^^^^^^ -For Debian 7.x (Wheezy) we provide packages in our APT repository. To use -our repository add the following lines to file ``/etc/apt/sources.list``: +We provide Debian packages in our APT repository. To use our packages on +Debian 7.x (Wheezy) add the following lines to file ``/etc/apt/sources.list``: ``deb http://apt.dev.grnet.gr wheezy/`` ``deb-src http://apt.dev.grnet.gr wheezy/`` +For Debian 8.x (Jessie) add the following lines: + +``deb http://apt.dev.grnet.gr jessie/`` + +``deb-src http://apt.dev.grnet.gr jessie/`` + After you update ``/etc/apt/sources.list`` import the repo's GPG key: .. code-block:: console @@ -39,13 +45,19 @@ the post install phase of the package installation. Ubuntu ^^^^^^ -For Ubuntu 14.04 LTS we provide packages in our APT repository. To use our -repository add the following lines to file ``/etc/apt/sources.list``: +We provide Ubuntu packages in our APT repository. To use our packages on +Ubuntu 14.04 LTS add the following lines to file ``/etc/apt/sources.list``: ``deb http://apt.dev.grnet.gr trusty/`` ``deb-src http://apt.dev.grnet.gr trusty/`` +For Ubuntu 16.04 LTS add the following lines: + +``deb http://apt.dev.grnet.gr xenial/`` + +``deb-src http://apt.dev.grnet.gr xenial/`` + After you update ``/etc/apt/sources.list`` import the repo's GPG key: .. code-block:: console @@ -65,6 +77,9 @@ the post install phase of the package installation. CentOS ^^^^^^ +.. warning:: + CentOS packages are no longer actively supported. + For CentOS 6.5 we provide packages in our Yum repository. To add the GRNET repository in your system, run: diff --git a/docs/interface.rst b/docs/interface.rst index e8beabb2a6489037c1c93b3f8b73dd4f7a0f7c61..d99930e5bae113081966fc723141271a85fa5ab9 100644 --- a/docs/interface.rst +++ b/docs/interface.rst @@ -228,6 +228,12 @@ All image formats properties you want to write a custom configuration task check :ref:`Configuration Tasks Environment`. + * **NM_NETWORKING=bool** + If this property is defined with a yes value, the *ConfigureNetwork* task + will try to configure the Ganeti-provided NICs by creating Network Manager + configuration files, instead of using the distro-specific network + configuration mechanism (*ifupdown* for Debian, *ifcfg* for Red Hat, etc.). + * **EXCLUDE_ALL_TASKS=bool** If this property is defined with a yes value, the image will not be configured at all, during the deployment. This is really handy because it diff --git a/docs/usage.rst b/docs/usage.rst index af95cfb8fe487247b50e4d621f9a6d7852c4570f..7fa80f5662be4f053d25270611c74bca91ca091d 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -12,45 +12,53 @@ that have been tested with *snf-image* and provided here for testing purposes: * Debian Jessie Base System - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * Debian Jessie Desktop - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * CentOS 7.x - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * Fedora Desktop 23 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] + * Ubuntu Desktop LTS 16.04 + [`diskdump `__] + [`md5sum `__] + [`metadata `__] + * Ubuntu Server LTS 16.04 + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * Ubuntu Desktop LTS 14.04 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * Ubuntu Server LTS 14.04 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * OpenSUSE Desktop 13.2 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * Oracle Linux 7.x - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * FreeBSD 10.2 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] * NetBSD 7.0 - [`diskdump `_] - [`md5sum `_] - [`metadata `_] + [`diskdump `__] + [`md5sum `__] + [`metadata `__] Sample Usage ^^^^^^^^^^^^ diff --git a/docs/version.py b/docs/version.py index 4c1ca3c80986ea8ec44ff096e50770c7e4f06ab8..34bfab3404f4eccfa14a6d19aa129e055fa96507 100644 --- a/docs/version.py +++ b/docs/version.py @@ -1 +1 @@ -__version__ = "0.19.1" +__version__ = "0.20rc2.dev0" diff --git a/snf-image-helper/common.sh.in b/snf-image-helper/common.sh.in index 3918ccaf346f5bb96ce5e1cfca478afb352ae6f8..63ed0b17d49b6fd285eee9d1ca02010f793951ce 100644 --- a/snf-image-helper/common.sh.in +++ b/snf-image-helper/common.sh.in @@ -290,6 +290,26 @@ get_distro() { fi } +get_networking_tool() { + local root_dir distro tool + root_dir=$1 + + if check_yes_no SNF_IMAGE_PROPERTY_NM_NETWORKING; then + tool="nm" + else + distro=$(get_base_distro "$root_dir") + if [ "$distro" = debian ]; then + tool=ifupdown + elif [ "$distro" = redhat ]; then + tool=ifcfg + else + tool=$distro + fi + fi + + echo "@networkingdir@/$tool.sh" +} + get_partition_table() { local dev output dev="$1" diff --git a/snf-image-helper/networking/Makefile.am b/snf-image-helper/networking/Makefile.am index 0eb5973ea05b6be55e0d19657b7dd25198072674..db8b97081738cef6ebd69a02ee5384c2703f200a 100644 --- a/snf-image-helper/networking/Makefile.am +++ b/snf-image-helper/networking/Makefile.am @@ -1,6 +1,6 @@ networkingdir=$(libdir)/$(PACKAGE)/networking -dist_networking_SCRIPTS = debian.sh redhat.sh freebsd.sh openbsd.sh netbsd.sh +dist_networking_SCRIPTS = ifupdown.sh ifcfg.sh freebsd.sh openbsd.sh netbsd.sh nm.sh edit = sed \ -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ diff --git a/snf-image-helper/networking/redhat.sh.in b/snf-image-helper/networking/ifcfg.sh.in similarity index 100% rename from snf-image-helper/networking/redhat.sh.in rename to snf-image-helper/networking/ifcfg.sh.in diff --git a/snf-image-helper/networking/debian.sh.in b/snf-image-helper/networking/ifupdown.sh.in similarity index 100% rename from snf-image-helper/networking/debian.sh.in rename to snf-image-helper/networking/ifupdown.sh.in diff --git a/snf-image-helper/networking/nm.sh.in b/snf-image-helper/networking/nm.sh.in new file mode 100644 index 0000000000000000000000000000000000000000..3120950f0cd5666945e22c4d081ba3d0d5bab57d --- /dev/null +++ b/snf-image-helper/networking/nm.sh.in @@ -0,0 +1,86 @@ +#! /bin/bash + +# Copyright (C) 2016 GRNET S.A. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301, USA. + +# Script for configuring network manager connections + +set -e +. "@commondir@/common.sh" + +networking_opts "$@" + +SYSTEM_CONNECTIONS="$SNF_IMAGE_TARGET/etc/NetworkManager/system-connections/" + +if [ "$initialize" = yes ]; then + exit 0 +fi + +if [ "$finalize" = yes ]; then + exit 0 +fi + +if [ "$ipv4" = "none" ]; then + ipv4_section+="method=disabled" +elif [ "$ipv4" = "dhcp" ]; then + ipv4_section+="method=auto" +else + ipv4_section+="method=manual +address1=$IP/${SUBNET##*/}" + + if [ -n "$GATEWAY" ]; then + ipv4_section+=",$GATEWAY" + fi +fi + +if [ "$ipv6" != 'none' ]; then + ipv6_section="method=auto" +else + ipv6_section="method=disabled" +fi + +connection="$SYSTEM_CONNECTIONS/Wired connection $index" +cat > "$connection" < "$SNF_IMAGE_TARGET/etc/machine-id" +else + warn "\`/etc/machine-id' file is not present" +fi + +# The machine-id may be also cached by dbus +if [ -f "$SNF_IMAGE_TARGET/var/lib/dbus/machine-id" ]; then + echo "$uuid" > "$SNF_IMAGE_TARGET/var/lib/dbus/machine-id" +else + warn "\`/var/lib/dbus/machine-id' file is not present" +fi + +exit 0 + +# vim: set sta sts=4 shiftwidth=4 sw=4 et ai : diff --git a/snf-image-helper/tasks/50ConfigureNetwork.in b/snf-image-helper/tasks/50ConfigureNetwork.in index 8ed86ec96b68e61224f853ebbd7f3ac9a22cb2bd..2f9e14fde56cc4b7aeb88f1ee72c4d555ce86a6c 100644 --- a/snf-image-helper/tasks/50ConfigureNetwork.in +++ b/snf-image-helper/tasks/50ConfigureNetwork.in @@ -42,17 +42,20 @@ fi # Check if the image overwrites the task check_if_overwritten -distro=$(get_base_distro "$SNF_IMAGE_TARGET") +networking_tool=$(get_networking_tool "$SNF_IMAGE_TARGET") -if [ ! -f "@networkingdir@/$distro.sh" ]; then +if [ ! -f "$networking_tool" ]; then warn "Don't know how to configure the network for this OS" exit 0 +else + echo "Using $networking_tool" fi export SNF_IMAGE_TARGET # Initialize the driver -"@networkingdir@/$distro.sh" -i +echo "Running: $networking_tool -i" +"$networking_tool" -i for index in $(seq 0 $((SNF_IMAGE_NIC_COUNT-1))); do dhcp=no @@ -129,12 +132,14 @@ for index in $(seq 0 $((SNF_IMAGE_NIC_COUNT-1))); do fi fi - "@networkingdir@/$distro.sh" $ARGS + echo "Running: $networking_tool $ARGS" + "$networking_tool" $ARGS done # Finalize the driver -"@networkingdir@/$distro.sh" -f +echo "Running: $networking_tool -f" +"$networking_tool" -f # vim: set sta sts=4 shiftwidth=4 sw=4 et ai : diff --git a/snf-image-helper/tasks/Makefile.am b/snf-image-helper/tasks/Makefile.am index 252a690bbea9d7c00752c2392d3a3d529ffa96a2..cca7de7304fbc0abe8525cd70fbb09ab8c3a0273 100644 --- a/snf-image-helper/tasks/Makefile.am +++ b/snf-image-helper/tasks/Makefile.am @@ -8,6 +8,7 @@ dist_tasks_SCRIPTS = \ 40FilesystemResizeMounted \ 50AddSwap \ 50AssignHostname \ + 50ChangeMachineId \ 50ChangePassword \ 50ConfigureNetwork \ 50DeleteSSHKeys \ diff --git a/snf-image-host/common.sh.in b/snf-image-host/common.sh.in index 96dd58adeb16aeb18b9fa80a476890723d588c5a..4beeb936894152e4081a3d762a04224bdc73c93c 100644 --- a/snf-image-host/common.sh.in +++ b/snf-image-host/common.sh.in @@ -102,7 +102,7 @@ get_api10_arguments() { esac instance=$INSTANCE_NAME - if [ $DISK_COUNT -lt 1 -o -z "$DISK_0_PATH" ]; then + if [ $DISK_COUNT -lt 1 ]; then log_error "At least one disk is needed" exit 1 fi @@ -111,21 +111,22 @@ get_api10_arguments() { log_error "Missing OS API Variable: \`EXPORT_DEVICE'" exit 1 fi - blockdev=$EXPORT_DEVICE + export_disk=$EXPORT_DEVICE elif [ "$SCRIPT_NAME" = "import" ]; then if [ -z "$IMPORT_DEVICE" ]; then log_error "Missing OS API Variable: \`IMPORT_DEVICE'" exit 1 fi - blockdev=$IMPORT_DEVICE + import_disk=$IMPORT_DEVICE else - blockdev=$DISK_0_PATH + disk0=$(find_disk 0) fi if [ "$SCRIPT_NAME" = "rename" -a -z "$OLD_INSTANCE_NAME" ]; then log_error "Missing OS API Variable: \`OLD_INSTANCE_NAME'" exit 1 fi old_name=$OLD_INSTANCE_NAME + disk_type=$INSTANCE_HV_disk_type } get_api20_arguments() { @@ -213,6 +214,47 @@ format_disk0() { EOF } +# This is used for export/import and in case we have an fsdump image type. +# If the give disk is a block device this is a no-op. +losetup_disk() { + local disk bdev + + disk="$1" + + if [ -b "$disk" ]; then + echo "$disk" + elif [ -f "$disk" ] ; then + bdev=$($LOSETUP --show -f "$disk") + add_cleanup $LOSETUP -d "$bdev" + echo $bdev + else + log_error "Cannot losetup $disk: not a regular file" + report_error "Could not use disk provided by Ganeti" + exit 1 + fi +} + +find_disk() { + local idx path uri + + idx=$1 + + eval path=\$DISK_${idx}_PATH + eval uri=\$DISK_${idx}_URI + + # Here we check if we have a valid disk path (block device or regular file) + if [ -b "$path" -o -f "$path" ]; then + echo $path + # Otherwise we choose the userspace URI and + # hope that snf-image can handle it + elif [ -n "$uri" ]; then + echo $uri + else + log_error "Disk information not exported by Ganeti" + exit 1 + fi +} + create_floppy() { local img target IFS index name tm diff --git a/snf-image-host/create b/snf-image-host/create index b325255713dd6267f5015817becfe1fb5201fa2f..ebe59b53d59831bca7e3f0ef9f117b81e2e401e3 100755 --- a/snf-image-host/create +++ b/snf-image-host/create @@ -133,17 +133,12 @@ case $BACKEND_TYPE in ;; esac -# If the target device is not a real block device we'll first losetup it. -# This is needed for file disks. -if [ ! -b "$blockdev" ]; then - original_blockdev="$blockdev" - blockdev=$($LOSETUP --show -f "$blockdev") - add_cleanup $LOSETUP -d "$blockdev" -fi - case "$IMAGE_TYPE" in ntfsdump|extdump) - # Create partitions + # Create a corresponding blockdev if required + blockdev=$(losetup_disk "$disk0") + + # Create partitions on blockdev format_disk0 "$blockdev" "$IMAGE_TYPE" # Install a new MBR @@ -159,7 +154,7 @@ case "$IMAGE_TYPE" in fi ;; diskdump) - target="$blockdev" + target="$disk0" ;; *) log_error "Unknown Image format: \`$IMAGE_TYPE'" diff --git a/snf-image-host/export b/snf-image-host/export index 39aff1e3449211adf4c95cfb38a864fdcd388d8e..738d15aa9fa93aa08d55bfc6c94bcff90d321fc4 100755 --- a/snf-image-host/export +++ b/snf-image-host/export @@ -26,11 +26,7 @@ ganeti_os_main # If the device we will export from, is not a real block device, # we'll first losetup it. This is needed for file disks. -if [ ! -b "$blockdev" ]; then - original_blockdev="$blockdev" - blockdev=$($LOSETUP --show -f "$blockdev") - add_cleanup $LOSETUP -d "$blockdev" -fi +blockdev=$(losetup_disk "$export_disk") # Export disk's predicted size to Ganeti. # Used to provide a time estimate of the export process to the user. diff --git a/snf-image-host/import b/snf-image-host/import index 57e7f44482384e8b25d12575e1ae165492dca55b..bc19c5517c7c6795f220bad1963ef7ffa8f3d13b 100755 --- a/snf-image-host/import +++ b/snf-image-host/import @@ -26,11 +26,7 @@ ganeti_os_main # If the device we will import to, is not a real block device, # we'll first losetup it. This is needed for file disks. -if [ ! -b "$blockdev" ]; then - original_blockdev="$blockdev" - blockdev=$($LOSETUP --show -f "$blockdev") - add_cleanup $LOSETUP -d "$blockdev" -fi +blockdev=$(losetup_disk "$import_disk") $DD of="$blockdev" bs=4M diff --git a/snf-image-host/kvm-common.sh b/snf-image-host/kvm-common.sh index 5db28ad133c149840c9f571e52fa9e5ebb763ca3..4a833a14bb5eb66b575413f50c17934435bb5302 100644 --- a/snf-image-host/kvm-common.sh +++ b/snf-image-host/kvm-common.sh @@ -1,4 +1,4 @@ -# Copyright (C) 2013-2015 GRNET S.A. and individual contributors +# Copyright (C) 2013-2016 GRNET S.A. and individual contributors # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -15,21 +15,45 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA # 02110-1301, USA. +get_img_dev() { + local idx=$1 + local letters=( a b c d e f g h e j k l m n ) + + case $disk_type in + # The helper's root disk is always paravirtual and thus /dev/vda + # will be occupied. + paravirtual) echo /dev/vd${letters[$idx+1]};; + scsi-generic|scsi-block|scsi-hd|scsi) echo /dev/sd${letters[$idx]};; + esac +} + +get_img_driver() { + case $disk_type in + paravirtual) echo virtio-blk-pci;; + scsi|scsi-hd) echo scsi-hd;; + scsi-block) echo scsi-block;; + scsi-generic) echo scsi-generic;; + esac +} + assign_disk_devices_to() { - local varname + local varname dev i + varname="$1" + # This will declare the given variable as an array eval $varname=\(\) - set -- b c d e f g h e j k l m n - + # For DISK_COUNT=3, disk_type=paravirtual, and varname=snf_export_DEV + # this will create snf_export_DEV array of ( /dev/vdb /dev/vdc /dev/vdd ) for ((i = 0; i < DISK_COUNT; i++)); do - eval $varname+=\(\"/dev/vd$1\"\); shift + dev=$(get_img_dev $i) + eval $varname+=\(\"$dev\"\) done } launch_helper() { - local result_file result snapshot rc floppy i disk_path disks + local result_file result snapshot rc floppy i disks floppy="$1" @@ -41,8 +65,8 @@ launch_helper() { disks="" for ((i=0; i < DISK_COUNT; i++)); do - eval disk_path=\"\$DISK_${i}_PATH\" - disks+=" -drive file=$disk_path,format=raw,if=virtio,cache=none" + disks+=" -drive file=$(find_disk $i),format=raw,if=none,cache=none,id=drive$i" + disks+=" -device $(get_img_driver),id=disk$i,drive=drive$i" done set +e @@ -53,11 +77,16 @@ launch_helper() { HELPER_DEBUG_ARG="" fi + if [[ "$disk_type" =~ ^scsi ]]; then + EXTRA_CONTROLLER_ARG="-device virtio-scsi-pci" + fi $TIMEOUT -k "$HELPER_HARD_TIMEOUT" "$HELPER_SOFT_TIMEOUT" \ $KVM -runas "$HELPER_USER" \ - -drive file="$HELPER_DIR/image",format=raw,if=virtio,readonly \ - $disks -m "$HELPER_MEMORY" -boot c -serial stdio \ + -drive file="$HELPER_DIR/image",format=raw,if=none,id=helper,readonly \ + -device virtio-blk-pci,id=helper,drive=helper \ + $EXTRA_CONTROLLER_ARG $disks \ + -m "$HELPER_MEMORY" -boot c -serial stdio \ -serial "file:$(printf "%q" "$result_file")" \ -serial file:>(./helper-monitor.py ${MONITOR_FD}) \ -serial pty \ diff --git a/snf-image-host/snf-image-update-helper.in b/snf-image-host/snf-image-update-helper.in index 5c52d000c12a345c592e65d0380b48bbb2f44107..bcd65be3e61a11a1101d535e52605240fb25f528 100644 --- a/snf-image-host/snf-image-update-helper.in +++ b/snf-image-host/snf-image-update-helper.in @@ -137,6 +137,16 @@ if [ "x$NO_CHECKSUM" != "xyes" ]; then exit 1 fi + # The filename listed inside the md5sum file may be different than the one + # of the image we have downloaded. This is due to redirections and the fact + # that we are not using --remote-header-name which would use the + # server-specified Content-Disposition filename instead of extracting a + # filename from the URL. This option is not used because some versions of + # curl will not attempt to decode %-sequences which may lead to unexpected + # filenames. It's better if we just replace the filename inside the md5sum + # file with the one we know. + sed -i 's/ .\+$/'" $IMAGE"'/' "$IMAGE_DIR/$IMAGE.md5sum" + $MD5SUM -c "$IMAGE_DIR/$IMAGE.md5sum" fi diff --git a/snf-image-host/xen-common.sh b/snf-image-host/xen-common.sh index 136afb37f3eb5c5a6876e83b8d187c38c10b9402..07340975eae986017746adc85c1521a54191e803 100644 --- a/snf-image-host/xen-common.sh +++ b/snf-image-host/xen-common.sh @@ -40,7 +40,7 @@ launch_helper() { set -- c d e f g h i j k l m n o p q r for ((i = 0; i < DISK_COUNT; i++)); do - eval disk_path=\"\$DISK_${i}_PATH\" + disk_path="$(find_disk $i)" case $(stat -L -c %F "$disk_path") in "regular file") ftype=file diff --git a/version b/version index 41915c799477856bc769e838128f06905e641650..7d7a1d1c243d62797daedde923d7191c6ebbb38f 100644 --- a/version +++ b/version @@ -1 +1 @@ -0.19.1 +0.20rc2 diff --git a/version.m4 b/version.m4 index 02cb360ddfa6d89749323e64c3f6e4bd3e807443..279bc7ca148abccf3617fcea8a277cb297d6fa44 100644 --- a/version.m4 +++ b/version.m4 @@ -1 +1 @@ -m4_define([devflow_version], [0.19.1]) +m4_define([devflow_version], [0.20rc2.dev0])