Commit 88c5e4ba authored by Nikos Skalkotos's avatar Nikos Skalkotos
Browse files

Merge branch 'release-0.18' into debian-release-0.18

parents d69d50d5 388c3654
2015-02-12, v0.18rc1
* HELPER: Auto-detect OSFAMILY and ROOT_PARTITION properties if missing
* HELPER: Add new CofigureNetwork image configuration task
* Add new DCHP_TAGS, STATEFUL_DHCPV6_TAGS and STATELESS_DHCPV6_TAGS
configuration parameters
* Update the documentation
* Fix small bugs
2015-01-15, v0.17
* Add support for btrfs
* Add support for xfs
......
......@@ -194,6 +194,14 @@ users whose passwords will changed is determined by the
task to run *SNF_IMAGE_TARGET* and *SNF_IMAGE_PASSWD* variables need to be
present.
**ConfigureNetwork**: Edit the OS's native network configuration files to
configure the instance's NICs. This works for most Linux and all the supported
\*BSD systems. In order to do this, all the NIC_* Ganeti provided environment
variables are exported to the task. The only variable required by this task is
*SNF_IMAGE_TARGET*. For this task to work correctly, the user may need to
adjust the *DHCP_TAGS* and the *\*_DHCPV6_TAGS* configuration parameters (see
:doc:`/configuration`).
**FilesystemResizeMounted**: For Windows VMs this task injects a script into
the VM's file system that will enlarge the last file system to cover up the
whole partition. The script will run during the specialize pass of the Windows
......@@ -242,6 +250,8 @@ only environment variable required is *SNF_IMAGE_TARGET*.
| | | | | |PROPERTY_OSFAMILY|
| | | | | |PASSWD |
+-------------------------------+---+------------------+-------------------------+-------------------------+-----------------+
|ConfigureNetwork |50 |InstallUnattend |EnforcePersonality |TARGET |NIC_* |
+-------------------------------+---+------------------+-------------------------+-------------------------+-----------------+
|FilesystemResizeMounted |50 |InstallUnattend |EnforcePersonality |TARGET |PROPERTY_OSFAMILY|
+-------------------------------+---+------------------+-------------------------+-------------------------+-----------------+
|EnforcePersonality |60 |MountImage |UmountImage |TARGET |PERSONALITY |
......
......@@ -86,6 +86,23 @@ some external programs in ``/etc/default/snf-image``:
# input of this program.
# PROGRESS_MONITOR=""
# DHCP_TAGS: Space separated list of Ganeti network tags. snf-image will
# configure a VM's NIC to use DHCP if the card is expected to have an IPv4
# address and any of those tags is present in the card's NETWORK_TAGS variable.
# DHCP_TAGS="auto dhcp nfdhcpd"
# STATEFUL_DHCPV6_TAGS: Space separated list of Ganeti network tags. snf-image
# will configure a VM's NIC to use DHCPv6 if the card is expected to have an
# IPv6 address and any of those tags is present in the card's NETWORK_TAGS
# variable.
# STATEFUL_DHCPV6_TAGS="dhcpv6 stateful_dhcpv6"
# STATELESS_DHCPV6_TAGS: Space separated list of Ganeti network tags. snf-image
# will configure a VM's NIC to perform SLAAC and Stateless DHCPv6 if the card
# is expected to have an IPv6 address and any of those tags is present in the
# card's NETWORK_TAGS variable.
# STATELESS_DHCPV6_TAGS="nfdhcpd stateless_dhcpv6
# UNATTEND: This variable overwrites the unattend.xml file used when deploying
# a Windows image. snf-image-helper will use its own unattend.xml file if this
# variable is empty. Please leave this empty, unless you really know what you
......@@ -117,6 +134,10 @@ The most common configuration parameters the user may need to overwrite are:
hosted, in case the user is accessing Pithos-hosted images
* **PROGRESS_MONITOR**: To specify an executable that will handle the
monitoring messages exported by *snf-image*
* **DHCP_TAGS**: To specify which Ganeti networks support DHCP
* **STATELESS_DHCPV6_TAGS**: To specify which Ganeti networks support SLAAC
and stateless DHCPv6
* **STATEFUL_DHCPV6_TAGS**: To specify which Ganeti networks support DHCPv6
Paths of external programs
^^^^^^^^^^^^^^^^^^^^^^^^^^
......
......@@ -100,19 +100,18 @@ Image Properties (img_properties)
*snf-image* may use a number of properties to properly configure the image.
Those image properties are passed to *snf-image* by Ganeti through the
**img_poroperties** OS parameter (see Ganeti OS Interface). The name of all
image properties is case-insensitive. For the *diskdump* format some properties
are mandatory. For *{ext,ntfs}dump* formats all image properties are optional.
image properties is case-insensitive. All image properties are optional.
We can group image properties in two categories:
1. Generic properties (*OSFAMILY*, *ROOT_PARTITION*, *USERS*)
1. Generic properties (*OSFAMILY*, *ROOT_PARTITION*, *USERS*, etc.)
2. Configuration tasks to run (*EXCLUDE_ALL_TASKS*, *EXCLUDE_TASK_<task_name>*)
(see here for :ref:`valid configuration tasks <image-configuration-tasks>`)
A list of all properties follows:
Mandatory properties (for diskdump only)
++++++++++++++++++++++++++++++++++++++++
Diskdump only properties
++++++++++++++++++++++++
* **OSFAMILY=linux|windows|freebsd|netbsd|openbsd**
This specifies whether the image is a Linux, a Windows or a \*BSD Image.
......@@ -122,13 +121,12 @@ Mandatory properties (for diskdump only)
earlier, for now, only primary partitions are supported. This property is
trivial for *{ext,ntfs}dump* formats (they only host one partition).
.. note:: On a diskdump image if no image property is defined, the deployment
will not fail. This situation is treated as a special case. All
configuration tasks will be prevented from running and the helper VM will
report SUCCESS after creating a warning about this.
.. note:: Those properties are necessary for the image deployment to work. If
any of those properties is missing, *snf-image* will try to auto-detect it's
value. The deployment will fail if the auto-detection fails.
Optional properties
+++++++++++++++++++
All image formats properties
++++++++++++++++++++++++++++
* **USERS="username1 username2...."**
This is a space-separated list of users, whose password will be reset by
......
__version__ = "0.17"
__version__ = "0.18rc1"
commondir=$(datarootdir)/$(PACKAGE)
networkingdir=$(libdir)/$(PACKAGE)/networking
scriptsdir=$(libdir)/$(PACKAGE)
tasksdir=$(scriptsdir)/tasks
export commondir scriptsdir tasksdir
export commondir networkingdir scriptsdir tasksdir
SUBDIRS = tasks
SUBDIRS = tasks networking
dist_doc_DATA = COPYING AUTHORS
dist_bin_SCRIPTS = snf-image-helper
......@@ -12,6 +13,7 @@ dist_common_DATA = common.sh unattend.xml
edit = sed \
-e 's|@commondir[@]|$(commondir)|g' \
-e 's|@networkingdir[@]|$(networkingdir)|g' \
-e 's|@scriptsdir[@]|$(scriptsdir)|g' \
-e 's|@tasksdir[@]|$(tasksdir)|g'
......
# Copyright (C) 2011, 2012, 2013 GRNET S.A.
# Copyright (C) 2011-2015 GRNET S.A.
# Copyright (C) 2007, 2008, 2009 Google Inc.
#
# This program is free software; you can redistribute it and/or modify
......@@ -506,6 +506,20 @@ bsd2linux() {
log_error "Couldn't find out mapping for BSD partition: \`$partition' in \`$device'"
}
cidr2mask() {
local zeroes=$((32-$1))
local shifts
local IFS=.
shift
for i in 1 2 3 4; do
((shifts = zeroes > 8 ? 8 : zeroes))
zeroes=$((zeroes - shifts))
set $((255 >> shifts << shifts)) "$@"
done
echo "$*"
}
find_mount_target() {
local device
device="$1"
......@@ -618,10 +632,18 @@ umount_all() {
}
get_ufstype() {
local device ufs
local device ufs pipefail
device="$1"
pipefail=$(set -o | grep pipefail | cut -f2)
if [ "$pipefail" = on ]; then
set +o pipefail
fi
ufs="$($DUMPFS_UFS "$device" | head -1 | awk -F "[()]" '{ for (i=2; i<NF; i+=2) print $i }')"
if [ "$pipefail" = on ]; then
set -o pipefail
fi
case "$ufs" in
UFS1)
......@@ -637,6 +659,142 @@ get_ufstype() {
esac
}
check_if_root() {
local os fd device is_root args exists windows_fs linux_fs freebsd_fs \
netbsd_fs openbsd_fs supported_fs
osfamily="$1"
fs="$2"
device="$3"
windows_fs="(ntfs|ntfs-3g|vfat|msdos|fat|fuse|fuseblk)"
linux_fs="(ext[234]|xfs|jfs|zfs|btrfs|reiserfs|reiser4)"
freebsd_fs="(ufs|zfs)"
netbsd_fs="(ufs|zfs)"
openbsd_fs="ufs"
supported_fs="${osfamily}_fs"
if [ "$fs" = "" ]; then
warn "Can't detect a file system on device: \`$device'"
return 0
fi
if [[ ! "$fs" =~ ${!supported_fs} ]]; then
warn "Ignoring device: \`$device' with file system: \`$fs' when checking for a $osfamily root"
return 0
fi
if [ "$fs" = ufs ]; then
args="-t ufs -o ufstype=$(get_ufstype "$device"),ro"
else
args="-o ro"
fi
is_root=no
$MOUNT "$device" "$SNF_IMAGE_TARGET" $args || {
warn "Unable to mount: \`$device' with file system: \`$fs'"
set -e
return 0
}
case "$osfamily" in
freebsd|linux)
if [ -f "$SNF_IMAGE_TARGET/etc/fstab" ]; then
is_root=yes
fi
;;
windows)
exists=$(find "$SNF_IMAGE_TARGET" -maxdepth 1 -iname windows)
if [ -n "$exists" -a -d "$exists" ]; then
is_root=yes
fi
;;
esac
umount "$SNF_IMAGE_TARGET"
if [ "$is_root" = yes ]; then
export DETECTED_OSFAMILY="$osfamily"
fi
}
examine_msdos_partition() {
local dev part
dev="$1"
part="$2"
num=$(cut -f1 -d: <<< "$part")
id=$($SFDISK --force --print-id "$dev" "$num") || true
fs=$($BLKID -s TYPE -o value "$dev$num") || true
case "$id" in
a5)
export DETECTED_OSFAMILY=freebsd
;;
a6)
export DETECTED_OSFAMILY=openbsd
;;
a9)
export DETECTED_OSFAMILY=netbsd
;;
7) # looks like Windows
check_if_root windows "$fs" "$dev$num"
;;
83) # looks like Linux
check_if_root linux "$fs" "$dev$num"
;;
esac
if [ -n "$DETECTED_OSFAMILY" ]; then
export DETECTED_ROOT_PARTITION="$num"
fi
}
examine_gpt_partition() {
local dev part ufs_type
dev="$1"
part="$2"
# It's OK if some of those fail
num=$(cut -f1 -d: <<< "$part")
fs=$($BLKID -s TYPE -o value "$dev$num") || true
id=$($SGDISK -i "$num" "$dev" | \
grep "^Partition GUID code:" | \
cut -f2 -d: | \
awk '{print $1}') || true
case "$id" in
516E7CB6-6ECF-11D6-8FF8-00022D09712B) # FreeBSD
check_if_root freebsd "$fs" "$dev$num"
;;
EBD0A0A2-B9E5-4433-87C0-68B6B72699C7) # Windows
check_if_root windows "$fs" "$dev$num"
;;
*) # Linux does not respect the Partition GUID code at all
check_if_root linux "$fs" "$dev$num"
;;
esac
if [ -n "$DETECTED_OSFAMILY" ]; then
export DETECTED_ROOT_PARTITION="$num"
fi
}
detect_image_properties() {
local dev table
dev="$1"
table=$(get_partition_table "$dev")
pttype=$(get_partition_table_type "$table")
partitions=$(tail -n +3 <<< "$table")
for part in $partitions; do
"examine_${pttype}_partition" "$dev" "$part"
if [ -n "$DETECTED_OSFAMILY" -a -n "$DETECTED_ROOT_PARTITION" ]; then
break
fi
done
}
cleanup() {
# if something fails here, it shouldn't call cleanup again...
trap - EXIT
......@@ -697,6 +855,58 @@ return_success() {
send_result_${HYPERVISOR} "SUCCESS"
}
networking_opts() {
local usage="$0 [-i | -f | -n <index>] [-4 (dhcp|static)] [-6 (dhcp|slaac|slaac_dhcp)]"
ipv4=none
ipv6=none
index=
initialize=no
finalize=no
while getopts "h?ifn:4:6:" opt; do
case "$opt" in
h|\?)
echo $usage >&2
exit 0
;;
i) initialize=yes
;;
f) finalize=yes
;;
n) index=$OPTARG
;;
4) ipv4=$OPTARG
;;
6) ipv6=$OPTARG
;;
esac
done
if [ -z "$index" -a "$initialize" = no -a "$finalize" = no ]; then
log_error "Either -i, -f or -n must be specified"
fi
if [ "$initialize" = yes -a "$finalize" = yes ]; then
log_error "-i and -f should not be both specified"
fi
if [[ "$index" && ("$initialize" == yes || "$finalize" == yes) ]]; then
log_error "-n cannot be combined with -i or -f"
fi
if [[ ! "$ipv4" =~ (dhcp|static|none) ]]; then
log_error "invalid ipv4: $ipv4"
fi
if [[ ! "$ipv6" =~ (dhcp|slaac|slaac_dhcp|none) ]]; then
log_error "invalid ipv6: $ipv6"
fi
if [[ (! "$index" =~ ^[0-9]+$) && "$prepare" = no ]]; then
log_error "invalid index: $index"
fi
}
trap cleanup EXIT
set -o pipefail
......
......@@ -61,6 +61,7 @@ fi
AC_CONFIG_FILES([
Makefile
tasks/Makefile
networking/Makefile
])
AC_OUTPUT
......
networkingdir=$(libdir)/$(PACKAGE)/networking
dist_networking_SCRIPTS = debian.sh redhat.sh freebsd.sh openbsd.sh netbsd.sh
edit = sed \
-e 's|@sysconfdir[@]|$(sysconfdir)|g' \
-e 's|@localstatedir[@]|$(localstatedir)|g' \
-e 's|@datarootdir[@]|$(datarootdir)|g' \
-e 's|@commondir[@]|$(commondir)|g' \
-e 's|@networkingdir[@]|$(networkingdir)|g' \
-e 's|@scriptsdir[@]|$(scriptsdir)|g'
%:%.in Makefile
rm -f $@ $@.tmp
srcdir=''; \
test -f ./$@.in || srcdir=$(srcdir)/; \
$(edit) $${srcdir}$@.in >$@.tmp
mv $@.tmp $@
CLEANFILES = $(dist_networking_SCRIPTS)
#! /bin/bash
# Copyright (C) 2015 GRNET S.A.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
set -e
. "@commondir@/common.sh"
INTERFACES="$SNF_IMAGE_TARGET/etc/network/interfaces"
networking_opts "$@"
# initialize
if [ "$initialize" = yes ]; then
if [ -f "$INTERFACES" ]; then
mv "$INTERFACES" "${INTERFACES}.snf-image-$RANDOM"
fi
cat > "$INTERFACES" <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# Entries were automatically added by snf-image's ConfigureNetwork task.
# The loopback network interface
auto lo
iface lo inet loopback
EOF
exit 0
fi
# finalize
if [ "$finalize" = yes ]; then
exit 0
fi
nic="auto eth${index}\n"
nic+="allow-hotplug eth${index}\n"
if [ "$ipv4" = "dhcp" ]; then
nic+="iface eth${index} inet dhcp\n"
elif [ "$ipv4" = "static" ]; then
netmask="$(cidr2mask "${SUBNET##*/}")"
nic+="iface eth${index} inet static\n"
nic+=" address $IP\n"
nic+=" netmask $netmask\n"
if [ -n "$GATEWAY" ]; then
nic+=" gateway $GATEWAY\n"
fi
fi
if [ "$ipv6" = "dhcp" ]; then
nic+="iface eth${index} inet6 dhcp\n"
elif [ "$ipv6" = "slaac" ]; then
nic+="iface eth${index} inet6 auto\n"
nic+=" privext 0\n"
elif [ "$ipv6" = "slaac_dhcp" ]; then
nic+="iface eth${index} inet6 auto\n"
nic+=" privext 0\n"
nic+=" dhcp 1\n"
fi
echo -e "$nic" >> "$INTERFACES"
# vim: set sta sts=4 shiftwidth=4 sw=4 et ai :
#! /bin/bash
# Copyright (C) 2015 GRNET S.A.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
set -e
. "@commondir@/common.sh"
CONF="$SNF_IMAGE_TARGET/etc/rc.conf"
networking_opts "$@"
if [ "$initialize" = yes ]; then
cat "$CONF" > "$CONF.snf_image-$(date +%Y%m%d%H%M%S)-$RANDOM"
exit 0
fi
if [ "$finalize" = yes ]; then
exit 0
fi
case "$TYPE" in
paravirtual)
driver=vtnet
driver_index=$paravirtual_index
;;
rtl8139)
driver=rl
driver_index=$rtl8139_index
;;
ne2k_isa|ne2k_pci)
driver=ed
driver_index=$((ne2k_isa_index + ne2k_pci_index))
;;
i82551|i82557b|i82559er)
driver=fxp
driver_index=$((i82551_index + i82557b_index + i82559er_index))
;;
pcnet)
driver=pcn
driver_index=$pcnet_index
;;
e1000)
driver=em
dirver_index=$e1000_index
;;
*)
log_error "Unknown card type: \`$TYPE'"
esac
index=$((driver_index - 1))
tmp_config=$(mktemp)
add_cleanup rm -f "$tmp_config"
grep -v "ifconfig_$driver$index" "$CONF" > "$tmp_config"
cat "$tmp_config" > "$CONF"
if [ "$ipv4" = 'static' ]; then
echo "ifconfig_$driver$index=\"inet $IP netmask $(cidr2mask "${SUBNET##*/}")\"" >> "$CONF"
if [ -n "$GATEWAY" ]; then
sed -i '/^defaultrouter=/d' "$CONF"
echo "defaultrouter=\"$GATEWAY\"" >> "$CONF"
fi
elif [ "$ipv4" = 'dhcp' ]; then
echo "ifconfig_$driver$index=\"DHCP\"" >> "$CONF"
fi
if [ "$ipv6" != 'none' ]; then
sed -i '/^ipv6_enable=/d' "$CONF"
echo 'ipv6_enable="YES"' >> "$CONF"
fi
if [ "$ipv6" = 'dhcp' ]; then
echo "ifconfig_$driver${index}_ipv6=\"inet6 -ifdisabled accept_rtadv\"" >> $CONF
echo -e "dhcp6c_enable=\"YES\"\ndhcp6c_interfaces=\"$driver$index\"" >> $CONF
elif [ "$ipv6" = 'slaac' -o "$ipv6" = 'slaac_dhcp' ]; then
echo "ifconfig_$driver${index}_ipv6=\"inet6 accept_rtadv\"" >> $CONF
sed -i '/^rtsold_enable=/d' "$CONF"
echo 'rtsold_enable="YES"' >> "$CONF"
fi
# vim: set sta sts=4 shiftwidth=4 sw=4 et ai :
#! /bin/bash
# Copyright (C) 2015 GRNET S.A.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
set -e
. "@commondir@/common.sh"
networking_opts "$@"
if [ "$initialize" = yes ]; then
exit 0
fi
if [ "$finalize" = yes ]; then
exit 0
fi
case "$TYPE" in
paravirtual)