Commit 96804b04 authored by Nikos Skalkotos's avatar Nikos Skalkotos
Browse files

Merge branch 'release-0.15' into debian-release-0.15

parents d18bb802 84f9d7ff
2014-13-05, v0.15rc1
* Add support for selectable pithos storage backend. Possible storage
backends for pithos are `nfs' or `rados'
* Fix bugs and update the documentation
2014-04-11, v0.14
* Add support for extending NetBSD partitions
* Change the default Virtual RAM size of the helper VM to 512M
......
[ packages ]
[[ devflow ]]
version_file = "version.m4"
version_template = "version_template"
version_file = "version.m4","docs/version.py"
version_template = "version_template","docs/version_template"
......@@ -106,8 +106,12 @@ newly created block device. The following back-ends are supported:
* **Pithos backend**:
*snf-image* contains a special command-line tool (*pithcat*) for retrieving
images residing on a Pithos installation. To set up snf-image's Pithos
backend the user needs to setup the ``PITHOS_DATA`` and ``PITHOS_DB``
variables inside ``/etc/default/snf-image`` accordingly.
backend the user needs to setup the ``PITHOS_BACKEND_STORAGE`` variable
inside ``/etc/default/snf-image``.
Possible values are ``nfs`` and ``rados``. If ``nfs`` is used the user needs
to setup ``PITHOS_DATA`` variable, and when ``rados`` is used the user needs
to setup ``PITHOS_RADOS_POOL_MAPS`` and ``PITHOS_RADOS_POOL_BLOCKS``
accordingly.
* **Null backend**:
If the null backend is selected, no image copying is performed. This
......
......@@ -12,7 +12,8 @@
# serve to show the default.
import sys, os
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
from version import __version__
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
......@@ -45,9 +46,9 @@ copyright = u'2011, 2012, 2013 GRNET S.A. All rights reserved'
# built documents.
#
# The short X.Y version.
version = '0.14'
version = __version__
# The full version, including alpha/beta/rc tags.
release = '0.14'
release = __version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......
......@@ -64,9 +64,25 @@ some external programs in ``/etc/default/snf-image``:
# PITHOS_DB: Pithos database in SQLAlchemy format
# PITHOS_DB="sqlite://///var/lib/pithos/backend.db"
# PITHOS_BACKEND_STORAGE: Select Pithos backend storage. Possible values are
# 'nfs' and 'rados'. According to the value you select, you need to set the
# corresponding variables that follow.
# If you select 'nfs' that's 'PITHOS_DATA'. If you select 'rados' then you
# need to set all the "*_RADOS_*" ones.
# PITHOS_BACKEND_STORAGE="nfs"
# PITHOS_DATA: Directory where Pithos data are hosted
# PITHOS_DATA="//var/lib/pithos/data"
# PITHOS_RADOS_CEPH_CONF: RADOS configuration file
# PITHOS_RADOS_CEPH_CONF="@sysconfdir@/ceph/ceph.conf"
# PITHOS_RADOS_POOL_MAPS: RADOS pool for storing Pithos maps
# PITHOS_RADOS_POOL_MAPS="maps"
# PITHOS_RADOS_POOL_BLOCKS: RADOS pool for storing Pithos blocks
# PITHOS_RADOS_POOL_BLOCKS="blocks"
# PROGRESS_MONITOR: External program that monitors the progress of image
# deployment. Monitoring messages will be redirected to the standard input of
# this program.
......
......@@ -8,17 +8,39 @@ identical for all nodes and should be repeated manually or automatically, e.g.,
with puppet.
Installing snf-image using packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-----------------------------------
For Debian GNU/Linux and Ubuntu we provide packages in our APT repository.
Debian GNU/Linux
^^^^^^^^^^^^^^^^
For Debian Wheezy add the following lines to ``/etc/apt/sources.list`` file:
For Debian 7.x (Wheezy) we provide packages in our APT repository. To use
our repository add the following lines to file ``/etc/apt/sources.list``:
``deb http://apt.dev.grnet.gr wheezy/``
``deb-src http://apt.dev.grnet.gr wheezy/``
For Ubuntu 12.04 LTS add the following lines:
After you update ``/etc/apt/sources.list`` import the repo's GPG key:
.. code-block:: console
# curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
To install the package use the following commands:
.. code-block:: console
# apt-get update
# apt-get install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
Ubuntu
^^^^^^
For Ubuntu 12.04 LTS we provide packages in our APT repository. To use our
repository add the following lines to file ``/etc/apt/sources.list``:
``deb http://apt.dev.grnet.gr precise/``
......@@ -28,20 +50,43 @@ After you update ``/etc/apt/sources.list`` import the repo's GPG key:
.. code-block:: console
$ curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
# curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
To install the package use the following commands:
.. code-block:: console
$ apt-get update
$ apt-get install snf-image
# apt-get update
# apt-get install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
CentOS
^^^^^^
For CentOS 6.5 we provide packages in our Yum repository.
To add the GRNET repository in your system, run:
.. code-block:: console
# yum localinstall https://dev.grnet.gr/files/grnet-repo.rpm
You can verify the authenticity of the package using our public key found
`here <https://dev.grnet.gr/files/apt-grnetdev.pub>`_.
To install snf-image run:
.. code-block:: console
# yum install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
Installing snf-image from source
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
--------------------------------
To install snf-image from source, download the provided source package:
......@@ -63,13 +108,13 @@ Install snf-image:
.. code-block:: console
$ make install
$ install -Dm600 defaults /etc/default/snf-image
$ mkdir -p /var/lib/snf-image/helper
# make install
# install -Dm600 defaults /etc/default/snf-image
# mkdir -p /var/lib/snf-image/helper
Finally, install the helper image by executing:
.. code-block:: console
$ snf-image-update-helper
# snf-image-update-helper
......@@ -66,10 +66,12 @@ to be used. If no prefix is used, it defaults to the local back-end:
* **Pithos backend**:
If the **img_id** is prefixed with ``pithos://`` or ``pithosmap://`` the
image is considered to reside on a Pithos deployment. For ``pithosmap://``
images, the user needs to have set a valid value for the
``PITHOS_DATA`` variable in snf-image's configuration file
(``/etc/default/snf-image`` by default). For ``pithos://`` images, in
addition to ``PITHOS_DATA``, the user needs to have set a valid value for the
images, the user needs to have set a valid value for the ``PITHOS_DATA``
variable in snf-image's configuration file (``/etc/default/snf-image`` by
default) if the storage backend is ``nfs`` or ``PITHOS_RADOS_POOL_MAPS`` and
``PITHOS_RADOS_POOL_BLOCKS`` if the storage backend is ``rados``.
For ``pithos://`` images, in addition to ``PITHOS_DATA`` or
``PITHOS_RADOS_POOL_*``, the user needs to have set a valid value for the
``PITHOS_DB`` variable, too.
| For example, if we want to deploy using a full Pithos URI:
......
......@@ -11,10 +11,6 @@ following images are basic installations of some popular Linux distributions,
that have been tested with snf-image and provided here for testing purposes:
* Debian Squeeze Base System
[`diskdump <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.meta>`_]
* Debian Wheezy Base System
[`diskdump <http://cdn.synnefo.org/debian_base-7.0-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/debian_base-7.0-x86_64.diskdump.md5sum>`_]
......@@ -31,38 +27,34 @@ that have been tested with snf-image and provided here for testing purposes:
[`diskdump <http://cdn.synnefo.org/fedora-20-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/fedora-20-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/fedora-20-x86_64.diskdump.meta>`_]
* Ubuntu Desktop LTS 12.04
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump.meta>`_]
* Kubuntu LTS 12.04
[`diskdump <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump.meta>`_]
* Ubuntu Desktop 13.10
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump.meta>`_]
* Kubuntu 13.10
[`diskdump <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump.meta>`_]
* Ubuntu Server 12.04
[`diskdump <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump.meta>`_]
* Ubuntu Desktop LTS 14.04
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump.meta>`_]
* Kubuntu LTS 14.04
[`diskdump <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump.meta>`_]
* Ubuntu Server LTS 14.04
[`diskdump <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump.meta>`_]
* OpenSUSE Desktop 13.1
[`diskdump <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump.meta>`_]
* Oracle Linux 6.5
[`diskdump <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump.meta>`_]
* FreeBSD 9.2
[`diskdump <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump.meta>`_]
* OpenBSD 5.4
[`diskdump <http://cdn.synnefo.org/openbsd-5.4-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/openbsd-5.4-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/openbsd-5.4-x86_64.diskdump.meta>`_]
* OpenBSD 5.5
[`diskdump <http://cdn.synnefo.org/openbsd-5.5-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/openbsd-5.5-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/openbsd-5.5-x86_64.diskdump.meta>`_]
* NetBSD 6.1
[`diskdump <http://cdn.synnefo.org/netbsd-6.1-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/netbsd-6.1-x86_64.diskdump.md5sum>`_]
......
__version__ = "0.15rc1"
__version__ = "%(DEVFLOW_VERSION)s"
......@@ -413,7 +413,7 @@ create_partition() {
# For GPT
start=${start:0:${#start}-1} # remove the s at the end
end=${end:0:${#end}-1} # remove the s at the end
$SGDISK -n "$id":"$start":"$end" -t "$id":"$ptype" "$device"
$SGDISK -a 1 -n "$id":"$start":"$end" -t "$id":"$ptype" "$device"
fi
}
......
......@@ -4,3 +4,4 @@ Constantinos Venetsanopoulos <cven@grnet.gr>
Vangelis Koukis <vkoukis@grnet.gr>
Giorgos Verigakis <verigak@grnet.gr>
Christos Stavrakakis <cstavr@grnet.gr>
Chrysostomos Nanakos <cnanakos@grnet.gr>
......@@ -454,6 +454,10 @@ fi
: ${HELPER_MEMORY:="512"}
: ${PITHOS_DB:="sqlite:////@localstatedir@/lib/pithos/backend.db"}
: ${PITHOS_DATA:="@localstatedir@/lib/pithos/data/"}
: ${PITHOS_BACKEND_STORAGE:="nfs"}
: ${PITHOS_RADOS_CEPH_CONF:="@sysconfdir@/ceph/ceph.conf"}
: ${PITHOS_RADOS_POOL_MAPS:="maps"}
: ${PITHOS_RADOS_POOL_BLOCKS:="blocks"}
: ${PROGRESS_MONITOR:="@PROGRESS_MONITOR@"}
: ${UNATTEND:="@UNATTEND@"}
: ${XEN_SCRIPTS_DIR="@sysconfdir@/xen/scripts"}
......
......@@ -111,6 +111,10 @@ case $BACKEND_TYPE in
# environmental variable.
export PITHCAT_INPUT_DB="$PITHOS_DB"
export PITHCAT_INPUT_DATA="$PITHOS_DATA"
export PITHCAT_BACKEND_STORAGE="$PITHOS_BACKEND_STORAGE"
export PITHCAT_RADOS_CEPH_CONF="$PITHOS_RADOS_CEPH_CONF"
export PITHCAT_RADOS_POOL_MAPS="$PITHOS_RADOS_POOL_MAPS"
export PITHCAT_RADOS_POOL_BLOCKS="$PITHOS_RADOS_POOL_BLOCKS"
cmd_args="$(printf "%q" "${IMAGE_NAME}")"
image_cmd="./pithcat $cmd_args"
image_size=$(./pithcat -s $cmd_args)
......
......@@ -50,13 +50,29 @@
# MULTISTRAP_APTPREFDIR="@MULTISTRAP_APTPREFDIR@"
# XEN_SCRIPTS_DIR: Directory where the Xen scripts are stored
# XEN_SCRIPTS_DIR=="@sysconfdir@/xen/scripts"
# XEN_SCRIPTS_DIR="@sysconfdir@/xen/scripts"
# PITHOS_DB: Pithos database in SQLAlchemy format
# PITHOS_DB="sqlite:////@localstatedir@/lib/pithos/backend.db"
# PITHOS_BACKEND_STORAGE: Select Pithos backend storage. Possible values are
# 'nfs' and 'rados'. According to the value you select, you need to set the
# corresponding variables that follow.
# If you select 'nfs' that's 'PITHOS_DATA'. If you select 'rados' then you need
# to set all the "*_RADOS_*" ones.
# PITHOS_BACKEND_STORAGE="nfs"
# PITHOS_DATA: Directory where Pithos data are hosted
# PITHOS_DATA="/@localstatedir@/lib/pithos/data"
#
# PITHOS_RADOS_CEPH_CONF: RADOS configuration file
# PITHOS_RADOS_CEPH_CONF="@sysconfdir@/ceph/ceph.conf"
# PITHOS_RADOS_POOL_MAPS: RADOS pool for storing Pithos maps
# PITHOS_RADOS_POOL_MAPS="maps"
# PITHOS_RADOS_POOL_BLOCKS: RADOS pool for storing Pithos blocks
# PITHOS_RADOS_POOL_BLOCKS="blocks"
# PROGRESS_MONITOR: External program that monitors the progress of the image
# deployment. The snf-image monitor messages will be redirected to the standard
......
......@@ -41,6 +41,14 @@ except ImportError:
parser = OptionParser(usage='%prog [options] <URL>')
parser.add_option('--data', dest='data', metavar='DIR',
help='path to the directory where data are stored')
parser.add_option('--backend', dest='backend', metavar='BACKEND',
help='Pithos backend storage type')
parser.add_option('--rados-conf', dest='rconf', metavar='RCONF',
help='RADOS configuration file to use')
parser.add_option('--rados-maps', dest='rmaps', metavar='RMAPS',
help='RADOS pool which Pithos maps reside')
parser.add_option('--rados-blocks', dest='rblocks', metavar='RBLOCKS',
help='RADOS pool which Pithos blocks reside')
parser.add_option('-s', action='store_true', dest='size', default=False,
help='print file size and exit')
group = OptionGroup(
......@@ -133,11 +141,22 @@ def main():
exit(1)
db_uri = environ['PITHCAT_INPUT_DB'] if not options.db else options.db
backend_storage = environ['PITHCAT_BACKEND_STORAGE'] if not \
options.backend else options.backend
rados_ceph_conf = environ['PITHCAT_RADOS_CEPH_CONF'] if not \
options.rconf else options.rconf
rados_maps = environ['PITHCAT_RADOS_POOL_MAPS'] if not options.rmaps else \
options.rmaps
rados_blocks = environ['PITHCAT_RADOS_POOL_BLOCKS'] if not \
options.rblocks else options.rblocks
block_params = {'mappool': rados_maps, 'blockpool': rados_blocks}
backend = ModularBackend(None,
db_uri if type(url) is LocationURL else None,
None,
data_path)
data_path, block_params=block_params,
backend_storage=backend_storage,
rados_ceph_conf=rados_ceph_conf)
if options.size:
print_size(backend, url)
......
m4_define([devflow_version], [0.14])
m4_define([devflow_version], [0.15rc1])
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment