Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
snf-image
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Incidents
Environments
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
itminedu
snf-image
Commits
d1fb3a36
Commit
d1fb3a36
authored
May 16, 2014
by
Nikos Skalkotos
Browse files
Options
Browse Files
Download
Plain Diff
Merge branch 'release-0.15'
parents
71842ea0
d3d77c06
Changes
18
Hide whitespace changes
Inline
Side-by-side
Showing
18 changed files
with
311 additions
and
94 deletions
+311
-94
ChangeLog
ChangeLog
+14
-3
devflow.conf
devflow.conf
+2
-2
docs/architecture.rst
docs/architecture.rst
+12
-8
docs/conf.py
docs/conf.py
+4
-3
docs/configuration.rst
docs/configuration.rst
+16
-0
docs/installation.rst
docs/installation.rst
+57
-12
docs/interface.rst
docs/interface.rst
+6
-4
docs/usage.rst
docs/usage.rst
+20
-28
docs/version.py
docs/version.py
+1
-0
docs/version_template
docs/version_template
+1
-0
snf-image-helper/common.sh.in
snf-image-helper/common.sh.in
+1
-1
snf-image-host/AUTHORS
snf-image-host/AUTHORS
+1
-0
snf-image-host/common.sh.in
snf-image-host/common.sh.in
+4
-0
snf-image-host/create
snf-image-host/create
+6
-2
snf-image-host/defaults.in
snf-image-host/defaults.in
+17
-1
snf-image-host/pithcat
snf-image-host/pithcat
+147
-28
version
version
+1
-1
version.m4
version.m4
+1
-1
No files found.
ChangeLog
View file @
d1fb3a36
2014-16-05, v0.15rc3
* Fix bugs in pithcat introduced in rc2
2014-15-05, v0.15rc2
* Add pithos backend version check in pithcat.
2014-13-05, v0.15rc1
* Add support for selectable pithos storage backend. Possible storage
backends for pithos are `nfs' or `rados'
* Fix bugs and update the documentation
2014-04-11, v0.14
* Add support for extending NetBSD partitions
* Change the default Virtual RAM size of the helper VM to 512M
...
...
@@ -151,9 +162,9 @@
* Use Coordinated Universal Time when logging.
* Fix a bug in the way snf-image-update-helper reads the links of the
kernel and the initrd in the helper VM.
* Use environment
al variables instead of input options to pass
sensitive data to pithcat. This is needed because when pithcat is
running,
every user can see the program's command line by using ps.
* Use environment
variables instead of input options to pass sensitive
data to pithcat. This is needed because when pithcat is running,
every user can see the program's command line by using ps.
2012-09-15, v0.6
* Fix import and export scripts
...
...
devflow.conf
View file @
d1fb3a36
[
packages
]
[[
devflow
]]
version_file
=
"version.m4"
version_template
=
"version_template"
version_file
=
"version.m4"
,
"docs/version.py"
version_template
=
"version_template"
,
"docs/version_template"
docs/architecture.rst
View file @
d1fb3a36
...
...
@@ -106,8 +106,12 @@ newly created block device. The following back-ends are supported:
* **Pithos backend**:
*snf-image* contains a special command-line tool (*pithcat*) for retrieving
images residing on a Pithos installation. To set up snf-image's Pithos
backend the user needs to setup the ``PITHOS_DATA`` and ``PITHOS_DB``
variables inside ``/etc/default/snf-image`` accordingly.
backend the user needs to setup the ``PITHOS_BACKEND_STORAGE`` variable
inside ``/etc/default/snf-image``.
Possible values are ``nfs`` and ``rados``. If ``nfs`` is used the user needs
to setup ``PITHOS_DATA`` variable, and when ``rados`` is used the user needs
to setup ``PITHOS_RADOS_POOL_MAPS`` and ``PITHOS_RADOS_POOL_BLOCKS``
accordingly.
* **Null backend**:
If the null backend is selected, no image copying is performed. This
...
...
@@ -127,20 +131,20 @@ below for a description of each one of them:
**FixPartitionTable**: Enlarges the last partition in the partition table of
the instance, to consume all the available space and optionally adds a swap
partition in the end. The task will fail if the environment
al
variable
partition in the end. The task will fail if the environment variable
*SNF_IMAGE_DEV*, which specifies the device file of the instance's hard disk,
is missing.
**FilesystemResizeUnmounted**: Extends the file system of the last partition to
cover up the whole partition. This only works for ext{2,3,4}, FFS and UFS2 file
systems. Any other file system type is ignored and a warning is triggered. The
task will fail if *SNF_IMAGE_DEV* environment
al
variable is missing.
task will fail if *SNF_IMAGE_DEV* environment variable is missing.
**MountImage**: Mounts the root partition of the instance, specified by the
*SNF_IMAGE_PROPERTY_ROOT_PARTITION* variable. On Linux systems after the root
fs is mounted, the instance's ``/etc/fstab`` file is examined and the rest of
the disk file systems are mounted too, in a correct order. The script will fail
if any of the environment
al
variables *SNF_IMAGE_DEV*,
if any of the environment variables *SNF_IMAGE_DEV*,
*SNF_IMAGE_PROPERTY_ROOT_PARTITION* or *SNF_IMAGE_TARGET* is unset or has a
non-sane value.
...
...
@@ -167,7 +171,7 @@ is needed by Windows in order to perform an unattended setup. The
**SELinuxAutorelabel**: Creates *.autorelabel* file in Red Hat images. This is
needed if SELinux is enabled to enforce an automatic file system relabeling
during the first boot. The only environment
al
variable required by this task is
during the first boot. The only environment variable required by this task is
*SNF_IMAGE_TARGET*.
**AssignHostname**: Assigns or changes the hostname of the instance. The task
...
...
@@ -200,7 +204,7 @@ missing a warning is produced. Only *SNF_IMAGE_TARGET* is required for this
task to run.
**UmountImage**: Umounts the file systems previously mounted by MountImage. The
only environment
al
variable required is *SNF_IMAGE_TARGET*.
only environment variable required is *SNF_IMAGE_TARGET*.
+-------------------------------+---+--------------------------------------------+--------------------------------------------------+
...
...
@@ -241,4 +245,4 @@ only environmental variable required is *SNF_IMAGE_TARGET*.
|UmountImage |80 |MountImage | |TARGET | |
+-------------------------------+---+------------------+-------------------------+-------------------------+------------------------+
.. [#] all environment
al
variables are prefixed with *SNF_IMAGE_*
.. [#] all environment variables are prefixed with *SNF_IMAGE_*
docs/conf.py
View file @
d1fb3a36
...
...
@@ -12,7 +12,8 @@
# serve to show the default.
import
sys
,
os
sys
.
path
.
insert
(
0
,
os
.
path
.
dirname
(
os
.
path
.
realpath
(
__file__
)))
from
version
import
__version__
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
...
...
@@ -45,9 +46,9 @@ copyright = u'2011, 2012, 2013 GRNET S.A. All rights reserved'
# built documents.
#
# The short X.Y version.
version
=
'0.14'
version
=
__version__
# The full version, including alpha/beta/rc tags.
release
=
'0.14'
release
=
__version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
...
...
docs/configuration.rst
View file @
d1fb3a36
...
...
@@ -64,9 +64,25 @@ some external programs in ``/etc/default/snf-image``:
# PITHOS_DB: Pithos database in SQLAlchemy format
# PITHOS_DB="sqlite://///var/lib/pithos/backend.db"
# PITHOS_BACKEND_STORAGE: Select Pithos backend storage. Possible values are
# 'nfs' and 'rados'. According to the value you select, you need to set the
# corresponding variables that follow.
# If you select 'nfs' that's 'PITHOS_DATA'. If you select 'rados' then you
# need to set all the "*_RADOS_*" ones.
# PITHOS_BACKEND_STORAGE="nfs"
# PITHOS_DATA: Directory where Pithos data are hosted
# PITHOS_DATA="//var/lib/pithos/data"
# PITHOS_RADOS_CEPH_CONF: RADOS configuration file
# PITHOS_RADOS_CEPH_CONF="@sysconfdir@/ceph/ceph.conf"
# PITHOS_RADOS_POOL_MAPS: RADOS pool for storing Pithos maps
# PITHOS_RADOS_POOL_MAPS="maps"
# PITHOS_RADOS_POOL_BLOCKS: RADOS pool for storing Pithos blocks
# PITHOS_RADOS_POOL_BLOCKS="blocks"
# PROGRESS_MONITOR: External program that monitors the progress of image
# deployment. Monitoring messages will be redirected to the standard input of
# this program.
...
...
docs/installation.rst
View file @
d1fb3a36
...
...
@@ -8,17 +8,39 @@ identical for all nodes and should be repeated manually or automatically, e.g.,
with puppet.
Installing snf-image using packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-----------------------------------
For Debian GNU/Linux and Ubuntu we provide packages in our APT repository.
Debian GNU/Linux
^^^^^^^^^^^^^^^^
For Debian Wheezy add the following lines to ``/etc/apt/sources.list`` file:
For Debian 7.x (Wheezy) we provide packages in our APT repository. To use
our repository add the following lines to file ``/etc/apt/sources.list``:
``deb http://apt.dev.grnet.gr wheezy/``
``deb-src http://apt.dev.grnet.gr wheezy/``
For Ubuntu 12.04 LTS add the following lines:
After you update ``/etc/apt/sources.list`` import the repo's GPG key:
.. code-block:: console
# curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
To install the package use the following commands:
.. code-block:: console
# apt-get update
# apt-get install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
Ubuntu
^^^^^^
For Ubuntu 12.04 LTS we provide packages in our APT repository. To use our
repository add the following lines to file ``/etc/apt/sources.list``:
``deb http://apt.dev.grnet.gr precise/``
...
...
@@ -28,20 +50,43 @@ After you update ``/etc/apt/sources.list`` import the repo's GPG key:
.. code-block:: console
$
curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
#
curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
To install the package use the following commands:
.. code-block:: console
$ apt-get update
$ apt-get install snf-image
# apt-get update
# apt-get install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
CentOS
^^^^^^
For CentOS 6.5 we provide packages in our Yum repository.
To add the GRNET repository in your system, run:
.. code-block:: console
# yum localinstall https://dev.grnet.gr/files/grnet-repo.rpm
You can verify the authenticity of the package using our public key found
`here <https://dev.grnet.gr/files/apt-grnetdev.pub>`_.
To install snf-image run:
.. code-block:: console
# yum install snf-image
The last command will also download and install the *snf-image-helper* image in
the post install phase of the package installation.
Installing snf-image from source
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
--------------------------------
To install snf-image from source, download the provided source package:
...
...
@@ -63,13 +108,13 @@ Install snf-image:
.. code-block:: console
$
make install
$
install -Dm600 defaults /etc/default/snf-image
$
mkdir -p /var/lib/snf-image/helper
#
make install
#
install -Dm600 defaults /etc/default/snf-image
#
mkdir -p /var/lib/snf-image/helper
Finally, install the helper image by executing:
.. code-block:: console
$
snf-image-update-helper
#
snf-image-update-helper
docs/interface.rst
View file @
d1fb3a36
...
...
@@ -66,10 +66,12 @@ to be used. If no prefix is used, it defaults to the local back-end:
* **Pithos backend**:
If the **img_id** is prefixed with ``pithos://`` or ``pithosmap://`` the
image is considered to reside on a Pithos deployment. For ``pithosmap://``
images, the user needs to have set a valid value for the
``PITHOS_DATA`` variable in snf-image's configuration file
(``/etc/default/snf-image`` by default). For ``pithos://`` images, in
addition to ``PITHOS_DATA``, the user needs to have set a valid value for the
images, the user needs to have set a valid value for the ``PITHOS_DATA``
variable in snf-image's configuration file (``/etc/default/snf-image`` by
default) if the storage backend is ``nfs`` or ``PITHOS_RADOS_POOL_MAPS`` and
``PITHOS_RADOS_POOL_BLOCKS`` if the storage backend is ``rados``.
For ``pithos://`` images, in addition to ``PITHOS_DATA`` or
``PITHOS_RADOS_POOL_*``, the user needs to have set a valid value for the
``PITHOS_DB`` variable, too.
| For example, if we want to deploy using a full Pithos URI:
...
...
docs/usage.rst
View file @
d1fb3a36
...
...
@@ -11,10 +11,6 @@ following images are basic installations of some popular Linux distributions,
that have been tested with snf-image and provided here for testing purposes:
* Debian Squeeze Base System
[`diskdump <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.meta>`_]
* Debian Wheezy Base System
[`diskdump <http://cdn.synnefo.org/debian_base-7.0-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/debian_base-7.0-x86_64.diskdump.md5sum>`_]
...
...
@@ -31,38 +27,34 @@ that have been tested with snf-image and provided here for testing purposes:
[`diskdump <http://cdn.synnefo.org/fedora-20-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/fedora-20-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/fedora-20-x86_64.diskdump.meta>`_]
* Ubuntu Desktop LTS 12.04
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-12.04-x86_64.diskdump.meta>`_]
* Kubuntu LTS 12.04
[`diskdump <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu_desktop-12.04-x86_64.diskdump.meta>`_]
* Ubuntu Desktop 13.10
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-13.10-x86_64.diskdump.meta>`_]
* Kubuntu 13.10
[`diskdump <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu_desktop-13.10-x86_64.diskdump.meta>`_]
* Ubuntu Server 12.04
[`diskdump <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_server-12.04-x86_64.diskdump.meta>`_]
* Ubuntu Desktop LTS 14.04
[`diskdump <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_desktop-14.04-x86_64.diskdump.meta>`_]
* Kubuntu LTS 14.04
[`diskdump <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/kubuntu-14.10-x86_64.diskdump.meta>`_]
* Ubuntu Server LTS 14.04
[`diskdump <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/ubuntu_server-14.04-x86_64.diskdump.meta>`_]
* OpenSUSE Desktop 13.1
[`diskdump <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/opensuse_desktop-13.1-x86_64.diskdump.meta>`_]
* Oracle Linux 6.5
[`diskdump <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/oraclelinux-6.5-x86_64.diskdump.meta>`_]
* FreeBSD 9.2
[`diskdump <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/freebsd-9.2-x86_64.diskdump.meta>`_]
* OpenBSD 5.
4
[`diskdump <http://cdn.synnefo.org/openbsd-5.
4
-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/openbsd-5.
4
-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/openbsd-5.
4
-x86_64.diskdump.meta>`_]
* OpenBSD 5.
5
[`diskdump <http://cdn.synnefo.org/openbsd-5.
5
-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/openbsd-5.
5
-x86_64.diskdump.md5sum>`_]
[`metadata <http://cdn.synnefo.org/openbsd-5.
5
-x86_64.diskdump.meta>`_]
* NetBSD 6.1
[`diskdump <http://cdn.synnefo.org/netbsd-6.1-x86_64.diskdump>`_]
[`md5sum <http://cdn.synnefo.org/netbsd-6.1-x86_64.diskdump.md5sum>`_]
...
...
docs/version.py
0 → 100644
View file @
d1fb3a36
__version__
=
"0.15rc3"
docs/version_template
0 → 100644
View file @
d1fb3a36
__version__ = "%(DEVFLOW_VERSION)s"
snf-image-helper/common.sh.in
View file @
d1fb3a36
...
...
@@ -413,7 +413,7 @@ create_partition() {
# For GPT
start=${start:0:${#start}-1} # remove the s at the end
end=${end:0:${#end}-1} # remove the s at the end
$SGDISK -n "$id":"$start":"$end" -t "$id":"$ptype" "$device"
$SGDISK -
a 1 -
n "$id":"$start":"$end" -t "$id":"$ptype" "$device"
fi
}
...
...
snf-image-host/AUTHORS
View file @
d1fb3a36
...
...
@@ -4,3 +4,4 @@ Constantinos Venetsanopoulos <cven@grnet.gr>
Vangelis Koukis <vkoukis@grnet.gr>
Giorgos Verigakis <verigak@grnet.gr>
Christos Stavrakakis <cstavr@grnet.gr>
Chrysostomos Nanakos <cnanakos@grnet.gr>
snf-image-host/common.sh.in
View file @
d1fb3a36
...
...
@@ -454,6 +454,10 @@ fi
: ${HELPER_MEMORY:="512"}
: ${PITHOS_DB:="sqlite:////@localstatedir@/lib/pithos/backend.db"}
: ${PITHOS_DATA:="@localstatedir@/lib/pithos/data/"}
: ${PITHOS_BACKEND_STORAGE:="nfs"}
: ${PITHOS_RADOS_CEPH_CONF:="@sysconfdir@/ceph/ceph.conf"}
: ${PITHOS_RADOS_POOL_MAPS:="maps"}
: ${PITHOS_RADOS_POOL_BLOCKS:="blocks"}
: ${PROGRESS_MONITOR:="@PROGRESS_MONITOR@"}
: ${UNATTEND:="@UNATTEND@"}
: ${XEN_SCRIPTS_DIR="@sysconfdir@/xen/scripts"}
...
...
snf-image-host/create
View file @
d1fb3a36
...
...
@@ -107,10 +107,14 @@ case $BACKEND_TYPE in
image_size
=
$(
$CURL
-sI
"
$IMAGE_NAME
"
|
grep
^Content-Length: |
cut
-d
" "
-f2
)
;;
pithos
)
# For security reasons pass the
database url to pithcat as an
# environment
al variable
.
# For security reasons pass the
various options to pithcat as
# environment
variables
.
export
PITHCAT_INPUT_DB
=
"
$PITHOS_DB
"
export
PITHCAT_INPUT_DATA
=
"
$PITHOS_DATA
"
export
PITHCAT_BACKEND_STORAGE
=
"
$PITHOS_BACKEND_STORAGE
"
export
PITHCAT_RADOS_CEPH_CONF
=
"
$PITHOS_RADOS_CEPH_CONF
"
export
PITHCAT_RADOS_POOL_MAPS
=
"
$PITHOS_RADOS_POOL_MAPS
"
export
PITHCAT_RADOS_POOL_BLOCKS
=
"
$PITHOS_RADOS_POOL_BLOCKS
"
cmd_args
=
"
$(
printf
"%q"
"
${
IMAGE_NAME
}
"
)
"
image_cmd
=
"./pithcat
$cmd_args
"
image_size
=
$(
./pithcat
-s
$cmd_args
)
...
...
snf-image-host/defaults.in
View file @
d1fb3a36
...
...
@@ -50,13 +50,29 @@
# MULTISTRAP_APTPREFDIR="@MULTISTRAP_APTPREFDIR@"
# XEN_SCRIPTS_DIR: Directory where the Xen scripts are stored
# XEN_SCRIPTS_DIR=
=
"@sysconfdir@/xen/scripts"
# XEN_SCRIPTS_DIR="@sysconfdir@/xen/scripts"
# PITHOS_DB: Pithos database in SQLAlchemy format
# PITHOS_DB="sqlite:////@localstatedir@/lib/pithos/backend.db"
# PITHOS_BACKEND_STORAGE: Select Pithos backend storage. Possible values are
# 'nfs' and 'rados'. According to the value you select, you need to set the
# corresponding variables that follow.
# If you select 'nfs' that's 'PITHOS_DATA'. If you select 'rados' then you need
# to set all the "*_RADOS_*" ones.
# PITHOS_BACKEND_STORAGE="nfs"
# PITHOS_DATA: Directory where Pithos data are hosted
# PITHOS_DATA="/@localstatedir@/lib/pithos/data"
#
# PITHOS_RADOS_CEPH_CONF: RADOS configuration file
# PITHOS_RADOS_CEPH_CONF="@sysconfdir@/ceph/ceph.conf"
# PITHOS_RADOS_POOL_MAPS: RADOS pool for storing Pithos maps
# PITHOS_RADOS_POOL_MAPS="maps"
# PITHOS_RADOS_POOL_BLOCKS: RADOS pool for storing Pithos blocks
# PITHOS_RADOS_POOL_BLOCKS="blocks"
# PROGRESS_MONITOR: External program that monitors the progress of the image
# deployment. The snf-image monitor messages will be redirected to the standard
...
...
snf-image-host/pithcat
View file @
d1fb3a36
#!/usr/bin/env python
# Copyright (C) 2011-201
3
GRNET S.A.
# Copyright (C) 2011-201
4
GRNET S.A.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
...
...
@@ -30,29 +30,60 @@ from sys import exit, stdout, stderr
from
os
import
environ
from
binascii
import
hexlify
,
unhexlify
from
collections
import
namedtuple
from
pkg_resources
import
parse_version
try
:
from
pithos.backends.modular
import
ModularBackend
from
pithos.backends.version
import
__version__
as
pithos_backend_version
except
ImportError
:
stderr
.
write
(
"Pithos backend was not found.
\n
"
)
exit
(
2
)
SELECTABLE_BE_VER
=
"0.15.1"
note
=
"""
NOTE: You can pass all arguments through environment variables instead of
the command line: Setting the environment variable PITHCAT_INPUX_XXX to
VALUE is equivalent to passing a '--xxx VALUE' argument.
Using the --db argument directly is dangerous, because it may
expose sensitive information in the output of 'ps'. Consider passing
the DB URI through the environment variable PITHOS_INPUT_DB instead.
\n
"""
OptionParser
.
format_epilog
=
lambda
self
,
formattxt
:
self
.
epilog
parser
=
OptionParser
(
usage
=
'%prog [options] <URL>'
,
epilog
=
note
)
if
parse_version
(
pithos_backend_version
)
>=
parse_version
(
SELECTABLE_BE_VER
):
backend_group
=
OptionGroup
(
parser
,
"Backend-specific Options"
,
"The backend-specific options depend on the specific "
"version of Pithos backend (package snf-pithos-backend) installed.
\n
"
"(Currently-installed version: Pithos backend '%s')"
%
pithos_backend_version
)
backend_group
.
add_option
(
'--backend'
,
dest
=
'backend'
,
metavar
=
'BACKEND'
,
help
=
'Pithos backend storage type'
,
default
=
None
)
backend_group
.
add_option
(
'--rados-conf'
,
dest
=
'rconf'
,
metavar
=
'RCONF'
,
help
=
'RADOS configuration file to use'
,
default
=
None
)
backend_group
.
add_option
(
'--rados-maps'
,
dest
=
'rmaps'
,
metavar
=
'RMAPS'
,
help
=
'RADOS pool which Pithos maps reside'
,
default
=
None
)
backend_group
.
add_option
(
'--rados-blocks'
,
dest
=
'rblocks'
,
metavar
=
'RBLOCKS'
,
help
=
'RADOS pool which Pithos blocks reside'
,
default
=
None
)
backend_group
.
add_option
(
'--data'
,
dest
=
'data'
,
metavar
=
'DIR'
,
help
=
'path to the directory where data are stored'
)
parser
.
add_option_group
(
backend_group
)
else
:
parser
.
add_option
(
'--data'
,
dest
=
'data'
,
metavar
=
'DIR'
,
help
=
'path to the directory where data are stored'
)
parser
=
OptionParser
(
usage
=
'%prog [options] <URL>'
)
parser
.
add_option
(
'--data'
,
dest
=
'data'
,
metavar
=
'DIR'
,
help
=
'path to the directory where data are stored'
)
parser
.
add_option
(
'-s'
,
action
=
'store_true'
,
dest
=
'size'
,
default
=
False
,
help
=
'print file size and exit'
)
group
=
OptionGroup
(
parser
,
"Dangerous Options"
,
"Caution: If the <URL> is a LocationURL (pithos://...), then you'll also "
"need to define a database URI. You can use the `--db' option to do so, "
"but this raises security concerns. For database URI's and pithos data "
"paths, the recommended way to define them is to use the PITHCAT_INPUT_DB "
"and PITHCAT_INPUT_DATA environmental variables respectfully."
)
group
.
add_option
(
'--db'
,
dest
=
'db'
,
metavar
=
'URI'
,
help
=
'SQLAlchemy URI of the database'
,
default
=
None
)
parser
.
add_option_group
(
group
)
parser
.
add_option
(
'--db'
,
dest
=
'db'
,
metavar
=
'URI'
,
help
=
'SQLAlchemy URI of the database [DANGEROUS: Do not use,'
'see NOTE below]'
,
default
=
None
)
LocationURL
=
namedtuple
(
'LocationURL'
,
[
'account'
,
'container'
,
'object'
])
HashmapURL
=
namedtuple
(
'HashmapURL'
,
[
'hash'
,
'size'
])
...
...
@@ -114,30 +145,118 @@ def main():
url
=
parse_url
(
args
[
0
])
if
not
options
.
data
and
'PITHCAT_INPUT_DATA'
not
in
environ
:
stderr
.
write
(
"Pithos data directory path is missing.
\n
"
"Use the PITHCAT_INPUT_DATA environmental variable (recommended) "
"or the --data command line option to define it.
\n
"
)
exit
(
1
)
data_path
=
None
if
parse_version
(
pithos_backend_version
)
>=
\
parse_version
(
SELECTABLE_BE_VER
):
data_path
=
environ
[
'PITHCAT_INPUT_DATA'
]
if
not
options
.
data
else
\
options
.
data
if
not
options
.
backend
and
'PITHCAT_BACKEND_STORAGE'
not
in
environ
:
stderr
.
write
(
"Pithos backend storage is missing.
\n
"
"Use the PITHCAT_BACKEND_STORAGE environment variable "
"(recommended) "
"or the --backend command line option to define it.
\n
"
)
exit
(
1
)
if
options
.
backend
==
'nfs'
or
\
environ
.
get
(
'PITHCAT_BACKEND_STORAGE'
)
==
'nfs'
:
if
not
options
.
data
and
'PITHCAT_INPUT_DATA'
not
in
environ
:
stderr
.
write
(
"Pithos data directory path is missing.
\n
"
"Use the PITHCAT_INPUT_DATA environment variable "
"(recommended) "
"or the --data command line option to define it.
\n
"
)
exit
(
1
)
data_path
=
environ
[
'PITHCAT_INPUT_DATA'
]
if
not
options
.
data
else
\
options
.
data
else
:
if
not
options
.
data
and
'PITHCAT_INPUT_DATA'
not
in
environ
:
stderr
.
write
(
"Pithos data directory path is missing.
\n
"
"Use the PITHCAT_INPUT_DATA environment variable "
"(recommended) "
"or the --data command line option to define it.
\n
"
)
exit
(
1
)
data_path
=
environ
[
'PITHCAT_INPUT_DATA'
]
if
not
options
.
data
else
\
options
.
data
if
options
.
db
is
None
and
'PITHCAT_INPUT_DB'
not
in
environ
and
\
type
(
url
)
is
LocationURL
:
stderr
.
write
(
"Pithos database URI is missing.
\n
"
"Use the PITHCAT_INPUT_DB environment
al
variable (recommended) "
"Use the PITHCAT_INPUT_DB environment variable (recommended) "
"or the --db command line option to define it.
\n
"
)
exit
(
1
)
db_uri
=
environ
[
'PITHCAT_INPUT_DB'
]
if
not
options
.
db
else
options
.
db
if
type
(
url
)
is
HashmapURL
:
db_uri
=
None
else
:
db_uri
=
environ
[
'PITHCAT_INPUT_DB'
]
if
not
options
.
db
else
options
.
db
backend
=
ModularBackend
(
None
,
db_uri
if
type
(
url
)
is
LocationURL
else
None
,
None
,
data_path
)
if
parse_version
(
pithos_backend_version
)
>=
\
parse_version
(
SELECTABLE_BE_VER
):
block_params
=
{
'mappool'
:
None
,
'blockpool'
:
None
}
rados_ceph_conf
=
None
backend_storage
=
environ
[
'PITHCAT_BACKEND_STORAGE'
]
if
not
\
options
.
backend
else
options
.
backend
if
backend_storage
==
'rados'
and
options
.
rconf
is
None
and
\
'PITHCAT_RADOS_CEPH_CONF'
not
in
environ
:
stderr
.
write
(
"RADOS storage backend is selected but "
"RADOS conf file is missing.
\n
"
"Use the PITHCAT_RADOS_CEPH_CONF environment variable or the "
"--rados-conf command line option to define it.
\n
"
)
exit
(
1
)
if
backend_storage
==
'rados'
:
rados_ceph_conf
=
environ
[
'PITHCAT_RADOS_CEPH_CONF'
]
if
not
\
options
.
rconf
else
options
.
rconf
if
options
.
rmaps
is
None
and
'PITHCAT_RADOS_POOL_MAPS'
not
in
\
environ
:
stderr
.
write
(
"RADOS storage backend is selected but "
"RADOS pool for Pithos maps is missing.
\n
"
"Use the PITHCAT_RADOS_POOL_MAPS environment variable or "
"the --rados-maps command line option to define it.
\n
"
)
exit
(
1
)
rados_maps
=
environ
[
'PITHCAT_RADOS_POOL_MAPS'
]
if
not
\
options
.
rmaps
else
options
.
rmaps
if
options
.
rblocks
is
None
and
'PITHCAT_RADOS_POOL_BLOCKS'
not
in
\
environ
:
stderr
.
write
(
"RADOS storage backend is selected but "
"RADOS pool for Pithos blocks is missing.
\n
"
"Use the PITHCAT_RADOS_POOL_BLOCKS environment variable "
"or the --rados-blocks command line option to define it.
\n
"
)
exit
(
1
)
rados_blocks
=
environ
[
'PITHCAT_RADOS_POOL_BLOCKS'
]
if
not
\
options
.
rblocks
else
options
.
rblocks
block_params
[
'blockpool'
]
=
rados_blocks
block_params
[
'mappool'
]
=
rados_maps
backend
=
ModularBackend
(
None
,
db_uri
if
type
(
url
)
is
LocationURL
else
None
,
None
,
data_path
,
block_params
=
block_params
,
backend_storage
=
backend_storage
,
rados_ceph_conf
=
rados_ceph_conf
)
else
:
backend
=
ModularBackend
(
None
,
db_uri
if
type
(
url
)
is
LocationURL
else
None
,
None
,
data_path
)
if
options
.
size
:
print_size
(
backend
,
url
)
...
...
version
View file @
d1fb3a36
0.1
4
0.1
5rc3
version.m4
View file @
d1fb3a36
m4_define([devflow_version], [0.1
4
])
m4_define([devflow_version], [0.1
5rc3
])
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment