Skip to content

GitLab

Commit fcda074a authored by Vangelis Koukis's avatar Vangelis Koukis
Browse files

Improve documentation on compute service 

Improve documentation on compute service,
document snf-webproject, snf-asterias-{app,ganeti-tools}
components.

Import initial version of developer guide,
merge developer-related documents into it.
parent fa97c0fa
Hide whitespace changes
Inline Side-by-side
snf-app/docs/src/admin-guide.rst → snf-app/docs/src/asterias-admin-guide.rst
View file @ fcda074a
......@@ -35,6 +35,8 @@ Changelogs
.. todo:: describe prerequisites -- e.g., Debian
.. todo:: describe setup of nginx, flup, synnefo packages, etc.
.. _snf-asterias-architecture:
Architecture
------------
......@@ -112,11 +114,6 @@ queue.
.. _WEBAPP_NODE:
WEBAPP
******
A WEBAPP node runs the web application provided by the synnefo component
:ref:`snf-asterias-app <snf-asterias-app>`.
Installation
------------
......@@ -150,7 +147,7 @@ MySQL must be installed first:
.. code-block:: console
$ sudo apt-get install libmysqlclient-dev
# apt-get install libmysqlclient-dev
if you are using MacPorts:
......@@ -207,7 +204,7 @@ You need to install the PostgreSQL binaries, e.g., for Debian:
.. code-block:: console
$ sudo apt-get install postgresql-8.4 libpq-dev
# apt-get install postgresql-8.4 libpq-dev
or ir you are using MacPorts:
......@@ -351,16 +348,25 @@ and implementation of snf-image.
Please see `https://code.grnet.gr/projects/snf-image/files`
for the latest packages.
Images should be stored under extdump format in a directory
Images should be stored in ``extdump``, or ``diskdump`` format in a directory
of your choice, configurable as ``IMAGE_DIR`` in
:file:`/etc/default/snf-image`.
synnefo components
******************
You need to install the following synnefo components on each node,
depending on its type. Please see the page of each synnefo software
component for specific installation instructions, where applicable.
You need to install the appropriate synnefo software components on each node,
depending on its type, see :ref:`Architecture <snf-asterias-architecture>`.
Most synnefo components have dependencies on additional Python packages.
The dependencies are described inside each package, and are setup
automatically when installing using :command:`pip`, or when installing
using your system's package manager.
Please see the page of each synnefo software component for specific
installation instructions, where applicable.
Install the following synnefo components:
Nodes of type :ref:`APISERVER <APISERVER_NODE>`
Components
......@@ -370,480 +376,19 @@ Nodes of type :ref:`APISERVER <APISERVER_NODE>`
Nodes of type :ref:`GANETI-MASTER <GANETI_MASTER>` and :ref:`GANETI-NODE <GANETI_NODE>`
Components
:ref:`snf-common <snf-common>`,
:ref:`snf-ganeti-tools <snf-ganeti-tools>`
:ref:`snf-asterias-ganeti-tools <snf-asterias-ganeti-tools>`
Nodes of type :ref:`LOGIC <LOGIC_NODE>`
Components
:ref:`snf-common <snf-common>`,
:ref:`snf-webproject <snf-webproject>`,
:ref:`snf-asterias-app <snf-asterias-app>`.
.. todo:: describe prerequisites -- e.g., Debian
Configuration
-------------
asterias uses :ref:`snf-common <snf-common>` for settings.
Web admin
`````````
synnefo web administration interface. Allows administrator users to manage the
synnefo application via web interface.
Web application
```````````````
Web interface which allows users to create/configure/manage their virtual
machines.
.. _dispatcher-deploy:
Dispatcher
----------
The logic dispatcher is part of the synnefo Django project and must run
on :ref:`LOGIC <LOGIC_NODE>` nodes.
The dispatcher retrieves messages from the queue and calls the appropriate
handler function as defined in the queue configuration in :file:`/etc/synnefo/*.conf`
files.
The default configuration should work directly without any modifications.
For the time being The dispatcher must be run by hand::
$ snf-dispatcher
The dispatcher should run in at least 2 instances to ensure high
(actually, increased) availability.
.. _webapp-deploy:
Web application deployment
--------------------------
.. _static-files:
Static files
************
* Choose an appropriate path (e.g. :file:`/var/lib/synnefo/static/`) from which
your web server will serve all static files (js/css) required by the synnefo
web frontend to run.
* Change the ``MEDIA_ROOT`` value in your settings to point to that directory.
* Run the following command::
$ snf-manage link_static
the command will create symlinks of the appropriate static files inside the
chosen directory.
.. todo:: describe an ``snf-manage copy_static`` command.
Using Apache
************
.. todo:: document apache configuration
Using nginx
***********
This section describes a sample nginx configuration which uses FastCGI
to relay requests to synnefo. Use a distribution-specific mechanism
(e.g., APT) to install nginx, then activate the following nginx configuration
file by placing it under ``/etc/nginx/sites-available`` and symlinking
under ``/etc/nginx/sites-enabled``:
.. literalinclude:: ../_static/synnefo.nginx.conf
`download <../_static/synnefo.nginx.conf>`_
then run the FastCGI server to receive incoming requests from nginx.
This requires installation of package flup, e.g. with::
# apt-get install flup
$ snf-manage runfcgi host=127.0.0.1 port=8015
Console scripts
---------------
snf-manage
**************
snf-dispatcher
******************
snf-admin
*************
snf-cloud
*************
snf-burnin
**************
.. _installation:
.. _database-setup:
Installing depedencies
**********************
Synnefo is written in Python 2.6 requires the some additional python packages
to run properly.
The easiest method for installation of the Django project is to setup a
working environment through virtualenv. Alternatively, you can use your
system's package manager to install the dependencies (e.g. Macports has them
all).
You can install these packages either using `pip` python package manager::
$ pip install <pypi-package-name>==<version>
or using the requirements.pip file that exists in Synnefo package repository::
$ pip install -r requirements.pip
or Debian's `apt-get`::
$ apt-get install <debian-package-name>
Required packages
`````````````````
.. todo::
Confirm debian package names
======================= =================== ==========
PyPi package name Debian package name version
======================= =================== ==========
django python-django 1.2.4
simplejson python-simplejson 2.1.3
pycurl python-pycurl 7.19.0
python-dateutil python-dateutil 1.4.1
IPy python-ipy 0.75
south python-django-south 0.7.1
amqplib python-amqplib 0.6.1
lockfile python-lockfile 0.8
python-daemon python-daemon 1.5.5
python-prctl python-prctl 1.3.0
======================= =================== ==========
.. note::
On Snow Leopard and linux (64-bit), you have to set the following
environment variable for pip to compile the dependencies correctly::
$ export ARCHFLAGS="-arch x86_64"
.. note::
On Ubuntu/Debian, a few more packages must be installed before installing the
prerequisite Python libraries::
$ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
.. note::
Depending on the permissions of your system’s Python, you might need to be the
root user to install those packages system-wide
Database driver
```````````````
Depending on the database software you choose to use one of the following:
========= ======================= =================== ==========
Database PyPi package name Debian package name version
========= ======================= =================== ==========
mysql MySQL-python python-mysql 1.2.3
postgres psycopg2 python-psycopg2 2.4
========= ======================= =================== ==========
.. note::
The python sqlite driver is available by default with Python so no
additional configuration is required. Also, most self-respecting systems
have the sqlite library installed by default.
Extra depedencies
`````````````````
Synnefo provides some optional features that require specific python packages to
be installed.
**Invitations and SSH Keys generation**
======================= =================== ==========
PyPi package name Debian package name version
======================= =================== ==========
pycrypto python-crypto 2.1.0
======================= =================== ==========
Installing Synnefo package
--------------------------
Using ``pip``::
$ pip install https://code.grnet.gr/projects/synnefo/synnefo-<version>.tar.gz --no-deps
by checking out git repository::
$ git clone https://code.grnet.gr/git/synnefo synnefo-repo
$ cd synnefo-repo
$ python setup.py install
this should be enough for synnefo to get installed in your system-wide or
``virtualenv`` python installation and the following commands should be
available from the command line::
$ snf-manage
$ snf-dispatcher
$ snf-admin
Notice that Synnefo installation does not handle the creation of
``/etc/synnefo/`` directory which is the place where custom configuration
files are loaded from. You are encouraged to create this directory and place a
file named ``settings.conf`` with the following contents:
.. _sample-settings:
.. literalinclude:: ../_static/sample_settings.conf
:language: python
`download <../_static/sample_settings.conf>`_
this is just to get you started on how to configure your Synnefo installation.
From this point you can continue your read to the `Initial configuration`_ section
in this document which contains quickstart instructions for some of the initial
configuration required for Synnefo to get up and running.
For additional instructions about Synnefo settings files and what the available
settings are, you can refer to the :ref:`configuration <configuration>` guide.
Initial configuration
---------------------
Synnefo comes with most of the required settings predefined with values that
would cover many of the most common installation scenarios. However some basic
settings must be set be set before running Synnefo for the first time.
:ref:`sample settings file <sample-settings>`
Database
********
Change ``DATABASES`` setting based on your :ref:`database setup <database-setup>`
and :ref:`initialize/update your database structure <database-initialization>`
.. seealso::
:ref:`database-configuration` /
:ref:`database-initialization`
Queue
*****
Change ``RABBIT_*`` settings to match your :ref:`RabbitMQ setup <rabbitmq-setup>`.
Backend
*******
Set ``GANETI_NODES``, ``GANETI_MASTER_IP``, ``GANETI_CLUSTER_INFO`` based on your :ref:`Ganeti
installation <ganeti-setup>` and change BACKEND_PREFIX_ID using an custom `prefix
id`.
Web application
***************
See the extended :ref:`deployment guide <webapp-deploy>` for instructions on how to
setup the Synnefo web application.
.. _database-setup:
Installing depedencies
**********************
Synnefo is written in Python 2.6 requires the some additional python packages
to run properly.
The easiest method for installation of the Django project is to setup a
working environment through virtualenv. Alternatively, you can use your
system's package manager to install the dependencies (e.g. Macports has them
all).
You can install these packages either using `pip` python package manager::
$ pip install <pypi-package-name>==<version>
or using the requirements.pip file that exists in Synnefo package repository::
$ pip install -r requirements.pip
or Debian's `apt-get`::
$ apt-get install <debian-package-name>
Required packages
`````````````````
.. todo::
Confirm debian package names
======================= =================== ==========
PyPi package name Debian package name version
======================= =================== ==========
django python-django 1.2.4
simplejson python-simplejson 2.1.3
pycurl python-pycurl 7.19.0
python-dateutil python-dateutil 1.4.1
IPy python-ipy 0.75
south python-django-south 0.7.1
amqplib python-amqplib 0.6.1
lockfile python-lockfile 0.8
python-daemon python-daemon 1.5.5
python-prctl python-prctl 1.3.0
======================= =================== ==========
.. note::
On Snow Leopard and linux (64-bit), you have to set the following
environment variable for pip to compile the dependencies correctly::
$ export ARCHFLAGS="-arch x86_64"
.. note::
On Ubuntu/Debian, a few more packages must be installed before installing the
prerequisite Python libraries::
$ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
.. note::
Depending on the permissions of your system’s Python, you might need to be the
root user to install those packages system-wide
Database driver
```````````````
Depending on the database software you choose to use one of the following:
========= ======================= =================== ==========
Database PyPi package name Debian package name version
========= ======================= =================== ==========
mysql MySQL-python python-mysql 1.2.3
postgres psycopg2 python-psycopg2 2.4
========= ======================= =================== ==========
.. note::
The python sqlite driver is available by default with Python so no
additional configuration is required. Also, most self-respecting systems
have the sqlite library installed by default.
Extra depedencies
`````````````````
Synnefo provides some optional features that require specific python packages to
be installed.
**Invitations and SSH Keys generation**
======================= =================== ==========
PyPi package name Debian package name version
======================= =================== ==========
pycrypto python-crypto 2.1.0
======================= =================== ==========
Installing Synnefo package
--------------------------
Using ``pip``::
$ pip install https://code.grnet.gr/projects/synnefo/synnefo-<version>.tar.gz --no-deps
by checking out git repository::
$ git clone https://code.grnet.gr/git/synnefo synnefo-repo
$ cd synnefo-repo
$ python setup.py install
this should be enough for synnefo to get installed in your system-wide or
``virtualenv`` python installation and the following commands should be
available from the command line::
$ snf-manage
$ snf-dispatcher
$ snf-admin
Notice that Synnefo installation does not handle the creation of
``/etc/synnefo/`` directory which is the place where custom configuration
files are loaded from. You are encouraged to create this directory and place a
file named ``settings.conf`` with the following contents:
.. _sample-settings:
.. literalinclude:: ../_static/sample_settings.conf
:language: python
`download <../_static/sample_settings.conf>`_
this is just to get you started on how to configure your Synnefo installation.
From this point you can continue your read to the `Initial configuration`_ section
in this document which contains quickstart instructions for some of the initial
configuration required for Synnefo to get up and running.
For additional instructions about Synnefo settings files and what the available
settings are, you can refer to the :ref:`configuration <configuration>` guide.
Initial configuration
---------------------
Synnefo comes with most of the required settings predefined with values that
would cover many of the most common installation scenarios. However some basic
settings must be set be set before running Synnefo for the first time.
:ref:`sample settings file <sample-settings>`
Database
********
Change ``DATABASES`` setting based on your :ref:`database setup <database-setup>`
and :ref:`initialize/update your database structure <database-initialization>`
.. seealso::
:ref:`database-configuration` /
:ref:`database-initialization`
Queue
*****
Change ``RABBIT_*`` settings to match your :ref:`RabbitMQ setup <rabbitmq-setup>`.
Backend
*******
Set ``GANETI_NODES``, ``GANETI_MASTER_IP``, ``GANETI_CLUSTER_INFO`` based on your :ref:`Ganeti
installation <ganeti-setup>` and change BACKEND_PREFIX_ID using an custom `prefix
id`.
Web application
***************
See the extended :ref:`deployment guide <webapp-deploy>` for instructions on how to
setup the Synnefo web application.
Please refer to the configuration sections of
:ref:`snf-webproject <snf-webproject>`,
:ref:`snf-asterias-app <snf-asterias-app>`,
:ref:`snf-asterias-ganeti-tools <snf-asterias-ganeti-tools>` for more
information on their configuration.
snf-app/docs/src/develop.rst → snf-app/docs/src/asterias-dev-guide.rst
View file @ fcda074a
Developers guide
================
.. _snf-asterias-developer-guide:
Information on how to setup a development environment.
===============
Developer Guide
===============
This file documents the installation of a development environment for Synnefo.
It should be read alongside the :ref:`installation guide <installation>`.
It contains development-specific ammendments to the basic installation steps
outlined in `installation guide <installation>`, and development-specific notes.
Prerequisites
-------------
For a basic development environment you need to follow steps
of `installation guide <installation>`, which should be read
in its entirety *before* this document.
This is the asterias developer guide.
It is intended for developers, wishing to implement new functionality
inside :ref:`asterias <snf-asterias>`.
Setting up development environment
----------------------------------
It assumes thorough familiarity with the :ref:`snf-asterias-admin-guide`.
Although not necessary it is suggested that you use a virtualenv as a base for
your development environment::
$ virtualenv synnefo-env
$ cd synnefo-dev
$ source bin/activate
* Clone Synnefo repository::
(synnefo-env)$ git clone https://code.grnet.gr/git/synnefo synnefo
It contains development-specific ammendments to the basic installation steps
outlined in `installation guide <installation>`, and development-specific
notes.
* Install synnefo depedencies::
Building a dev environment
--------------------------
(synnefo-env)$ pip install -r synnefo/requirements.pip
virtualenv
**********
* Install development helpers::
The easiest method to deploy a development environment is using
:command:`virtualenv`. Alternatively, you can use your system's package manager
to install any dependencies of synnefo components (e.g. Macports has them all).
(synnefo-env)$ pip install django_extensions
.. code-block:: console
$ virtualenv ~/synnefo-env
$ . ~/synnefo-env/bin/activate
(synnefo-env)$
Any :command:`pip` commands executed from now on, affect the ``synnefo-env``
virtualenv.
* Install synnefo package::
* It is also recommended to install development helpers:
(synnefo-env)$ cd synnefo/snf-app
(synnefo-env)$ python setup.py develop -N
.. code-block:: console
(synnefo-env)$ pip install django_extensions
* Create a custom development settings module and set environment variable to
use this module as the entry point of synnefo settings::
* Create a custom settings directory for :ref:`snf-common <snf-common>` and set
the ``SYNNEFO_SETTINGS_DIR`` environment variable to use development-specific
file:`*.conf` files inside this directory.
(synnefo-env)$ touch settings_dev.py
(synnefo-env)$ export DJANGO_SETTINGS_MODULE=settings_dev
(synnefo-env)$ export PYTHONPATH=`pwd`:$PYTHONPATH
(synnefo-env)$ vi settings_dev.py
(synnefo-env)$ mkdir ~/synnefo-settings-dir