Commit 29aeafad authored by Vangelis Koukis's avatar Vangelis Koukis
Browse files

Improvements to documentation

Changelogs for compute service,
list synnefo software components,
add Implementation Guide for Plankton.
parent d3744db6
Changelog
=========
---------
2011-11-29, v0.7.4
******************
FIXES:
OKEANOS_INTRO:
......@@ -10,7 +10,7 @@ FIXES:
2011-10-21, v0.7.3
------------------
******************
FIXES:
UI:
......@@ -18,7 +18,7 @@ FIXES:
2011-10-19, v0.7.2
------------------
******************
FIXES:
UI:
......@@ -27,7 +27,7 @@ FIXES:
2011-10-17, v0.7.1
------------------
******************
FIXES:
UI:
......@@ -44,7 +44,7 @@ FIXES:
2011-10-13, v0.7
----------------
****************
NEW FEATURES:
UI:
......@@ -113,7 +113,7 @@ FIXES:
2011-09-15, v0.6.2
------------------
******************
FIXES:
UI:
......@@ -140,7 +140,7 @@ NEW FEATURES:
2011-09-13, v0.6.1
------------------
******************
FIXES:
UI:
......@@ -148,7 +148,7 @@ FIXES:
2011-09-12, v0.6
----------------
****************
NEW FEATURES:
Admin:
......@@ -192,7 +192,7 @@ FIXES:
2011-08-29, v0.5.5
------------------
******************
FIXES:
Logic:
......@@ -201,7 +201,7 @@ FIXES:
2011-07-27, v0.5.4
------------------
******************
FIXES:
UI:
......@@ -239,7 +239,7 @@ FEATURES:
2011-07-19, v0.5.3.1
--------------------
******************--
FIXES:
API:
......@@ -252,7 +252,7 @@ FIXES:
2011-07-19, v0.5.3
------------------
******************
FIXES:
GUI:
......@@ -265,7 +265,7 @@ FIXES:
2011-07-18, v0.5.2
------------------
******************
FIXES:
GUI:
......@@ -280,7 +280,7 @@ FIXES:
2011-07-14, v0.5.1
------------------
******************
FIXES:
GUI:
......@@ -325,7 +325,7 @@ NEW FEATURES
2011-07-01, v0.5
----------------
****************
NEW FEATURES
GUI:
......@@ -378,7 +378,7 @@ FIXES:
2011-06-06, v0.4
----------------
****************
NEW FEATURES:
GUI:
......@@ -463,7 +463,7 @@ KNOWN DEFECTS:
2011-05-10, v0.3
----------------
****************
FIXES/NEW FEATURES:
......@@ -505,7 +505,7 @@ KNOWN DEFECTS:
2011-04-19, v0.2.2
------------------
******************
Bug fix release:
GUI:
......@@ -513,7 +513,7 @@ Bug fix release:
2011-04-19, v0.2.1
------------------
******************
Bug fix release:
GUI:
......@@ -521,7 +521,7 @@ Bug fix release:
2011-04-19, v0.2
----------------
****************
FIXES/NEW FEATURES:
......
......@@ -12,22 +12,25 @@ This is the main synnefo documentation page.
synnefo comprises the following major components:
.. todo:: turn all of the following to links to the documentation, with
with intersphinx links.
intersphinx links.
.. toctree::
:maxdepth: 1
asterias (name TBD): Compute Service <src/asterias.rst>
pithos+: File storage service <http://docs.dev.grnet.gr/pithos>
plankton: Image registry <src/snf-plankton.rst>
plankton: Image registry <src/plankton.rst>
archipelagos: Volume storage service <http://docs.dev.grnet.gr/archipelagos>
astakos: Identity management module <http://docs.dev.grnet.gr/astakos>
aquarium: Billing module <http://docs.dev.grnet.gr/aquarium>
image: Secure image deployment tool <http://docs.dev.grnet.gr/snf-image>
kamaki: Command-line cloud management tool <http://docs.dev.grnet.gr/kamaki>
You may also see the detailed configuration for all
:ref:`synnefo software components <components>`.
Indices and tables
==================
------------------
* :ref:`genindex`
......
......@@ -29,11 +29,9 @@ Installation
Configuration
Configuration of the various software components comprising an asterias
deployment.
Upgrades
Changelogs
.. todo:: describe prerequisites -- e.g., Debian
.. todo:: describe setup of nginx, flup, synnefo packages, etc.
Upgrades/Changelogs
Upgrades of existing deployments of asterias to newer versions, associated
Changelogs.
.. _asterias-architecture:
......@@ -110,7 +108,7 @@ queue.
* the synnefo Ganeti hook, ``ganeti/snf-ganeti-hook.py``.
* on every :ref:`GANETI-NODE <GANETI_NODE>`:
* a deployment-specific KVM ifup script
* properly configured :ref:`NFDHCPD <nfdhcpd-setup>`
* properly configured :ref:`NFDHCPD <asterias-nfdhcpd-setup>`
.. _WEBAPP_NODE:
......@@ -124,122 +122,28 @@ Installation of asterias is a two step process:
Prerequisites
*************
.. _ganeti-setup:
.. _asterias-install-ganeti:
1. Ganeti installation
``````````````````````
Ganeti installation
```````````````````
Synnefo requires a working Ganeti installation at the backend. Installation
of Ganeti is not covered by this document, please refer to
`ganeti documentation <http://docs.ganeti.org/ganeti/current/html>`_ for all the
gory details. A successful Ganeti installation concludes with a working
:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs <GANETI_NODES>`.
2. Database
```````````
SQLite
~~~~~~
Most self-respecting systems have ``sqlite`` installed by default.
MySQL
~~~~~
MySQL must be installed first:
.. code-block:: console
# apt-get install libmysqlclient-dev
if you are using MacPorts:
.. code-block:: console
$ sudo port install mysql5
.. note::
On MacOSX with Mysql install from MacPorts the above command will
fail complaining that it cannot find the mysql_config command. Do
the following and restart the installation:
.. code-block:: console
$ echo "mysql_config = /opt/local/bin/mysql_config5" >> ./build/MySQL-python/site.cfg
Configure a MySQL db/account for the Django project:
.. code-block:: console
$ mysql -u root -p;
.. code-block:: mysql
CREATE DATABASE <database name>;
SHOW DATABASES;
GRANT ALL ON <database name>.* TO <db username> IDENTIFIED BY '<db password>';
.. warning::
MySQL *must* be set in ``READ-COMMITED`` mode, e.g. by setting:
.. code-block:: ini
transaction-isolation = READ-COMMITTED
in the ``[mysqld]`` section of :file:`/etc/mysql/my.cnf`.
Alternatively, make sure the following code fragment stays enabled
in :file:`/etc/synnefo/10-database.conf` file:
.. code-block:: python
if DATABASES['default']['ENGINE'].endswith('mysql'):
DATABASES['default']['OPTIONS'] = {
'init_command': 'SET storage_engine=INNODB; ' +
'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
}
PostgreSQL
~~~~~~~~~~
You need to install the PostgreSQL binaries, e.g., for Debian:
.. code-block:: console
# apt-get install postgresql-8.4 libpq-dev
or ir you are using MacPorts:
.. code-block:: console
$ sudo port install postgresql84
.. _asterias-install-db:
To configure a postgres db/account for synnefo,
Database
````````
* Become the postgres user, connect to PostgreSQL:
Database installation is done as part of the
:ref:`snf-webproject <snf-webproject>` component.
.. code-block:: console
.. _asterias-install-rabbitmq:
$ sudo su - postgres
$ psql
* Run the following commands:
.. code-block:: sql
DROP DATABASE <database name>;
DROP USER <db username>;
CREATE USER <db username> WITH PASSWORD '<db password>';
CREATE DATABASE <database name>;
GRANT ALL PRIVILEGES ON DATABASE <database name> TO <db username>;
ALTER DATABASE <database name> OWNER TO <db username>;
ALTER USER <db username> CREATEDB;
.. note::
The last line enables the newly created user to create own databases. This
is needed for Django to create and drop the ``test_synnefo`` database for
unit testing.
3. RabbitMQ
```````````
RabbitMQ
````````
RabbitMQ is used as a generic message broker for asterias. It should be
installed on two seperate :ref:`QUEUE <QUEUE_NODE>` nodes in a high availability
......@@ -261,8 +165,10 @@ The values set for the user and password must be mirrored in the
.. todo:: Document an active-active configuration based on the latest version
of RabbitMQ.
4. vncauthproxy
```````````````
.. _asterias-install-vncauthproxy:
vncauthproxy
````````````
To support OOB console access to the VMs over VNC, the vncauthproxy
daemon must be running on every :ref:`APISERVER <APISERVER_NODE>` node.
......@@ -306,10 +212,10 @@ Alternatively, build and install Debian packages.
.. todo:: Mention vncauthproxy bug, snf-vncauthproxy, inability to install using pip
.. todo:: kpap: fix installation commands
.. _nfdhcpd-setup:
.. _asterias-install-nfdhcpd:
5. NFDHCPD
``````````
NFDHCPD
```````
Setup Synnefo-specific networking on the Ganeti backend.
This part is deployment-specific and must be customized based on the
......@@ -330,10 +236,10 @@ to NFDHCPD's state directory, usually ``/var/lib/nfdhcpd``.
.. todo:: soc: document NFDHCPD installation, settle on KVM ifup script
.. _rabbitmq-setup:
.. _asterias-install-snfimage:
6. snf-image
````````````
snf-image
`````````
Install the :ref:`snf-image <snf-image>` Ganeti OS provider for image
deployment.
......@@ -386,9 +292,63 @@ Nodes of type :ref:`LOGIC <LOGIC_NODE>`
Configuration
-------------
This section targets the configuration of the prerequisites for asterias,
and the configuration of the associated synnefo software components.
synnefo components
******************
asterias uses :ref:`snf-common <snf-common>` for settings.
Please refer to the configuration sections of
:ref:`snf-webproject <snf-webproject>`,
:ref:`snf-asterias-app <snf-asterias-app>`,
:ref:`snf-asterias-ganeti-tools <snf-asterias-ganeti-tools>` for more
information on their configuration.
Ganeti
``````
Set ``GANETI_NODES``, ``GANETI_MASTER_IP``, ``GANETI_CLUSTER_INFO`` based on
your :ref:`Ganeti installation <asterias-install-ganeti>` and change the
`BACKEND_PREFIX_ID`` setting, using an custom ``PREFIX_ID``.
Database
````````
Once all components are installed and configured,
initialize the Django DB:
.. code-block:: console
$ snf-manage syncdb
$ snf-manage migrate
and load fixtures ``{users, flavors, images}``,
which make the API usable by end users by defining a sample set of users,
hardware configurations (flavors) and OS images:
.. code-block:: console
$ snf-manage loaddata /path/to/users.json
$ snf-manage loaddata flavors
$ snf-manage loaddata images
.. warning::
Be sure to load a custom users.json and select a unique token
for each of the initial and any other users defined in this file.
**DO NOT LEAVE THE SAMPLE AUTHENTICATION TOKENS** enabled in deployed
configurations.
sample users.json file:
.. literalinclude:: ../../synnefo/db/fixtures/users.json
`download <../_static/users.json>`_
RabbitMQ
````````
Change ``RABBIT_*`` settings to match your :ref:`RabbitMQ setup
<asterias-install-rabbitmq>`.
.. include:: ../../Changelog
......@@ -11,10 +11,6 @@ inside :ref:`asterias <asterias>`.
It assumes thorough familiarity with the :ref:`asterias-admin-guide`.
It contains development-specific ammendments to the basic installation steps
outlined in `installation guide <installation>`, and development-specific
notes.
Building a dev environment
--------------------------
......@@ -296,6 +292,7 @@ must write the script manually. The process is the following:
Test coverage
-------------
.. warning: This section may be out of date.
In order to get code coverage reports you need to install django-test-coverage
......@@ -314,6 +311,7 @@ Then configure the test runner inside Django settings:
Building Synnefo package
------------------------
.. warning: This section may be out of date.
To create a python package from the Synnefo source code run
......@@ -343,4 +341,5 @@ Make sure you have ``sphinx`` installed.
html files are generated in the ``snf-app/docs/_build/html`` directory.
.. warning: This section may be out of date.
.. include:: ci.rst
.. include:: ../../Changelog
.. _components:
===========================
Synnefo software components
===========================
Synnefo comprises a number of software components.
.. toctree::
:maxdepth: 1
snf-common
snf-webproject
snf-asterias-app
snf-asterias-ganeti-tools
snf-pithos-app
snf-pithos-backends
snf-astakos-app
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. _configuration:
Configuration
=============
.. _settings-guide:
Customizing Synnefo settings
----------------------------
To ease up the configuration of the application Synnefo includes settings
defined in ``/etc/synnefo/*.conf`` files. The location can be altered by
setting an enviromental variable named ``SYNNEFO_SETTINGS_DIR`` to the
appropriate path, or by using the ``--settings-dir`` option of the
``snf-manage`` tool.
Synnefo package bundles a `Django` project with predefined common `settings`
and `urls` set. The corresponding ``manage.py`` for the bundled project is
``snf-manage``. After the package installation the tool should be available
as a command from your system's terminal. Due to this nature of `snf-manage`
it is possible to alter settings not only using ``.conf`` files but also by
providing a custom python module by using ``DJANGO_SETTINGS_MODULE``
evnironmental variable or ``--settings`` option of the tool.
.. seealso::
https://docs.djangoproject.com/en/dev/topics/settings/
If you are using a custom settings module, you are strongly encouraged to
import the synnefo default settings prior to your customized ones e.g. :
.. code-block:: python
from synnefo.settings import *
CUSTOM_SETTING1 = "...."
CUSTOM_SETTING2 = "...."
.. _database-configuration:
Database configuration
----------------------
Add the following to your custom :ref:`settings <settings-guide>`, depending on your choice
of DB:
SQLite
******
.. code-block:: python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/path/to/synnefo.db'
}
}
.. warning:: `NAME` must be an absolute path to the sqlite3 database file
MySQL
*****
.. code-block:: python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'synnefo',
'USER': 'USERNAME',
'PASSWORD': 'PASSWORD',
'HOST': 'HOST',
'PORT': 'PORT',
'OPTIONS': {
'init_command': 'SET storage_engine=INNODB',
}
}
}
PostgreSQL
**********
.. code-block:: python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'DATABASE',
'USER': 'USERNAME',
'PASSWORD': 'PASSWORD',
'HOST': 'HOST',
'PORT': 'PORT',
}
}
Try it out. The following command will attempt to connect to the DB and
print out DDL statements. It should not fail::
$ snf-manage sql db
.. _database-initialization:
Database initialization
-----------------------
You need to initialize the Synnefo DB::
$ snf-manage syncdb
$ snf-manage migrate
and load fixtures {users,flavors,images},
which make the API usable by end users by defining a sample set of users,
hardware configurations (flavors) and OS images::
$ snf-manage loaddata /path/to/users.json
$ snf-manage loaddata flavors
$ snf-manage loaddata images
.. warning::
Be sure to load a custom users.json and select a unique token
for each of the initial and any other users defined in this file.
**DO NOT LEAVE THE SAMPLE AUTHENTICATION TOKENS** enabled in deployed
configurations.
sample users.json file:
.. literalinclude:: ../../synnefo/db/fixtures/users.json
`download <../_static/users.json>`_
.. _additional-configuration:
.. include settings reference
.. include: settings.rst
Intro
=====
.. todo:: write introductory documentation