install-guide-debian.rst

.. _install-guide-debian:

Administrator's Installation Guide (on Debian Wheezy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is the Administrator's installation guide on Debian Wheezy.

It describes how to install the whole Synnefo stack on two (2) physical nodes,
with minimum configuration. It installs Synnefo from Debian packages, and
assumes the nodes run Debian Wheezy. After successful installation, you will
have the following services running:

    * Identity Management (Astakos)
    * Object Storage Service (Pithos)
    * Compute Service (Cyclades)
    * Image Service (part of Cyclades)
    * Network Service (part of Cyclades)

and a single unified Web UI to manage them all.

If you just want to install the Object Storage Service (Pithos), follow the
guide and just stop after the "Testing of Pithos" section.


Installation of Synnefo / Introduction
======================================

For the rest of the documentation we will refer to the first physical node as
"node1" and the second as "node2". We will also assume that their domain names
are "node1.example.com" and "node2.example.com" and their public IPs are "203.0.113.1" and
"203.0.113.2" respectively. It is important that the two machines are under the same domain name.
In case you choose to follow a private installation you will need to
set up a private dns server, using dnsmasq for example. See node1 below for
more information on how to do so.

General Prerequisites
=====================

These are the general Synnefo prerequisites, that you need on node1 and node2
and are related to all the services (Astakos, Pithos, Cyclades).

To be able to download all Synnefo components you need to add the following
lines in your ``/etc/apt/sources.list`` file:

| ``deb http://apt.dev.grnet.gr wheezy/``
| ``deb-src http://apt.dev.grnet.gr wheezy/``

and import the repo's GPG key:

| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``

Update your list of packages and continue with the installation:

.. code-block:: console

   # apt-get update

You also need a shared directory visible by both nodes. Pithos will save all
data inside this directory. By 'all data', we mean files, images, and Pithos
specific mapping data. If you plan to upload more than one basic image, this
directory should have at least 50GB of free space. During this guide, we will
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
to node2 (be sure to set no_root_squash flag). Node2 has this directory
mounted under ``/srv/pithos``, too.

Before starting the Synnefo installation, you will need basic third party
software to be installed and configured on the physical nodes. We will describe
each node's general prerequisites separately. Any additional configuration,
specific to a Synnefo service for each node, will be described at the service's
section.

Finally, it is required for Cyclades and Ganeti nodes to have synchronized
system clocks (e.g. by running ntpd).

Node1
-----


General Synnefo dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

		* apache (http server)
		* public certificate
		* gunicorn (WSGI http server)
		* postgresql (database)
		* rabbitmq (message queue)
		* ntp (NTP daemon)
		* gevent
		* dnsmasq (DNS server)
		* Archipelago

You can install apache2, postgresql, ntp and rabbitmq by running:

.. code-block:: console

   # apt-get install apache2 postgresql ntp rabbitmq-server

To install gunicorn and gevent, run:

.. code-block:: console

   # apt-get install gunicorn python-gevent

On node1, we will create our databases, so you will also need the
python-psycopg2 package:

.. code-block:: console

   # apt-get install python-psycopg2

Database setup
~~~~~~~~~~~~~~

On node1, we create a database called ``snf_apps``, that will host all django
apps related tables. We also create the user ``synnefo`` and grant him all
privileges on the database. We do this by running:

.. code-block:: console

    root@node1:~ # su - postgres
    postgres@node1:~ $ psql
    postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
    postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;

We also create the database ``snf_pithos`` needed by the Pithos backend and
grant the ``synnefo`` user all privileges on the database. This database could
be created on node2 instead, but we do it on node1 for simplicity. We will
create all needed databases on node1 and then node2 will connect to them.

.. code-block:: console

    postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;

Configure the database to listen to all network interfaces. You can do this by
editing the file ``/etc/postgresql/9.1/main/postgresql.conf`` and change
``listen_addresses`` to ``'*'`` :

.. code-block:: console

    listen_addresses = '*'

Furthermore, edit ``/etc/postgresql/9.1/main/pg_hba.conf`` to allow node1 and
node2 to connect to the database. Add the following lines under ``#IPv4 local
connections:`` :

.. code-block:: console

    host		all	all	203.0.113.1/32	md5
    host		all	all	203.0.113.2/32	md5

Make sure to substitute "203.0.113.1" and "203.0.113.2" with node1's and node2's
actual IPs. Now, restart the server to apply the changes:

.. code-block:: console

   # /etc/init.d/postgresql restart


Certificate Creation
~~~~~~~~~~~~~~~~~~~~~

Node1 will host Cyclades. Cyclades should communicate with the other Synnefo
Services and users over a secure channel. In order for the connection to be
trusted, the keys provided to Apache below should be signed with a certificate.
This certificate should be added to all nodes. In case you don't have signed keys you can create a self-signed certificate
and sign your keys with this. To do so on node1 run:

.. code-block:: console

		# apt-get install openvpn
		# mkdir /etc/openvpn/easy-rsa
		# cp -ai /usr/share/doc/openvpn/examples/easy-rsa/2.0/ /etc/openvpn/easy-rsa
		# cd /etc/openvpn/easy-rsa/2.0
		# vim vars

In vars you can set your own parameters such as KEY_COUNTRY

.. code-block:: console

	# . ./vars
	# ./clean-all

Now you can create the certificate

.. code-block:: console

		# ./build-ca

The previous will create a ``ca.crt`` file in the directory ``/etc/openvpn/easy-rsa/2.0/keys``.
Copy this file under ``/usr/local/share/ca-certificates/`` directory and run :

.. code-block:: console

		# update-ca-certificates

to update the records. You will have to do the following on node2 as well.

Now you can create the keys and sign them with the certificate

.. code-block:: console

		# ./build-key-server node1.example.com

This will create a ``01.pem`` and a ``node1.example.com.key`` files in the
``/etc/openvpn/easy-rsa/2.0/keys`` directory. Copy these in ``/etc/ssl/certs/``
and ``/etc/ssl/private/`` respectively and use them in the apache2
configuration file below instead of the defaults.

Apache2 setup
~~~~~~~~~~~~~

Create the file ``/etc/apache2/sites-available/synnefo`` containing the
following:

.. code-block:: console

    <VirtualHost *:80>
        ServerName node1.example.com

        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    </VirtualHost>


Create the file ``/etc/apache2/sites-available/synnefo-ssl`` containing the
following:

.. code-block:: console

    <IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName node1.example.com

        Alias /static "/usr/share/synnefo/static"

        #  SetEnv no-gzip
        #  SetEnv dont-vary

       AllowEncodedSlashes On

       RequestHeader set X-Forwarded-Protocol "https"

    <Proxy * >
        Order allow,deny
        Allow from all
    </Proxy>

        SetEnv                proxy-sendchunked
        SSLProxyEngine        off
        ProxyErrorOverride    off

        ProxyPass        /static !
        ProxyPass        / http://localhost:8080/ retry=0
        ProxyPassReverse / http://localhost:8080/

        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]

        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
    </VirtualHost>
    </IfModule>

Now enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl
   # a2enmod rewrite
   # a2dissite default
   # a2ensite synnefo
   # a2ensite synnefo-ssl
   # a2enmod headers
   # a2enmod proxy_http

.. note:: This isn't really needed, but it's a good security practice to disable
    directory listing in apache::

        # a2dismod autoindex


.. warning:: Do NOT start/restart the server yet. If the server is running::

       # /etc/init.d/apache2 stop


.. _rabbitmq-setup:

Message Queue setup
~~~~~~~~~~~~~~~~~~~

The message queue will run on node1, so we need to create the appropriate
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
exchanges:

.. code-block:: console

   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"

We do not need to initialize the exchanges. This will be done automatically,
during the Cyclades setup.

Pithos data directory setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~

As mentioned in the General Prerequisites section, there should be a directory
called ``/srv/pithos`` visible by both nodes. We create and setup the ``data``
directory inside it along with the ``maps`` and ``blocks`` subdirectories:

.. code-block:: console

   # mkdir /srv/pithos
   # cd /srv/pithos
   # mkdir data
   # mkdir -p data/{maps,blocks}

This directory must be shared via `NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
In order to do this, run:

.. code-block:: console

   # apt-get install rpcbind nfs-kernel-server

Now edit ``/etc/exports`` and add the following line:

.. code-block:: console

   /srv/pithos/ 203.0.113.2(rw,no_root_squash,sync,subtree_check)

Once done, run:

.. code-block:: console

   # /etc/init.d/nfs-kernel-server restart

Archipelago setup
~~~~~~~~~~~~~~~~~

To install Archipelago, run:

.. code-block:: console

   # apt-get install archipelago


Now edit ``/etc/archipelago/archipelago.conf`` and tweak the following settings:

* ``SEGMENT_SIZE``: Adjust shared memory segment size according to your machine's
  RAM. The default value is 2GB which in some situations might exceed your
  machine's physical RAM.

In section ``blockerb`` set:

* ``archip_dir``: ``/srv/pithos/data/blocks``

In section ``blockerm`` set:

* ``archip_dir``: ``/srv/pithos/data/maps``

Finally, restart Archipelago:

.. code-block:: console

   # /etc/init.d/archipelago restart

DNS server setup
~~~~~~~~~~~~~~~~

If your machines are not under the same domain name you have to set up a dns server.
In order to set up a dns server using dnsmasq do the following:

.. code-block:: console

   # apt-get install dnsmasq

Then edit your ``/etc/hosts/`` file as follows:

.. code-block:: console

		203.0.113.1     node1.example.com
		203.0.113.2     node2.example.com

dnsmasq will serve any IPs/domains found in ``/etc/hosts``.

There is a `"bug" in libevent 2.0.5 <http://sourceforge.net/p/levent/bugs/193/>`_
, where if you have multiple nameservers in your ``/etc/resolv.conf``, libevent
will round-robin against them. To avoid this, you must use a single nameserver
for all your needs. Edit your ``/etc/resolv.conf`` to include your dns server:

.. code-block:: console

   nameserver 203.0.113.1

Because of the aforementioned bug, you can't specify more than one DNS servers
in your ``/etc/resolv.conf``. In order for dnsmasq to serve domains not in
``/etc/hosts``, edit ``/etc/dnsmasq.conf`` and change the line starting with
``#resolv-file=`` to:

.. code-block:: console

   resolv-file=/etc/external-dns

Now create the file ``/etc/external-dns`` and specify any extra DNS servers you
want dnsmasq to query for domains, e.g., 8.8.8.8:

.. code-block:: console

   nameserver 8.8.8.8

In the ``/etc/dnsmasq.conf`` file, you can also specify the ``listen-address``
and the ``interface`` you would like dnsmasq to listen to.

Finally, restart dnsmasq:

.. code-block:: console

   # /etc/init.d/dnsmasq restart

You are now ready with all general prerequisites concerning node1. Let's go to
node2.

Node2
-----

General Synnefo dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    * apache (http server)
    * gunicorn (WSGI http server)
    * postgresql (database)
    * ntp (NTP daemon)
    * gevent
    * certificates
    * dnsmasq (DNS server)
    * Archipelago

You can install the above by running:

.. code-block:: console

   # apt-get install apache2 postgresql ntp

To install gunicorn and gevent, run:

.. code-block:: console

   # apt-get install gunicorn python-gevent

Node2 will connect to the databases on node1, so you will also need the
python-psycopg2 package:

.. code-block:: console

   # apt-get install python-psycopg2

Database setup
~~~~~~~~~~~~~~

All databases have been created and setup on node1, so we do not need to take
any action here. From node2, we will just connect to them. When you get familiar
with the software you may choose to run different databases on different nodes,
for performance/scalability/redundancy reasons, but those kind of setups are out
of the purpose of this guide.

Apache2 setup
~~~~~~~~~~~~~

Create the file ``/etc/apache2/sites-available/synnefo`` containing the
following:

.. code-block:: console

    <VirtualHost *:80>
        ServerName node2.example.com

        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    </VirtualHost>

Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
containing the following:

.. code-block:: console

    <IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName node2.example.com

        Alias /static "/usr/share/synnefo/static"

        SetEnv no-gzip
        SetEnv dont-vary
        AllowEncodedSlashes On

        RequestHeader set X-Forwarded-Protocol "https"

        <Proxy * >
            Order allow,deny
            Allow from all
        </Proxy>

        SetEnv                proxy-sendchunked
        SSLProxyEngine        off
        ProxyErrorOverride    off

        ProxyPass        /static !
        ProxyPass        / http://localhost:8080/ retry=0
        ProxyPassReverse / http://localhost:8080/

        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
    </VirtualHost>
    </IfModule>

As in node1, enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl
   # a2enmod rewrite
   # a2dissite default
   # a2ensite synnefo
   # a2ensite synnefo-ssl
   # a2enmod headers
   # a2enmod proxy_http

.. note:: This isn't really needed, but it's a good security practice to disable
    directory listing in apache::

        # a2dismod autoindex

.. warning:: Do NOT start/restart the server yet. If the server is running::

       # /etc/init.d/apache2 stop


Acquire certificate
~~~~~~~~~~~~~~~~~~~

Copy the certificate you created before on node1 (`ca.crt`) under the directory
``/usr/local/share/ca-certificate`` and run:

.. code-block:: console

   # update-ca-certificates

to update the records.


DNS Setup
~~~~~~~~~

Add the following line in ``/etc/resolv.conf`` file

.. code-block:: console

   nameserver 203.0.113.1

to inform the node about the new DNS server.

As mentioned before, this should be the only ``nameserver`` entry in
``/etc/resolv.conf``.

We are now ready with all general prerequisites for node2. Now that we have
finished with all general prerequisites for both nodes, we can start installing
the services. First, let's install Astakos on node1.

Installation of Astakos on node1
================================

To install Astakos, grab the package from our repository (make sure  you made
the additions needed in your ``/etc/apt/sources.list`` file and updated, as
described previously), by running:

.. code-block:: console

   # apt-get install snf-astakos-app snf-pithos-backend

.. _conf-astakos:

Configuration of Astakos
========================

Gunicorn setup
--------------

Copy the file ``/etc/gunicorn.d/synnefo.example`` to
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:

.. code-block:: console

    # cp /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo


.. warning:: Do NOT start the server yet, because it won't find the
    ``synnefo.settings`` module. Also, change ``--worker-class=gevent`` to
    ``--worker-class=pithos.workers.gevent_archipelago.GeventArchipelagoWorker``
    and set ``--config=/etc/synnefo/pithos.conf.py``.
    We will start the server after successful installation of Astakos.
    If the server is running::

       # /etc/init.d/gunicorn stop

Conf Files
----------

After Astakos is successfully installed, you will find the directory
``/etc/synnefo`` and some configuration files inside it. The files contain
commented configuration options, which are the default options. While installing
new snf-* components, new configuration files will appear inside the directory.
In this guide (and for all services), we will edit only the minimum necessary
configuration options, to reflect our setup. Everything else will remain as is.

After getting familiar with Synnefo, you will be able to customize the software
as you wish and fits your needs. Many options are available, to empower the
administrator with extensively customizable setups.

For the snf-webproject component (installed as an Astakos dependency), we
need the following:

Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
uncomment and edit the ``DATABASES`` block to reflect our database:

.. code-block:: console

    DATABASES = {
     'default': {
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
         # ATTENTION: This *must* be the absolute path if using sqlite3.
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
         'NAME': 'snf_apps',
         'USER': 'synnefo',                      # Not used with sqlite3.
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
         # Set to empty string for localhost. Not used with sqlite3.
         'HOST': '203.0.113.1',
         # Set to empty string for default. Not used with sqlite3.
         'PORT': '5432',
     }
    }

Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
``SECRET_KEY``. This is a Django specific setting which is used to provide a
seed in secret-key hashing algorithms. Set this to a random string of your
choice and keep it private:

.. code-block:: console

    SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'

For Astakos specific configuration, edit the following options in
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :

.. code-block:: console

    ASTAKOS_COOKIE_DOMAIN = '.example.com'

    ASTAKOS_BASE_URL = 'https://node1.example.com/astakos'

The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all
services). ``ASTAKOS_BASE_URL`` is the Astakos top-level URL. Appending an
extra path (``/astakos`` here) is recommended in order to distinguish
components, if more than one are installed on the same machine.

.. note:: For the purpose of this guide, we don't enable recaptcha authentication.
    If you would like to enable it, you have to edit the following options:

    .. code-block:: console

        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
        ASTAKOS_RECAPTCHA_USE_SSL = True
        ASTAKOS_RECAPTCHA_ENABLED = True

    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
    go to https://www.google.com/recaptcha/admin/create and create your own pair.

Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :

.. code-block:: console

    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'

    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/astakos/ui/get_services'

    CLOUDBAR_MENU_URL = 'https://node1.example.com/astakos/ui/get_menu'

Those settings have to do with the black cloudbar endpoints and will be
described in more detail later on in this guide. For now, just edit the domain
to point at node1 which is where we have installed Astakos.

If you are an advanced user and want to use the Shibboleth Authentication
method, read the relative :ref:`section <shibboleth-auth>`.

.. _email-configuration:

Email delivery configuration
----------------------------

Many of the ``Astakos`` operations require the server to notify service users
and administrators via email. e.g. right after the signup process, the service
sents an email to the registered email address containing an verification url.
After the user verifies the email address, Astakos once again needs to
notify administrators with a notice that a new account has just been verified.

More specifically Astakos sends emails in the following cases

- An email containing a verification link after each signup process.
- An email to the people listed in ``ADMINS`` setting after each email
  verification if ``ASTAKOS_MODERATION`` setting is ``True``. The email
  notifies administrators that an additional action is required in order to
  activate the user.
- A welcome email to the user email and an admin notification to ``ADMINS``
  right after each account activation.
- Feedback messages submited from Astakos contact view and Astakos feedback
  API endpoint are sent to contacts listed in ``HELPDESK`` setting.
- Project application request notifications to people included in ``HELPDESK``
  and ``MANAGERS`` settings.
- Notifications after each project members action (join request, membership
  accepted/declinde etc.) to project members or project owners.

Astakos uses the Django internal email delivering mechanism to send email
notifications. A simple configuration, using an external smtp server to
deliver messages, is shown below. Alter the following example to meet your
smtp server characteristics. Notice that the smtp server is needed for a proper
installation.

Edit ``/etc/synnefo/00-snf-common-admins.conf``:

.. code-block:: python

    EMAIL_HOST = "mysmtp.server.example.com"
    EMAIL_HOST_USER = "<smtpuser>"
    EMAIL_HOST_PASSWORD = "<smtppassword>"

    # this gets appended in all email subjects
    EMAIL_SUBJECT_PREFIX = "[example.com] "

    # Address to use for outgoing emails
    DEFAULT_FROM_EMAIL = "server@example.com"

    # Email where users can contact for support. This is used in html/email
    # templates.
    CONTACT_EMAIL = "server@example.com"

    # The email address that error messages come from
    SERVER_EMAIL = "server-errors@example.com"

Notice that since email settings might be required by applications other than
Astakos, they are defined in a different configuration file than the one
previously used to set Astakos specific settings.

Refer to
`Django documentation <https://docs.djangoproject.com/en/1.4/topics/email/>`_
for additional information on available email settings.

As refered in the previous section, based on the operation that triggers
an email notification, the recipients list differs. Specifically, for
emails whose recipients include contacts from your service team
(administrators, managers, helpdesk etc) synnefo provides the following
settings located in ``00-snf-common-admins.conf``:

.. code-block:: python

    ADMINS = (('Admin name', 'admin@example.com'),
              ('Admin2 name', 'admin2@example.com))
    MANAGERS = (('Manager name', 'manager@example.com'),)
    HELPDESK = (('Helpdesk user name', 'helpdesk@example.com'),)

Alternatively, it may be convenient to send e-mails to a file, instead of an actual smtp server, using the file backend. Do so by creating a configuration file ``/etc/synnefo/99-local.conf`` including the folowing:

.. code-block:: python

    EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
    EMAIL_FILE_PATH = '/tmp/app-messages'


Enable Pooling
--------------

This section can be bypassed, but we strongly recommend you apply the following,
since they result in a significant performance boost.

Synnefo includes a pooling DBAPI driver for PostgreSQL, as a thin wrapper
around Psycopg2. This allows independent Django requests to reuse pooled DB
connections, with significant performance gains.

To use, first monkey-patch psycopg2. For Django, run this before the
``DATABASES`` setting in ``/etc/synnefo/10-snf-webproject-database.conf``:

.. code-block:: console

    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
    monkey_patch_psycopg2()

Since we are running with greenlets, we should modify psycopg2 behavior, so it
works properly in a greenlet context:

.. code-block:: console

    from synnefo.lib.db.psyco_gevent import make_psycopg_green
    make_psycopg_green()

Use the Psycopg2 driver as usual. For Django, this means using
``django.db.backends.postgresql_psycopg2`` without any modifications. To enable
connection pooling, pass a nonzero ``synnefo_poolsize`` option to the DBAPI
driver, through ``DATABASES.OPTIONS`` in Django.

All the above will result in an ``/etc/synnefo/10-snf-webproject-database.conf``
file that looks like this:

.. code-block:: console

    # Monkey-patch psycopg2
    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
    monkey_patch_psycopg2()

    # If running with greenlets
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
    make_psycopg_green()

    DATABASES = {
     'default': {
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
         'OPTIONS': {'synnefo_poolsize': 8},

         # ATTENTION: This *must* be the absolute path if using sqlite3.
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
         'NAME': 'snf_apps',
         'USER': 'synnefo',                      # Not used with sqlite3.
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
         # Set to empty string for localhost. Not used with sqlite3.
         'HOST': '203.0.113.1',
         # Set to empty string for default. Not used with sqlite3.
         'PORT': '5432',
     }
    }

Database Initialization
-----------------------

After configuration is done, we initialize the database by running:

.. code-block:: console

    # snf-manage syncdb

At this example we don't need to create a django superuser, so we select
``[no]`` to the question. After a successful sync, we run the migration needed
for Astakos:

.. code-block:: console

    # snf-manage migrate im
    # snf-manage migrate quotaholder_app
    # snf-manage migrate oa2

Then, we load the pre-defined user groups

.. code-block:: console

    # snf-manage loaddata groups

.. _services-reg:

Services Registration
---------------------

When the database is ready, we need to register the services. The following
command will ask you to register the standard Synnefo components (Astakos,
Cyclades and Pithos) along with the services they provide. Note that you
have to register at least Astakos in order to have a usable authentication
system. For each component, you will be asked to provide two URLs: its base
URL and its UI URL.

The former is the location where the component resides; it should equal
the ``<component_name>_BASE_URL`` as specified in the respective component
settings. For example, the base URL for Astakos would be
``https://node1.example.com/astakos``.

The latter is the URL that appears in the Cloudbar and leads to the
component UI. If you want to follow the default setup, set
the UI URL to ``<base_url>/ui/`` where ``base_url`` the component's base
URL as explained before. (You can later change the UI URL with
``snf-manage component-modify <component_name> --ui-url new_ui_url``.)

The command will also register automatically the resource definitions
offered by the services.

.. code-block:: console

    # snf-component-register

.. note::

   This command is equivalent to running the following series of commands;
   it registers the three components in Astakos and then in each host it
   exports the respective service definitions, copies the exported json file
   to the Astakos host, where it finally imports it:

    .. code-block:: console

       astakos-host$ snf-manage component-add astakos --base-url astakos_base_url --ui-url astakos_ui_url
       astakos-host$ snf-manage component-add cyclades --base-url cyclades_base_url --ui-url cyclades_ui_url
       astakos-host$ snf-manage component-add pithos --base-url pithos_base_url --ui-url pithos_ui_url
       astakos-host$ snf-manage service-export-astakos > astakos.json
       astakos-host$ snf-manage service-import --json astakos.json
       cyclades-host$ snf-manage service-export-cyclades > cyclades.json
       # copy the file to astakos-host
       astakos-host$ snf-manage service-import --json cyclades.json
       pithos-host$ snf-manage service-export-pithos > pithos.json
       # copy the file to astakos-host
       astakos-host$ snf-manage service-import --json pithos.json

Notice that in this installation astakos and cyclades are in node1 and pithos is in node2.

Setting Default Base Quota for Resources
----------------------------------------

We now have to specify the limit on resources that each user can employ
(exempting resources offered by projects). When specifying storage or
memory size limits you can append a unit to the value, i.e. 10240 MB,
10 GB etc. Use the special value ``inf``, if you don't want to restrict a
resource.

.. code-block:: console

    # snf-manage resource-modify cyclades.vm --system-default 2

Setting Resource Visibility
---------------------------

It is possible to control whether a resource is visible to the users via the
API or the Web UI. The default value for these options is denoted inside the
default resource definitions. Note that the system always checks and
enforces resource quota, regardless of their visibility. You can inspect the
current status with::

   # snf-manage resource-list

You can change a resource's visibility with::

   # snf-manage resource-modify <resource> --api-visible=True (or --ui-visible=True)

.. _pithos_view_registration:

Register pithos view as an OAuth 2.0 client
-------------------------------------------

Starting from Synnefo version 0.15, the pithos view, in order to get access to
the data of a protected pithos resource, has to be granted authorization for
the specific resource by astakos.

During the authorization grant procedure, it has to authenticate itself with
astakos since the latter has to prevent serving requests by
unknown/unauthorized clients.

Each oauth 2.0 client is identified by a client identifier (client_id).
Moreover, the confidential clients are authenticated via a password
(client_secret).
Then, each client has to declare at least a redirect URI so that astakos will
be able to validate the redirect URI provided during the authorization code
request.
If a client is trusted (like a pithos view), astakos grants access on behalf
of the resource owner, otherwise the resource owner has to be asked.

To register the pithos view as an OAuth 2.0 client in astakos, we have to run
the following command::

    snf-manage oauth2-client-add pithos-view --secret=<secret> --is-trusted --url https://node2.example.com/pithos/ui/view

Servers Initialization
----------------------

Finally, we initialize the servers on node1:

.. code-block:: console

    root@node1:~ # /etc/init.d/gunicorn restart
    root@node1:~ # /etc/init.d/apache2 restart

We have now finished the Astakos setup. Let's test it now.


Testing of Astakos
==================

Open your favorite browser and go to:

``http://node1.example.com/astakos``

If this redirects you to ``https://node1.example.com/astakos/ui/`` and you can see