Skip to content

GitLab

Commit f534e423 authored by Dionysis Grigoropoulos's avatar Dionysis Grigoropoulos
Browse files

docs: Update quick install admin guide for 0.14.10

parent 6d965b36
Hide whitespace changes
Inline Side-by-side
docs/quick-install-admin-guide.rst
View file @ f534e423
......@@ -5,9 +5,9 @@ Administrator's Installation Guide
This is the Administrator's installation guide.
It describes how to install the whole synnefo stack on two (2) physical nodes,
It describes how to install the whole Synnefo stack on two (2) physical nodes,
with minimum configuration. It installs synnefo from Debian packages, and
assumes the nodes run Debian Squeeze. After successful installation, you will
assumes the nodes run Debian Wheezy. After successful installation, you will
have the following services running:
* Identity Management (Astakos)
......@@ -18,9 +18,6 @@ have the following services running:
and a single unified Web UI to manage them all.
The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are
not released yet.
If you just want to install the Object Storage Service (Pithos), follow the
guide and just stop after the "Testing of Pithos" section.
......@@ -37,10 +34,11 @@ snf-cyclades-app component (scheduled to be fixed in the next version).
For the rest of the documentation we will refer to the first physical node as
"node1" and the second as "node2". We will also assume that their domain names
are "node1.example.com" and "node2.example.com" and their public IPs are "4.3.2.1" and
"4.3.2.2" respectively. It is important that the two machines are under the same domain name.
are "node1.example.com" and "node2.example.com" and their public IPs are "203.0.113.1" and
"203.0.113.2" respectively. It is important that the two machines are under the same domain name.
In case you choose to follow a private installation you will need to
set up a private dns server, using dnsmasq for example. See node1 below for more.
set up a private dns server, using dnsmasq for example. See node1 below for
more information on how to do so.
General Prerequisites
=====================
......@@ -51,22 +49,21 @@ and are related to all the services (Astakos, Pithos, Cyclades).
To be able to download all synnefo components you need to add the following
lines in your ``/etc/apt/sources.list`` file:
| ``deb http://apt.dev.grnet.gr squeeze/``
| ``deb-src http://apt.dev.grnet.gr squeeze/``
| ``deb http://apt.dev.grnet.gr wheezy/``
| ``deb-src http://apt.dev.grnet.gr wheezy/``
and import the repo's GPG key:
| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``
Also add the following line to enable the ``squeeze-backports`` repository,
which may provide more recent versions of certain packages. The repository
is deactivated by default and must be specified expicitly in ``apt-get``
operations:
Update your list of packages and continue with the installation:
.. code-block:: console
| ``deb http://backports.debian.org/debian-backports squeeze-backports main``
# apt-get update
You also need a shared directory visible by both nodes. Pithos will save all
data inside this directory. By 'all data', we mean files, images, and pithos
data inside this directory. By 'all data', we mean files, images, and Pithos
specific mapping data. If you plan to upload more than one basic image, this
directory should have at least 50GB of free space. During this guide, we will
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
......@@ -96,7 +93,7 @@ General Synnefo dependencies
* rabbitmq (message queue)
* ntp (NTP daemon)
* gevent
* dns server
* dnsmasq (DNS server)
You can install apache2, postgresql, ntp and rabbitmq by running:
......@@ -104,18 +101,11 @@ You can install apache2, postgresql, ntp and rabbitmq by running:
# apt-get install apache2 postgresql ntp rabbitmq-server
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
the official debian backports:
To install gunicorn and gevent, run:
.. code-block:: console
# apt-get -t squeeze-backports install gunicorn
Also, make sure to install gevent >= 0.13.6. Again from the debian backports:
.. code-block:: console
# apt-get -t squeeze-backports install python-gevent
# apt-get install gunicorn python-gevent
On node1, we will create our databases, so you will also need the
python-psycopg2 package:
......@@ -124,7 +114,6 @@ python-psycopg2 package:
# apt-get install python-psycopg2
Database setup
~~~~~~~~~~~~~~
......@@ -151,59 +140,42 @@ create all needed databases on node1 and then node2 will connect to them.
postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
Configure the database to listen to all network interfaces. You can do this by
editting the file ``/etc/postgresql/8.4/main/postgresql.conf`` and change
editting the file ``/etc/postgresql/9.1/main/postgresql.conf`` and change
``listen_addresses`` to ``'*'`` :
.. code-block:: console
listen_addresses = '*'
Furthermore, edit ``/etc/postgresql/8.4/main/pg_hba.conf`` to allow node1 and
Furthermore, edit ``/etc/postgresql/9.1/main/pg_hba.conf`` to allow node1 and
node2 to connect to the database. Add the following lines under ``#IPv4 local
connections:`` :
.. code-block:: console
host all all 4.3.2.1/32 md5
host all all 4.3.2.2/32 md5
host all all 203.0.113.1/32 md5
host all all 203.0.113.2/32 md5
Make sure to substitute "4.3.2.1" and "4.3.2.2" with node1's and node2's
Make sure to substitute "203.0.113.1" and "203.0.113.2" with node1's and node2's
actual IPs. Now, restart the server to apply the changes:
.. code-block:: console
# /etc/init.d/postgresql restart
Gunicorn setup
~~~~~~~~~~~~~~
Rename the file ``/etc/gunicorn.d/synnefo.example`` to
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:
.. code-block:: console
# mv /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
.. warning:: Do NOT start the server yet, because it won't find the
``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``
instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to
``--worker-class=sync``. We will start the server after successful
installation of astakos. If the server is running::
# /etc/init.d/gunicorn stop
Certificate Creation
~~~~~~~~~~~~~~~~~~~~~
Node1 will host Cyclades. Cyclades should communicate with the other snf tools over a trusted connection.
In order for the connection to be trusted, the keys provided to apache below should be signed with a certificate.
Node1 will host Cyclades. Cyclades should communicate with the other Synnefo
Services and users over a secure channel. In order for the connection to be
trusted, the keys provided to Apache below should be signed with a certificate.
This certificate should be added to all nodes. In case you don't have signed keys you can create a self-signed certificate
and sign your keys with this. To do so on node1 run
and sign your keys with this. To do so on node1 run:
.. code-block:: console
# aptitude install openvpn
# apt-get install openvpn
# mkdir /etc/openvpn/easy-rsa
# cp -ai /usr/share/doc/openvpn/examples/easy-rsa/2.0/ /etc/openvpn/easy-rsa
# cd /etc/openvpn/easy-rsa/2.0
......@@ -222,8 +194,8 @@ Now you can create the certificate
# ./build-ca
The previous will create a ``ca.crt`` file. Copy this file under
``/usr/local/share/ca-certificates/`` directory and run :
The previous will create a ``ca.crt`` file in the directory ``/etc/openvpn/easy-rsa/2.0/keys``.
Copy this file under ``/usr/local/share/ca-certificates/`` directory and run :
.. code-block:: console
......@@ -237,9 +209,10 @@ Now you can create the keys and sign them with the certificate
# ./build-key-server node1.example.com
This will create a .pem and a .key file in your current folder. Copy these in
``/etc/ssl/certs/`` and ``/etc/ssl/private/`` respectively and
use them in the apache2 configuration file below instead of the defaults.
This will create a ``01.pem`` and a ``node1.example.com.key`` files in the
``/etc/openvpn/easy-rsa/2.0/keys`` directory. Copy these in ``/etc/ssl/certs/``
and ``/etc/ssl/private/`` respectively and use them in the apache2
configuration file below instead of the defaults.
Apache2 setup
~~~~~~~~~~~~~
......@@ -343,42 +316,90 @@ during the Cyclades setup.
Pithos data directory setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~
As mentioned in the General Prerequisites section, there is a directory called
``/srv/pithos`` visible by both nodes. We create and setup the ``data``
As mentioned in the General Prerequisites section, there should be a directory
called ``/srv/pithos`` visible by both nodes. We create and setup the ``data``
directory inside it:
.. code-block:: console
# mkdir /srv/pithos
# cd /srv/pithos
# mkdir data
# chown www-data:www-data data
# chmod g+ws data
This directory must be shared via `NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
In order to do this, run:
.. code-block:: console
# apt-get install rpcbind nfs-kernel-server
Now edit ``/etc/exports`` and add the following line:
.. code-block:: console
/srv/pithos/ 203.0.113.2(rw,no_root_squash,sync,subtree_check)
Once done, run:
.. code-block:: console
# /etc/init.d/nfs-kernel-server restart
DNS server setup
~~~~~~~~~~~~~~~~
If your machines are not under the same domain nameyou have to set up a dns server.
In order to set up a dns server using dnsmasq do the following
If your machines are not under the same domain name you have to set up a dns server.
In order to set up a dns server using dnsmasq do the following:
.. code-block:: console
# apt-get install dnsmasq
# apt-get install dnsmasq
Then edit you ``/etc/hosts/`` as follows
Then edit your ``/etc/hosts/`` file as follows:
.. code-block:: console
4.3.2.1 node1.example.com
4.3.2.2 node2.example.com
203.0.113.1 node1.example.com
203.0.113.2 node2.example.com
Finally edit the ``/etc/dnsmasq.conf`` file and specify the ``listen-address`` and
the ``interface`` you would like to listen to.
dnsmasq will serve any IPs/domains found in ``/etc/resolv.conf``.
Also add the following in your ``/etc/resolv.conf`` file
There is a `"bug" in libevent 2.0.5 <http://sourceforge.net/p/levent/bugs/193/>`_
, where if you have multiple nameservers in your ``/etc/resolv.conf``, libevent
will round-robin against them. To avoid this, you must use a single nameserver
for all your needs. Edit your ``/etc/resolv.conf`` to include your dns server:
.. code-block:: console
nameserver 4.3.2.1
nameserver 203.0.113.1
Because of the aforementioned bug, you can't specify more than one DNS servers
in your ``/etc/resolv.conf``. In order for dnsmasq to serve domains not in
``/etc/hosts``, edit ``/etc/dnsmasq.conf`` and change the line starting with
``#resolv-file=`` to:
.. code-block:: console
resolv-file=/etc/external-dns
Now create the file ``/etc/external-dns`` and specify any extra DNS servers you
want dnsmasq to query for domains, e.g., 8.8.8.8:
.. code-block:: console
nameserver 8.8.8.8
In the ``/etc/dnsmasq.conf`` file, you can also specify the ``listen-address``
and the ``interface`` you would like dnsmasq to listen to.
Finally, restart dnsmasq:
.. code-block:: console
# /etc/init.d/dnsmasq restart
You are now ready with all general prerequisites concerning node1. Let's go to
node2.
......@@ -395,7 +416,7 @@ General Synnefo dependencies
* ntp (NTP daemon)
* gevent
* certificates
* dns setup
* dnsmasq (DNS server)
You can install the above by running:
......@@ -403,18 +424,11 @@ You can install the above by running:
# apt-get install apache2 postgresql ntp
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
the official debian backports:
To install gunicorn and gevent, run:
.. code-block:: console
# apt-get -t squeeze-backports install gunicorn
Also, make sure to install gevent >= 0.13.6. Again from the debian backports:
.. code-block:: console
# apt-get -t squeeze-backports install python-gevent
# apt-get install gunicorn python-gevent
Node2 will connect to the databases on node1, so you will also need the
python-psycopg2 package:
......@@ -432,26 +446,6 @@ with the software you may choose to run different databases on different nodes,
for performance/scalability/redundancy reasons, but those kind of setups are out
of the purpose of this guide.
Gunicorn setup
~~~~~~~~~~~~~~
Rename the file ``/etc/gunicorn.d/synnefo.example`` to
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file
(as happened for node1):
.. code-block:: console
# mv /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
.. warning:: Do NOT start the server yet, because it won't find the
``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``
instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to
``--worker-class=sync``. We will start the server after successful
installation of astakos. If the server is running::
# /etc/init.d/gunicorn stop
Apache2 setup
~~~~~~~~~~~~~
......@@ -531,13 +525,11 @@ Acquire certificate
~~~~~~~~~~~~~~~~~~~
Copy the certificate you created before on node1 (`ca.crt`) under the directory
``/usr/local/share/ca-certificate``
and run:
``/usr/local/share/ca-certificate`` and run:
.. code-block:: console
# update-ca-certificates
# update-ca-certificates
to update the records.
......@@ -549,9 +541,12 @@ Add the following line in ``/etc/resolv.conf`` file
.. code-block:: console
nameserver 4.3.2.1
nameserver 203.0.113.1
to inform the node about the new dns server.
to inform the node about the new DNS server.
As mentioned before, this should be the only ``nameserver`` entry in
``/etc/resolv.conf``.
We are now ready with all general prerequisites for node2. Now that we have
finished with all general prerequisites for both nodes, we can start installing
......@@ -560,9 +555,9 @@ the services. First, let's install Astakos on node1.
Installation of Astakos on node1
================================
To install astakos, grab the package from our repository (make sure you made
the additions needed in your ``/etc/apt/sources.list`` file, as described
previously), by running:
To install Astakos, grab the package from our repository (make sure you made
the additions needed in your ``/etc/apt/sources.list`` file and updated, as
described previously), by running:
.. code-block:: console
......@@ -573,21 +568,40 @@ previously), by running:
Configuration of Astakos
========================
Gunicorn setup
--------------
Copy the file ``/etc/gunicorn.d/synnefo.example`` to
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:
.. code-block:: console
# mv /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
.. warning:: Do NOT start the server yet, because it won't find the
``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``
instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to
``--worker-class=sync``. We will start the server after successful
installation of Astakos. If the server is running::
# /etc/init.d/gunicorn stop
Conf Files
----------
After astakos is successfully installed, you will find the directory
After Astakos is successfully installed, you will find the directory
``/etc/synnefo`` and some configuration files inside it. The files contain
commented configuration options, which are the default options. While installing
new snf-* components, new configuration files will appear inside the directory.
In this guide (and for all services), we will edit only the minimum necessary
configuration options, to reflect our setup. Everything else will remain as is.
After getting familiar with synnefo, you will be able to customize the software
After getting familiar with Synnefo, you will be able to customize the software
as you wish and fits your needs. Many options are available, to empower the
administrator with extensively customizable setups.
For the snf-webproject component (installed as an astakos dependency), we
For the snf-webproject component (installed as an Astakos dependency), we
need the following:
Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
......@@ -605,7 +619,7 @@ uncomment and edit the ``DATABASES`` block to reflect our database:
'USER': 'synnefo', # Not used with sqlite3.
'PASSWORD': 'example_passw0rd', # Not used with sqlite3.
# Set to empty string for localhost. Not used with sqlite3.
'HOST': '4.3.2.1',
'HOST': '203.0.113.1',
# Set to empty string for default. Not used with sqlite3.
'PORT': '5432',
}
......@@ -620,7 +634,7 @@ choice and keep it private:
SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
For astakos specific configuration, edit the following options in
For Astakos specific configuration, edit the following options in
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
.. code-block:: console
......@@ -630,7 +644,7 @@ For astakos specific configuration, edit the following options in
ASTAKOS_BASE_URL = 'https://node1.example.com/astakos'
The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all
services). ``ASTAKOS_BASE_URL`` is the astakos top-level URL. Appending an
services). ``ASTAKOS_BASE_URL`` is the Astakos top-level URL. Appending an
extra path (``/astakos`` here) is recommended in order to distinguish
components, if more than one are installed on the same machine.
......@@ -669,13 +683,13 @@ method, read the relative :ref:`section <shibboleth-auth>`.
Email delivery configuration
----------------------------
Many of the ``astakos`` operations require server to notify service users and
administrators via email. e.g. right after the signup process the service sents
an email to the registered email address containing an email verification url,
after the user verifies the email address astakos once again needs to notify
administrators with a notice that a new account has just been verified.
Many of the ``Astakos`` operations require the server to notify service users
and administrators via email. e.g. right after the signup process, the service
sents an email to the registered email address containing an verification url.
After the user verifies the email address, Astakos once again needs to
notify administrators with a notice that a new account has just been verified.
More specifically astakos sends emails in the following cases
More specifically Astakos sends emails in the following cases
- An email containing a verification link after each signup process.
- An email to the people listed in ``ADMINS`` setting after each email
......@@ -684,7 +698,7 @@ More specifically astakos sends emails in the following cases
activate the user.
- A welcome email to the user email and an admin notification to ``ADMINS``
right after each account activation.
- Feedback messages submited from astakos contact view and astakos feedback
- Feedback messages submited from Astakos contact view and Astakos feedback
API endpoint are sent to contacts listed in ``HELPDESK`` setting.
- Project application request notifications to people included in ``HELPDESK``
and ``MANAGERS`` settings.
......@@ -695,48 +709,49 @@ Astakos uses the Django internal email delivering mechanism to send email
notifications. A simple configuration, using an external smtp server to
deliver messages, is shown below. Alter the following example to meet your
smtp server characteristics. Notice that the smtp server is needed for a proper
installation
installation.
Edit ``/etc/synnefo/00-snf-common-admins.conf``:
.. code-block:: python
# /etc/synnefo/00-snf-common-admins.conf
EMAIL_HOST = "mysmtp.server.synnefo.org"
EMAIL_HOST = "mysmtp.server.example.com"
EMAIL_HOST_USER = "<smtpuser>"
EMAIL_HOST_PASSWORD = "<smtppassword>"
# this gets appended in all email subjects
EMAIL_SUBJECT_PREFIX = "[example.synnefo.org] "
EMAIL_SUBJECT_PREFIX = "[example.com] "
# Address to use for outgoing emails
DEFAULT_FROM_EMAIL = "server@example.synnefo.org"
DEFAULT_FROM_EMAIL = "server@example.com"
# Email where users can contact for support. This is used in html/email
# templates.
CONTACT_EMAIL = "server@example.synnefo.org"
CONTACT_EMAIL = "server@example.com"
# The email address that error messages come from
SERVER_EMAIL = "server-errors@example.synnefo.org"
SERVER_EMAIL = "server-errors@example.com"
Notice that since email settings might be required by applications other than
astakos they are defined in a different configuration file than the one
previously used to set astakos specific settings.
Astakos, they are defined in a different configuration file than the one
previously used to set Astakos specific settings.
Refer to
`Django documentation <https://docs.djangoproject.com/en/1.4/topics/email/>`_
for additional information on available email settings.
As refered in the previous section, based on the operation that triggers
an email notification, the recipients list differs. Specifically for
an email notification, the recipients list differs. Specifically, for
emails whose recipients include contacts from your service team
(administrators, managers, helpdesk etc) synnefo provides the following
settings located in ``00-snf-common-admins.conf``:
.. code-block:: python
ADMINS = (('Admin name', 'admin@example.synnefo.org'),
('Admin2 name', 'admin2@example.synnefo.org))
MANAGERS = (('Manager name', 'manager@example.synnefo.org'),)
HELPDESK = (('Helpdesk user name', 'helpdesk@example.synnefo.org'),)
ADMINS = (('Admin name', 'admin@example.com'),
('Admin2 name', 'admin2@example.com))
MANAGERS = (('Manager name', 'manager@example.com'),)
HELPDESK = (('Helpdesk user name', 'helpdesk@example.com'),)
Alternatively, it may be convenient to send e-mails to a file, instead of an actual smtp server, using the file backend. Do so by creating a configuration file ``/etc/synnefo/99-local.conf`` including the folowing:
......@@ -803,7 +818,7 @@ file that looks like this:
'USER': 'synnefo', # Not used with sqlite3.
'PASSWORD': 'example_passw0rd', # Not used with sqlite3.
# Set to empty string for localhost. Not used with sqlite3.
'HOST': '4.3.2.1',
'HOST': '203.0.113.1',
# Set to empty string for default. Not used with sqlite3.
'PORT': '5432',
}
......@@ -820,7 +835,7 @@ After configuration is done, we initialize the database by running:
At this example we don't need to create a django superuser, so we select
``[no]`` to the question. After a successful sync, we run the migration needed
for astakos:
for Astakos:
.. code-block:: console
......@@ -839,15 +854,15 @@ Services Registration
---------------------
When the database is ready, we need to register the services. The following
command will ask you to register the standard Synnefo components (astakos,
cyclades, and pithos) along with the services they provide. Note that you
have to register at least astakos in order to have a usable authentication
command will ask you to register the standard Synnefo components (Astakos,
Cyclades and Pithos) along with the services they provide. Note that you
have to register at least Astakos in order to have a usable authentication
system. For each component, you will be asked to provide two URLs: its base
URL and its UI URL.
The former is the location where the component resides; it should equal
the ``<component_name>_BASE_URL`` as specified in the respective component
settings. For example, the base URL for astakos would be
settings. For example, the base URL for Astakos would be
``https://node1.example.com/astakos``.
The latter is the URL that appears in the Cloudbar and leads to the
......@@ -866,9 +881,9 @@ offered by the services.
.. note::
This command is equivalent to running the following series of commands;
it registers the three components in astakos and then in each host it
it registers the three components in Astakos and then in each host it
exports the respective service definitions, copies the exported json file
to the astakos host, where it finally imports it:
to the Astakos host, where it finally imports it:
.. code-block:: console
......@@ -884,7 +899,7 @@ offered by the services.
# copy the file to astakos-host
astakos-host$ snf-manage service-import --json pithos.json
Notice that in this installation astakos and cyclades are in node1 and pithos is in node2
Notice that in this installation astakos and cyclades are in node1 and pithos is in node2.
Setting Default Base Quota for Resources
----------------------------------------
......@@ -977,7 +992,7 @@ him/her an activation email. See how to do this at the :ref:`User