install-guide-debian.rst 86.5 KB
Newer Older
1
.. _install-guide-debian:
2

3
4
Administrator's Installation Guide (on Debian Wheezy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

6
This is the Administrator's installation guide on Debian Wheezy.
7

8
It describes how to install the whole Synnefo stack on two (2) physical nodes,
9
with minimum configuration. It installs Synnefo from Debian packages, and
10
assumes the nodes run Debian Wheezy. After successful installation, you will
11
12
have the following services running:

13
    * Identity Management (Astakos)
14
    * Object Storage Service (Pithos)
15
    * Compute Service (Cyclades)
16
17
    * Image Service (part of Cyclades)
    * Network Service (part of Cyclades)
18
19
20

and a single unified Web UI to manage them all.

21
22
If you just want to install the Object Storage Service (Pithos), follow the
guide and just stop after the "Testing of Pithos" section.
23
24
25
26
27
28
29


Installation of Synnefo / Introduction
======================================

For the rest of the documentation we will refer to the first physical node as
"node1" and the second as "node2". We will also assume that their domain names
30
31
are "node1.example.com" and "node2.example.com" and their public IPs are "203.0.113.1" and
"203.0.113.2" respectively. It is important that the two machines are under the same domain name.
32
In case you choose to follow a private installation you will need to
33
set up a private dns server, using dnsmasq for example. See node1 below for
34
more information on how to do so.
35
36
37
38

General Prerequisites
=====================

39
These are the general Synnefo prerequisites, that you need on node1 and node2
40
and are related to all the services (Astakos, Pithos, Cyclades).
41

42
To be able to download all Synnefo components you need to add the following
43
44
lines in your ``/etc/apt/sources.list`` file:

45
46
| ``deb http://apt.dev.grnet.gr wheezy/``
| ``deb-src http://apt.dev.grnet.gr wheezy/``
47

48
49
50
51
and import the repo's GPG key:

| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``

52
53
54
Update your list of packages and continue with the installation:

.. code-block:: console
55

56
   # apt-get update
57

58
You also need a shared directory visible by both nodes. Pithos will save all
59
data inside this directory. By 'all data', we mean files, images, and Pithos
60
61
62
specific mapping data. If you plan to upload more than one basic image, this
directory should have at least 50GB of free space. During this guide, we will
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
63
64
to node2 (be sure to set no_root_squash flag). Node2 has this directory
mounted under ``/srv/pithos``, too.
65

66
Before starting the Synnefo installation, you will need basic third party
67
68
software to be installed and configured on the physical nodes. We will describe
each node's general prerequisites separately. Any additional configuration,
69
specific to a Synnefo service for each node, will be described at the service's
70
71
section.

72
73
74
Finally, it is required for Cyclades and Ganeti nodes to have synchronized
system clocks (e.g. by running ntpd).

75
76
77
Node1
-----

78

79
80
81
General Synnefo dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

82
83
84
85
86
87
88
		* apache (http server)
		* public certificate
		* gunicorn (WSGI http server)
		* postgresql (database)
		* rabbitmq (message queue)
		* ntp (NTP daemon)
		* gevent
89
		* dnsmasq (DNS server)
90
		* Archipelago
91

92
You can install apache2, postgresql, ntp and rabbitmq by running:
93
94
95

.. code-block:: console

96
   # apt-get install apache2 postgresql ntp rabbitmq-server
97

98
To install gunicorn and gevent, run:
99
100
101

.. code-block:: console

102
   # apt-get install gunicorn python-gevent
103

104
105
On node1, we will create our databases, so you will also need the
python-psycopg2 package:
106
107
108
109
110
111
112
113
114
115
116
117
118
119

.. code-block:: console

   # apt-get install python-psycopg2

Database setup
~~~~~~~~~~~~~~

On node1, we create a database called ``snf_apps``, that will host all django
apps related tables. We also create the user ``synnefo`` and grant him all
privileges on the database. We do this by running:

.. code-block:: console

120
121
122
123
124
    root@node1:~ # su - postgres
    postgres@node1:~ $ psql
    postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
    postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;
125

126
We also create the database ``snf_pithos`` needed by the Pithos backend and
127
128
129
130
131
132
grant the ``synnefo`` user all privileges on the database. This database could
be created on node2 instead, but we do it on node1 for simplicity. We will
create all needed databases on node1 and then node2 will connect to them.

.. code-block:: console

133
134
    postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
135
136

Configure the database to listen to all network interfaces. You can do this by
137
editing the file ``/etc/postgresql/9.1/main/postgresql.conf`` and change
138
139
140
141
``listen_addresses`` to ``'*'`` :

.. code-block:: console

142
    listen_addresses = '*'
143

144
Furthermore, edit ``/etc/postgresql/9.1/main/pg_hba.conf`` to allow node1 and
145
146
147
148
149
node2 to connect to the database. Add the following lines under ``#IPv4 local
connections:`` :

.. code-block:: console

150
151
    host		all	all	203.0.113.1/32	md5
    host		all	all	203.0.113.2/32	md5
152

153
Make sure to substitute "203.0.113.1" and "203.0.113.2" with node1's and node2's
154
155
156
157
158
159
160
actual IPs. Now, restart the server to apply the changes:

.. code-block:: console

   # /etc/init.d/postgresql restart


161
162
163
Certificate Creation
~~~~~~~~~~~~~~~~~~~~~

164
165
Node1 will host Cyclades. Cyclades should communicate with the other Synnefo
Services and users over a secure channel. In order for the connection to be
166
trusted, the keys provided to Apache below should be signed with a certificate.
167
This certificate should be added to all nodes. In case you don't have signed keys you can create a self-signed certificate
168
and sign your keys with this. To do so on node1 run:
169
170
171

.. code-block:: console

172
		# apt-get install openvpn
173
174
175
176
177
178
179
180
181
182
183
184
185
186
		# mkdir /etc/openvpn/easy-rsa
		# cp -ai /usr/share/doc/openvpn/examples/easy-rsa/2.0/ /etc/openvpn/easy-rsa
		# cd /etc/openvpn/easy-rsa/2.0
		# vim vars

In vars you can set your own parameters such as KEY_COUNTRY

.. code-block:: console

	# . ./vars
	# ./clean-all

Now you can create the certificate

187
188
.. code-block:: console

189
190
		# ./build-ca

191
192
The previous will create a ``ca.crt`` file in the directory ``/etc/openvpn/easy-rsa/2.0/keys``.
Copy this file under ``/usr/local/share/ca-certificates/`` directory and run :
193
194
195
196
197
198
199
200
201
202

.. code-block:: console

		# update-ca-certificates

to update the records. You will have to do the following on node2 as well.

Now you can create the keys and sign them with the certificate

.. code-block:: console
203

204
205
		# ./build-key-server node1.example.com

206
This will create a ``01.pem`` and a ``node1.example.com.key`` files in the
207
``/etc/openvpn/easy-rsa/2.0/keys`` directory. Copy these in ``/etc/ssl/certs/``
208
and ``/etc/ssl/private/`` respectively and use them in the apache2
209
configuration file below instead of the defaults.
210

211
212
213
Apache2 setup
~~~~~~~~~~~~~

214
215
Create the file ``/etc/apache2/sites-available/synnefo`` containing the
following:
216
217
218

.. code-block:: console

219
220
    <VirtualHost *:80>
        ServerName node1.example.com
221

222
223
224
225
226
        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    </VirtualHost>
227

228

229
230
Create the file ``/etc/apache2/sites-available/synnefo-ssl`` containing the
following:
231
232
233

.. code-block:: console

234
235
236
    <IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName node1.example.com
237

238
        Alias /static "/usr/share/synnefo/static"
239

240
241
        #  SetEnv no-gzip
        #  SetEnv dont-vary
242

243
       AllowEncodedSlashes On
244

245
       RequestHeader set X-Forwarded-Protocol "https"
246

247
248
249
250
    <Proxy * >
        Order allow,deny
        Allow from all
    </Proxy>
251

252
253
254
        SetEnv                proxy-sendchunked
        SSLProxyEngine        off
        ProxyErrorOverride    off
255

256
257
258
        ProxyPass        /static !
        ProxyPass        / http://localhost:8080/ retry=0
        ProxyPassReverse / http://localhost:8080/
259

260
261
262
        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]
263

264
265
266
267
268
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
    </VirtualHost>
    </IfModule>
269
270
271
272
273
274
275
276
277
278
279
280
281

Now enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl
   # a2enmod rewrite
   # a2dissite default
   # a2ensite synnefo
   # a2ensite synnefo-ssl
   # a2enmod headers
   # a2enmod proxy_http

282
283
284
285
286
287
.. note:: This isn't really needed, but it's a good security practice to disable
    directory listing in apache::

        # a2dismod autoindex


288
.. warning:: Do NOT start/restart the server yet. If the server is running::
289

290
       # /etc/init.d/apache2 stop
291

292

293
294
.. _rabbitmq-setup:

295
296
297
298
299
300
301
302
303
Message Queue setup
~~~~~~~~~~~~~~~~~~~

The message queue will run on node1, so we need to create the appropriate
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
exchanges:

.. code-block:: console

Ilias Tsitsimpis's avatar
Ilias Tsitsimpis committed
304
   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"
305
306
307
308
309
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"

We do not need to initialize the exchanges. This will be done automatically,
during the Cyclades setup.

310
311
Pithos data directory setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~
312

313
As mentioned in the General Prerequisites section, there should be a directory
314
called ``/srv/pithos`` visible by both nodes. We create and setup the ``data``
315
directory inside it along with the ``maps`` and ``blocks`` subdirectories:
316
317
318

.. code-block:: console

319
   # mkdir /srv/pithos
320
321
   # cd /srv/pithos
   # mkdir data
322
   # mkdir -p data/{maps,blocks}
323

324
325
326
327
328
329
330
331
332
333
This directory must be shared via `NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
In order to do this, run:

.. code-block:: console

   # apt-get install rpcbind nfs-kernel-server

Now edit ``/etc/exports`` and add the following line:

.. code-block:: console
334

335
336
337
338
339
340
341
342
   /srv/pithos/ 203.0.113.2(rw,no_root_squash,sync,subtree_check)

Once done, run:

.. code-block:: console

   # /etc/init.d/nfs-kernel-server restart

343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
Archipelago setup
~~~~~~~~~~~~~~~~~

To install Archipelago, run:

.. code-block:: console

   # apt-get install archipelago


Now edit ``/etc/archipelago/archipelago.conf`` and tweak the following settings:

* ``SEGMENT_SIZE``: Adjust shared memory segment size according to your machine's
  RAM. The default value is 2GB which in some situations might exceed your
  machine's physical RAM.

In section ``blockerb`` set:

* ``archip_dir``: ``/srv/pithos/data/blocks``

In section ``blockerm`` set:

* ``archip_dir``: ``/srv/pithos/data/maps``

Finally, restart Archipelago:

.. code-block:: console

   # /etc/init.d/archipelago restart
372

373
374
375
DNS server setup
~~~~~~~~~~~~~~~~

376
377
If your machines are not under the same domain name you have to set up a dns server.
In order to set up a dns server using dnsmasq do the following:
378
379

.. code-block:: console
380

381
   # apt-get install dnsmasq
382

383
Then edit your ``/etc/hosts/`` file as follows:
384
385
386

.. code-block:: console

387
388
		203.0.113.1     node1.example.com
		203.0.113.2     node2.example.com
389

390
dnsmasq will serve any IPs/domains found in ``/etc/hosts``.
391

392
393
394
There is a `"bug" in libevent 2.0.5 <http://sourceforge.net/p/levent/bugs/193/>`_
, where if you have multiple nameservers in your ``/etc/resolv.conf``, libevent
will round-robin against them. To avoid this, you must use a single nameserver
395
for all your needs. Edit your ``/etc/resolv.conf`` to include your dns server:
396
397
398

.. code-block:: console

399
400
401
   nameserver 203.0.113.1

Because of the aforementioned bug, you can't specify more than one DNS servers
402
403
in your ``/etc/resolv.conf``. In order for dnsmasq to serve domains not in
``/etc/hosts``, edit ``/etc/dnsmasq.conf`` and change the line starting with
404
405
406
407
408
409
410
411
412
413
414
415
416
``#resolv-file=`` to:

.. code-block:: console

   resolv-file=/etc/external-dns

Now create the file ``/etc/external-dns`` and specify any extra DNS servers you
want dnsmasq to query for domains, e.g., 8.8.8.8:

.. code-block:: console

   nameserver 8.8.8.8

417
In the ``/etc/dnsmasq.conf`` file, you can also specify the ``listen-address``
418
419
420
421
422
423
424
and the ``interface`` you would like dnsmasq to listen to.

Finally, restart dnsmasq:

.. code-block:: console

   # /etc/init.d/dnsmasq restart
425

426
427
428
429
430
431
432
433
434
You are now ready with all general prerequisites concerning node1. Let's go to
node2.

Node2
-----

General Synnefo dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

435
436
437
438
439
    * apache (http server)
    * gunicorn (WSGI http server)
    * postgresql (database)
    * ntp (NTP daemon)
    * gevent
440
    * certificates
441
    * dnsmasq (DNS server)
442
    * Archipelago
443
444
445
446
447

You can install the above by running:

.. code-block:: console

448
   # apt-get install apache2 postgresql ntp
449

450
To install gunicorn and gevent, run:
451
452
453

.. code-block:: console

454
   # apt-get install gunicorn python-gevent
455

456
457
Node2 will connect to the databases on node1, so you will also need the
python-psycopg2 package:
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473

.. code-block:: console

   # apt-get install python-psycopg2

Database setup
~~~~~~~~~~~~~~

All databases have been created and setup on node1, so we do not need to take
any action here. From node2, we will just connect to them. When you get familiar
with the software you may choose to run different databases on different nodes,
for performance/scalability/redundancy reasons, but those kind of setups are out
of the purpose of this guide.

Apache2 setup
~~~~~~~~~~~~~
474

475
476
Create the file ``/etc/apache2/sites-available/synnefo`` containing the
following:
477

478
479
.. code-block:: console

480
481
    <VirtualHost *:80>
        ServerName node2.example.com
482

483
484
485
486
487
        RewriteEngine On
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
        RewriteRule ^(.*)$ - [F,L]
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    </VirtualHost>
488
489
490
491
492
493

Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
containing the following:

.. code-block:: console

494
495
496
    <IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName node2.example.com
497

498
        Alias /static "/usr/share/synnefo/static"
499

500
501
502
        SetEnv no-gzip
        SetEnv dont-vary
        AllowEncodedSlashes On
503

504
        RequestHeader set X-Forwarded-Protocol "https"
505

506
507
508
509
        <Proxy * >
            Order allow,deny
            Allow from all
        </Proxy>
510

511
512
513
        SetEnv                proxy-sendchunked
        SSLProxyEngine        off
        ProxyErrorOverride    off
514

515
516
517
        ProxyPass        /static !
        ProxyPass        / http://localhost:8080/ retry=0
        ProxyPassReverse / http://localhost:8080/
518

519
520
521
522
523
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
    </VirtualHost>
    </IfModule>
524
525
526
527
528
529
530
531
532
533
534
535
536

As in node1, enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl
   # a2enmod rewrite
   # a2dissite default
   # a2ensite synnefo
   # a2ensite synnefo-ssl
   # a2enmod headers
   # a2enmod proxy_http

537
538
539
540
541
.. note:: This isn't really needed, but it's a good security practice to disable
    directory listing in apache::

        # a2dismod autoindex

542
.. warning:: Do NOT start/restart the server yet. If the server is running::
543

544
       # /etc/init.d/apache2 stop
545

546
547
548
549
550

Acquire certificate
~~~~~~~~~~~~~~~~~~~

Copy the certificate you created before on node1 (`ca.crt`) under the directory
551
``/usr/local/share/ca-certificate`` and run:
552
553
554

.. code-block:: console

555
   # update-ca-certificates
556
557
558
559
560
561
562
563
564
565

to update the records.


DNS Setup
~~~~~~~~~

Add the following line in ``/etc/resolv.conf`` file

.. code-block:: console
566

567
   nameserver 203.0.113.1
568

569
570
to inform the node about the new DNS server.

571
As mentioned before, this should be the only ``nameserver`` entry in
572
``/etc/resolv.conf``.
573

574
575
576
577
578
579
580
We are now ready with all general prerequisites for node2. Now that we have
finished with all general prerequisites for both nodes, we can start installing
the services. First, let's install Astakos on node1.

Installation of Astakos on node1
================================

581
To install Astakos, grab the package from our repository (make sure  you made
582
the additions needed in your ``/etc/apt/sources.list`` file and updated, as
583
described previously), by running:
584
585
586

.. code-block:: console

587
   # apt-get install snf-astakos-app snf-pithos-backend
588

589
590
.. _conf-astakos:

591
592
593
Configuration of Astakos
========================

594
595
596
597
598
599
600
601
Gunicorn setup
--------------

Copy the file ``/etc/gunicorn.d/synnefo.example`` to
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:

.. code-block:: console

602
    # cp /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
603
604
605


.. warning:: Do NOT start the server yet, because it won't find the
606
607
608
609
610
    ``synnefo.settings`` module. Also, change ``--worker-class=gevent`` to
    ``--worker-class=pithos.workers.gevent_archipelago.GeventArchipelagoWorker``
    and set ``--config=/etc/synnefo/pithos.conf.py``.
    We will start the server after successful installation of Astakos.
    If the server is running::
611
612
613

       # /etc/init.d/gunicorn stop

614
615
616
Conf Files
----------

617
After Astakos is successfully installed, you will find the directory
618
619
620
621
622
623
``/etc/synnefo`` and some configuration files inside it. The files contain
commented configuration options, which are the default options. While installing
new snf-* components, new configuration files will appear inside the directory.
In this guide (and for all services), we will edit only the minimum necessary
configuration options, to reflect our setup. Everything else will remain as is.

624
After getting familiar with Synnefo, you will be able to customize the software
625
626
627
as you wish and fits your needs. Many options are available, to empower the
administrator with extensively customizable setups.

628
For the snf-webproject component (installed as an Astakos dependency), we
629
630
631
632
633
634
635
need the following:

Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
uncomment and edit the ``DATABASES`` block to reflect our database:

.. code-block:: console

636
637
638
    DATABASES = {
     'default': {
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
639
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
640
641
         # ATTENTION: This *must* be the absolute path if using sqlite3.
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
642
643
644
645
         'NAME': 'snf_apps',
         'USER': 'synnefo',                      # Not used with sqlite3.
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
         # Set to empty string for localhost. Not used with sqlite3.
646
         'HOST': '203.0.113.1',
647
648
649
         # Set to empty string for default. Not used with sqlite3.
         'PORT': '5432',
     }
650
651
652
    }

Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
653
``SECRET_KEY``. This is a Django specific setting which is used to provide a
654
seed in secret-key hashing algorithms. Set this to a random string of your
Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
655
choice and keep it private:
656
657
658

.. code-block:: console

659
    SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
660

661
For Astakos specific configuration, edit the following options in
662
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
663
664
665

.. code-block:: console

666
    ASTAKOS_COOKIE_DOMAIN = '.example.com'
667

Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
668
    ASTAKOS_BASE_URL = 'https://node1.example.com/astakos'
669

670
The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all
671
services). ``ASTAKOS_BASE_URL`` is the Astakos top-level URL. Appending an
672
673
extra path (``/astakos`` here) is recommended in order to distinguish
components, if more than one are installed on the same machine.
674

675
676
.. note:: For the purpose of this guide, we don't enable recaptcha authentication.
    If you would like to enable it, you have to edit the following options:
677
678
679
680
681
682
683
684
685
686

    .. code-block:: console

        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
        ASTAKOS_RECAPTCHA_USE_SSL = True
        ASTAKOS_RECAPTCHA_ENABLED = True

    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
    go to https://www.google.com/recaptcha/admin/create and create your own pair.
687

688
689
690
691
Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :

.. code-block:: console

692
    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
693

Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
694
    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/astakos/ui/get_services'
695

Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
696
    CLOUDBAR_MENU_URL = 'https://node1.example.com/astakos/ui/get_menu'
697

698
699
700
Those settings have to do with the black cloudbar endpoints and will be
described in more detail later on in this guide. For now, just edit the domain
to point at node1 which is where we have installed Astakos.
701

702
703
If you are an advanced user and want to use the Shibboleth Authentication
method, read the relative :ref:`section <shibboleth-auth>`.
Sofia Papagiannaki's avatar
Sofia Papagiannaki committed
704

705
.. _email-configuration:
706
707
708
709

Email delivery configuration
----------------------------

710
711
712
713
Many of the ``Astakos`` operations require the server to notify service users
and administrators via email. e.g. right after the signup process, the service
sents an email to the registered email address containing an verification url.
After the user verifies the email address, Astakos once again needs to
714
notify administrators with a notice that a new account has just been verified.
715

716
More specifically Astakos sends emails in the following cases
717
718

- An email containing a verification link after each signup process.
719
720
721
- An email to the people listed in ``ADMINS`` setting after each email
  verification if ``ASTAKOS_MODERATION`` setting is ``True``. The email
  notifies administrators that an additional action is required in order to
722
  activate the user.
723
- A welcome email to the user email and an admin notification to ``ADMINS``
724
  right after each account activation.
725
- Feedback messages submited from Astakos contact view and Astakos feedback
726
  API endpoint are sent to contacts listed in ``HELPDESK`` setting.
727
- Project application request notifications to people included in ``HELPDESK``
728
  and ``MANAGERS`` settings.
729
- Notifications after each project members action (join request, membership
730
731
  accepted/declinde etc.) to project members or project owners.

732
733
734
Astakos uses the Django internal email delivering mechanism to send email
notifications. A simple configuration, using an external smtp server to
deliver messages, is shown below. Alter the following example to meet your
735
smtp server characteristics. Notice that the smtp server is needed for a proper
736
737
738
installation.

Edit ``/etc/synnefo/00-snf-common-admins.conf``:
739
740

.. code-block:: python
741

742
    EMAIL_HOST = "mysmtp.server.example.com"
743
744
745
746
    EMAIL_HOST_USER = "<smtpuser>"
    EMAIL_HOST_PASSWORD = "<smtppassword>"

    # this gets appended in all email subjects
747
    EMAIL_SUBJECT_PREFIX = "[example.com] "
748

749
    # Address to use for outgoing emails
750
    DEFAULT_FROM_EMAIL = "server@example.com"
751

752
    # Email where users can contact for support. This is used in html/email
753
    # templates.
754
    CONTACT_EMAIL = "server@example.com"
755
756

    # The email address that error messages come from
757
    SERVER_EMAIL = "server-errors@example.com"
758
759

Notice that since email settings might be required by applications other than
760
761
Astakos, they are defined in a different configuration file than the one
previously used to set Astakos specific settings.
762

763
Refer to
764
`Django documentation <https://docs.djangoproject.com/en/1.4/topics/email/>`_
765
766
for additional information on available email settings.

767
As refered in the previous section, based on the operation that triggers
768
an email notification, the recipients list differs. Specifically, for
769
770
emails whose recipients include contacts from your service team
(administrators, managers, helpdesk etc) synnefo provides the following
771
settings located in ``00-snf-common-admins.conf``:
772
773
774

.. code-block:: python

775
776
777
778
    ADMINS = (('Admin name', 'admin@example.com'),
              ('Admin2 name', 'admin2@example.com))
    MANAGERS = (('Manager name', 'manager@example.com'),)
    HELPDESK = (('Helpdesk user name', 'helpdesk@example.com'),)
779

780
781
782
783
784
Alternatively, it may be convenient to send e-mails to a file, instead of an actual smtp server, using the file backend. Do so by creating a configuration file ``/etc/synnefo/99-local.conf`` including the folowing:

.. code-block:: python

    EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
785
    EMAIL_FILE_PATH = '/tmp/app-messages'
786
787


788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
Enable Pooling
--------------

This section can be bypassed, but we strongly recommend you apply the following,
since they result in a significant performance boost.

Synnefo includes a pooling DBAPI driver for PostgreSQL, as a thin wrapper
around Psycopg2. This allows independent Django requests to reuse pooled DB
connections, with significant performance gains.

To use, first monkey-patch psycopg2. For Django, run this before the
``DATABASES`` setting in ``/etc/synnefo/10-snf-webproject-database.conf``:

.. code-block:: console

803
804
    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
    monkey_patch_psycopg2()
805

806
807
Since we are running with greenlets, we should modify psycopg2 behavior, so it
works properly in a greenlet context:
808
809
810

.. code-block:: console

811
812
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
    make_psycopg_green()
813
814
815
816

Use the Psycopg2 driver as usual. For Django, this means using
``django.db.backends.postgresql_psycopg2`` without any modifications. To enable
connection pooling, pass a nonzero ``synnefo_poolsize`` option to the DBAPI
817
driver, through ``DATABASES.OPTIONS`` in Django.
818
819
820
821
822
823

All the above will result in an ``/etc/synnefo/10-snf-webproject-database.conf``
file that looks like this:

.. code-block:: console

824
825
826
    # Monkey-patch psycopg2
    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
    monkey_patch_psycopg2()
827

828
829
830
    # If running with greenlets
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
    make_psycopg_green()
831

832
833
834
    DATABASES = {
     'default': {
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
835
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
836
         'OPTIONS': {'synnefo_poolsize': 8},
837
838
839

         # ATTENTION: This *must* be the absolute path if using sqlite3.
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
840
841
842
843
         'NAME': 'snf_apps',
         'USER': 'synnefo',                      # Not used with sqlite3.
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
         # Set to empty string for localhost. Not used with sqlite3.
844
         'HOST': '203.0.113.1',
845
846
847
         # Set to empty string for default. Not used with sqlite3.
         'PORT': '5432',
     }
848
    }
Christos Stavrakakis's avatar
Christos Stavrakakis committed
849

850
851
852
Database Initialization
-----------------------

853
After configuration is done, we initialize the database by running:
854

855
856
.. code-block:: console

857
    # snf-manage syncdb
858
859
860

At this example we don't need to create a django superuser, so we select
``[no]`` to the question. After a successful sync, we run the migration needed
861
for Astakos:
862
863
864

.. code-block:: console

865
    # snf-manage migrate im
866
    # snf-manage migrate quotaholder_app
867
    # snf-manage migrate oa2
868

869
Then, we load the pre-defined user groups
Sofia Papagiannaki's avatar
Sofia Papagiannaki committed
870
871
872

.. code-block:: console

873
    # snf-manage loaddata groups
Sofia Papagiannaki's avatar
Sofia Papagiannaki committed
874

875
876
877
878
879
.. _services-reg:

Services Registration
---------------------

880
When the database is ready, we need to register the services. The following
881
882
883
command will ask you to register the standard Synnefo components (Astakos,
Cyclades and Pithos) along with the services they provide. Note that you
have to register at least Astakos in order to have a usable authentication
884
885
886
887
888
system. For each component, you will be asked to provide two URLs: its base
URL and its UI URL.

The former is the location where the component resides; it should equal
the ``<component_name>_BASE_URL`` as specified in the respective component
889
settings. For example, the base URL for Astakos would be
890
891
892
893
894
895
``https://node1.example.com/astakos``.

The latter is the URL that appears in the Cloudbar and leads to the
component UI. If you want to follow the default setup, set
the UI URL to ``<base_url>/ui/`` where ``base_url`` the component's base
URL as explained before. (You can later change the UI URL with
896
``snf-manage component-modify <component_name> --ui-url new_ui_url``.)
897
898

The command will also register automatically the resource definitions
899
offered by the services.
900
901
902

.. code-block:: console

903
    # snf-component-register
904
905
906
907

.. note::

   This command is equivalent to running the following series of commands;
908
   it registers the three components in Astakos and then in each host it
909
   exports the respective service definitions, copies the exported json file
910
   to the Astakos host, where it finally imports it:
911
912
913

    .. code-block:: console

914
915
916
       astakos-host$ snf-manage component-add astakos --base-url astakos_base_url --ui-url astakos_ui_url
       astakos-host$ snf-manage component-add cyclades --base-url cyclades_base_url --ui-url cyclades_ui_url
       astakos-host$ snf-manage component-add pithos --base-url pithos_base_url --ui-url pithos_ui_url
917
918
919
920
921
922
923
924
       astakos-host$ snf-manage service-export-astakos > astakos.json
       astakos-host$ snf-manage service-import --json astakos.json
       cyclades-host$ snf-manage service-export-cyclades > cyclades.json
       # copy the file to astakos-host
       astakos-host$ snf-manage service-import --json cyclades.json
       pithos-host$ snf-manage service-export-pithos > pithos.json
       # copy the file to astakos-host
       astakos-host$ snf-manage service-import --json pithos.json
925

926
Notice that in this installation astakos and cyclades are in node1 and pithos is in node2.
927

928
929
930
931
Setting Default Base Quota for Resources
----------------------------------------

We now have to specify the limit on resources that each user can employ
932
(exempting resources offered by projects). When specifying storage or
933
934
935
memory size limits you can append a unit to the value, i.e. 10240 MB,
10 GB etc. Use the special value ``inf``, if you don't want to restrict a
resource.
936
937
938

.. code-block:: console

939
    # snf-manage resource-modify cyclades.vm --system-default 2
940

941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
Setting Resource Visibility
---------------------------

It is possible to control whether a resource is visible to the users via the
API or the Web UI. The default value for these options is denoted inside the
default resource definitions. Note that the system always checks and
enforces resource quota, regardless of their visibility. You can inspect the
current status with::

   # snf-manage resource-list

You can change a resource's visibility with::

   # snf-manage resource-modify <resource> --api-visible=True (or --ui-visible=True)

956
957
958
959
960
.. _pithos_view_registration:

Register pithos view as an OAuth 2.0 client
-------------------------------------------

961
Starting from Synnefo version 0.15, the pithos view, in order to get access to
962
963
the data of a protected pithos resource, has to be granted authorization for
the specific resource by astakos.
964
965

During the authorization grant procedure, it has to authenticate itself with
966
967
astakos since the latter has to prevent serving requests by
unknown/unauthorized clients.
968

969
970
971
972
973
974
Each oauth 2.0 client is identified by a client identifier (client_id).
Moreover, the confidential clients are authenticated via a password
(client_secret).
Then, each client has to declare at least a redirect URI so that astakos will
be able to validate the redirect URI provided during the authorization code
request.
975
If a client is trusted (like a pithos view), astakos grants access on behalf
976
977
of the resource owner, otherwise the resource owner has to be asked.

978
979
980
981
To register the pithos view as an OAuth 2.0 client in astakos, we have to run
the following command::

    snf-manage oauth2-client-add pithos-view --secret=<secret> --is-trusted --url https://node2.example.com/pithos/ui/view
982
983
984
985
986
987
988
989

Servers Initialization
----------------------

Finally, we initialize the servers on node1:

.. code-block:: console

990
991
    root@node1:~ # /etc/init.d/gunicorn restart
    root@node1:~ # /etc/init.d/apache2 restart
992
993

We have now finished the Astakos setup. Let's test it now.
994
995
996
997
998
999
1000


Testing of Astakos
==================

Open your favorite browser and go to:

Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
1001
``http://node1.example.com/astakos``
1002

Giorgos Korfiatis's avatar
Giorgos Korfiatis committed
1003
If this redirects you to ``https://node1.example.com/astakos/ui/`` and you can see
1004
1005