Skip to content

GitLab

Commit b7566dcc authored by Leonidas Poulopoulos's avatar Leonidas Poulopoulos
Browse files

Update docs for a new release

parent 9c4124cb
Hide whitespace changes
Inline Side-by-side
Showing with 154 additions and 343 deletions
ChangeLog
View file @ b7566dcc
===========
1.0.0 RELEASE
Major UI redesign, Debian Wheezy version, Django 1.4
Improvements:
-New UI based on Bootstrap3 theme
-Minor fixes in long polling init
-Debian Wheezy - Django 1.4 ready
===========
0.9.9 RELEASE
Major documentation improvements. Minor app enhancements. Minor bug fixes
......
doc/source/conf.py
View file @ b7566dcc
......@@ -38,16 +38,16 @@ master_doc = 'index'
# General information about the project.
project = u'fod'
copyright = u'2013, Leonidas Poulopoulos'
copyright = u'2014, Leonidas Poulopoulos (@leopoul), GRNET S.A'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '0.9.9'
version = '1.0.0'
# The full version, including alpha/beta/rc tags.
release = '0.9.9'
release = '1.0.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......
doc/source/index.rst
View file @ b7566dcc
......@@ -29,15 +29,9 @@ In GRNET's case the flowspec capable device is an EX4200.
.. attention::
Make sure your FoD server has ssh access to your flowspec device.
Installation Considerations
===========================
You can find the installation instructions for Ubuntu 12.04.3 (64) with Django 1.3.x here: :doc:`Install FoD <install>`.
FoD depends on a bunch of packages. Installing in Debian Squeeze proved to be really tough as the majority of the required packages are not provided by any repos and need to be installed manually. This guide presents the installation procedures for Ubuntu 12.04.3 (64) with Django 1.3.x. Really soon, we will provide a guide for Debian Wheezy.
However, users who wish to go for Wheezy, need to read the Django 1.4 changelist. One of the most significant changes in Django 1.4 is that the application dir layout has to be restructured.
Also bear in mind that Django 1.4 introduces new aspects when it comes to application library inclussions.
Soon we will post a branch of FoD tailored for Django 1.4.
.. attention::
Installation instructions assume a Debian Wheezy installation with Django 1.4
Contact
=======
You can find more about FoD or raise your issues at `GRNET FoD repository <https://code.grnet.gr/projects/flowspy>`_.
......
doc/source/install.rst
View file @ b7566dcc
......@@ -5,7 +5,7 @@ Installation
.. toctree::
:maxdepth: 2
Ubuntu 12.04.3 (64) - Django 1.3.x
Debian Wheezy (x64) - Django 1.4.x
==================================
This guide assumes that installation is carried out in /srv/flowspy directory. If other directory is to be used, please change the corresponding configuration files. It is also assumed that the root user will perform every action.
......@@ -17,7 +17,7 @@ Update and install the required packages::
apt-get update
apt-get upgrade
apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-extensions python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
.. note::
Set username and password for mysql if used
......@@ -60,7 +60,8 @@ Application configuration
=========================
Copy settings.py.dist to settings.py::
cd flowspy
cp settings.py.dist settings.py
Edit settings.py file and set the following according to your configuration::
......@@ -68,9 +69,10 @@ Edit settings.py file and set the following according to your configuration::
ADMINS: set your admin name and email (assuming that your server can send notifications)
DATABASES (to point to your local database). You could use views instead of tables for models: peer, peercontacts, peernetworks. For this to work we suggest MySQL with MyISAM db engine
SECRET_KEY : Make this unique, and don't share it with anybody
STATIC_ROOT: /srv/flowspy/static (or your installation directory)
STATIC_URL (static media directory) . If you have followed the above this should be: /srv/flowspy/static
TEMPLATE_DIRS : If you have followed the above this should be: /srv/flowspy/templates
CACHE_BACKEND: If you have followed the above this should be: memcached://127.0.0.1:11211/?timeout=3600
CACHE_BACKEND: Enable Memcached for production or leave to DummyCache for development environments
Alternatively you could go for redis with the corresponding Django client lib.
NETCONF_DEVICE (tested with Juniper EX4200 but any BGP enabled Juniper should work). This is the flowspec capable device
NETCONF_USER (enable ssh and netconf on device)
......@@ -108,11 +110,12 @@ If you have not installed an outgoing mail server you can always use your own ac
.. note::
Soon we will release a version with django-registration as a means to add users and Shibboleth as an alternative
Soon we will release a version with django-registration as a means to add users and Shibboleth will become an alternative
Let's move on with some copies and dir creations::
cp urls.py.dist urls.py
cd ..
mkdir log
chown -R root:www-data log/
chmod -R g+w log
......@@ -156,183 +159,25 @@ create and edit /etc/gunicorn.d/fod::
FoD is served via gunicorn and is then proxied by Apache. If the above directory conventions have been followed so far, then your configuration should be::
CONFIG = {
'mode': 'django',
'working_dir': '/srv/flowspy',
'args': (
'--bind=127.0.0.1:8081',
'--workers=1',
'--timeout=360',
'--worker-class=egg:gunicorn#gevent',
'--log-level=debug',
'settings.py',
),
}
CONFIG = {
'mode': 'django',
'working_dir': '/srv/flowspy',
'args': (
'--bind=127.0.0.1:8081',
'--workers=1',
'--worker-class=egg:gunicorn#gevent',
'--timeout=360',
'--log-level=debug',
'--log-file=/tmp/fod.log',
),
}
celery.d
--------
Celery is used over beanstalkd to apply firewall rules in a serial manner so that locks are avoided on the flowspec capable device. In our setup celery runs via django. That is why the python-django-celery package was installed.
Create the celeryd daemon at /etc/init.d/celeryd::
vim /etc/init.d/celeryd
The configuration should be::
#!/bin/sh -e
# ============================================
# celeryd - Starts the Celery worker daemon.
# ============================================
#
# :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
# :Configuration file: /etc/default/celeryd
#
# See http://docs.celeryq.org/en/latest/cookbook/daemonizing.html#init-script-celeryd
### BEGIN INIT INFO
# Provides: celeryd
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: celery task worker daemon
### END INIT INFO
#set -e
DEFAULT_PID_FILE="/var/run/celeryd@%n.pid"
DEFAULT_LOG_FILE="/var/log/celeryd@%n.log"
DEFAULT_LOG_LEVEL="INFO"
DEFAULT_NODES="celery"
DEFAULT_CELERYD="-m celery.bin.celeryd_detach"
# /etc/init.d/celeryd: start and stop the celery task worker daemon.
CELERY_DEFAULTS=${CELERY_DEFAULTS:-"/etc/default/celeryd"}
test -f "$CELERY_DEFAULTS" && . "$CELERY_DEFAULTS"
if [ -f "/etc/default/celeryd" ]; then
. /etc/default/celeryd
fi
CELERYD_PID_FILE=${CELERYD_PID_FILE:-${CELERYD_PIDFILE:-$DEFAULT_PID_FILE}}
CELERYD_LOG_FILE=${CELERYD_LOG_FILE:-${CELERYD_LOGFILE:-$DEFAULT_LOG_FILE}}
CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
CELERYD_MULTI=${CELERYD_MULTI:-"celeryd-multi"}
CELERYD=${CELERYD:-$DEFAULT_CELERYD}
CELERYCTL=${CELERYCTL:="celeryctl"}
CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES}
export CELERY_LOADER
if [ -n "$2" ]; then
CELERYD_OPTS="$CELERYD_OPTS $2"
fi
CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE`
if [ ! -d "$CELERYD_LOG_DIR" ]; then
mkdir -p $CELERYD_LOG_DIR
fi
if [ ! -d "$CELERYD_PID_DIR" ]; then
mkdir -p $CELERYD_PID_DIR
fi
# Extra start-stop-daemon options, like user/group.
if [ -n "$CELERYD_USER" ]; then
DAEMON_OPTS="$DAEMON_OPTS --uid=$CELERYD_USER"
chown "$CELERYD_USER" $CELERYD_LOG_DIR $CELERYD_PID_DIR
fi
if [ -n "$CELERYD_GROUP" ]; then
DAEMON_OPTS="$DAEMON_OPTS --gid=$CELERYD_GROUP"
chgrp "$CELERYD_GROUP" $CELERYD_LOG_DIR $CELERYD_PID_DIR
fi
if [ -n "$CELERYD_CHDIR" ]; then
DAEMON_OPTS="$DAEMON_OPTS --workdir=\"$CELERYD_CHDIR\""
fi
check_dev_null() {
if [ ! -c /dev/null ]; then
echo "/dev/null is not a character device!"
exit 1
fi
}
export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
stop_workers () {
$CELERYD_MULTI stop $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
}
start_workers () {
$CELERYD_MULTI start $CELERYD_NODES $DAEMON_OPTS \
--pidfile="$CELERYD_PID_FILE" \
--logfile="$CELERYD_LOG_FILE" \
--loglevel="$CELERYD_LOG_LEVEL" \
--cmd="$CELERYD" \
$CELERYD_OPTS
}
restart_workers () {
$CELERYD_MULTI restart $CELERYD_NODES $DAEMON_OPTS \
--pidfile="$CELERYD_PID_FILE" \
--logfile="$CELERYD_LOG_FILE" \
--loglevel="$CELERYD_LOG_LEVEL" \
--cmd="$CELERYD" \
$CELERYD_OPTS
}
case "$1" in
start)
check_dev_null
start_workers
;;
stop)
check_dev_null
stop_workers
;;
reload|force-reload)
echo "Use restart"
;;
status)
$CELERYCTL status $CELERYCTL_OPTS
;;
restart)
check_dev_null
restart_workers
;;
try-restart)
check_dev_null
restart_workers
;;
*)
echo "Usage: /etc/init.d/celeryd {start|stop|restart|try-restart|kill}"
exit 1
;;
esac
exit 0
and make it executable::
chmod +x /etc/init.d/celeryd
celeryd requires a /etc/default/celeryd file to be in place.
Thus we are going to create this file (/etc/default/celeryd)::
......@@ -340,35 +185,39 @@ Thus we are going to create this file (/etc/default/celeryd)::
Again if the directory conventions have been followed the file is (pay attention to the CELERYD_USER, CELERYD_GROUP and change accordingly) ::
# Name of nodes to start, here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"
# Where to chdir at start.
CELERYD_CHDIR="/srv/flowspy"
# How to call "manage.py celeryd_multi"
CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"
# Default: false
ENABLED="true"
# Name of nodes to start, here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"
# Where to chdir at start.
CELERYD_CHDIR="/srv/flowspy"
# How to call "manage.py celeryd_multi"
CELERYD_MULTI="python $CELERYD_CHDIR/manage.py celeryd_multi"
# How to call "manage.py celeryctl"
CELERYCTL="python $CELERYD_CHDIR/manage.py celeryctl"
# Extra arguments to celeryd
#CELERYD_OPTS="--time-limit=300 --concurrency=8"
CELERYD_OPTS="-E -B"
# Name of the celery config module.
CELERY_CONFIG_MODULE="celeryconfig"
# %n will be replaced with the nodename.
CELERYD_LOG_FILE="$CELERYD_CHDIR/celery_var/log/celery/%n.log"
CELERYD_PID_FILE="$CELERYD_CHDIR/celery_var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
CELERYD_USER="user"
CELERYD_GROUP="users"
# Name of the projects settings module.
export DJANGO_SETTINGS_MODULE="flowspy.settings"
# How to call "manage.py celeryctl"
CELERYCTL="$CELERYD_CHDIR/manage.py celeryctl"
# Extra arguments to celeryd
#CELERYD_OPTS="--time-limit=300 --concurrency=8"
CELERYD_OPTS="-E -B"
# Name of the celery config module.
CELERY_CONFIG_MODULE="celeryconfig"
# %n will be replaced with the nodename.
CELERYD_LOG_FILE="$CELERYD_CHDIR/celery_var/log/celery/%n.log"
CELERYD_PID_FILE="$CELERYD_CHDIR/celery_var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
CELERYD_USER="user"
CELERYD_GROUP="user"
# Name of the projects settings module.
export DJANGO_SETTINGS_MODULE="settings"
Apache
------
......@@ -378,127 +227,90 @@ Apache proxies gunicorn. Things are more flexible here as you may follow your ow
Again if the directory conventions have been followed the file should be::
<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName <server_name>
DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^/(.*) https://<server_name>/$1 [L,R]
</VirtualHost>
<VirtualHost *:443>
ServerName <server_name>
ServerAdmin <admin_mail>
ServerSignature On
SSLEngine on
SSLCertificateFile /etc/ssl/certs/example.com.crt
SSLCertificateChainFile /etc/ssl/certs/example.com.crt
SSLCertificateKeyFile /etc/ssl/private/example.com.key
AddDefaultCharset UTF-8
IndexOptions +Charset=UTF-8
# Shibboleth configuration
ShibConfig /etc/shibboleth/shibboleth2.xml
Alias /shibboleth-sp /usr/share/shibboleth
<Location /fod/login>
AuthType shibboleth
ShibRequireSession On
ShibUseHeaders On
require valid-user
</Location>
# Shibboleth debugging CGI script
ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
<Location /shibboleth/test>
AuthType shibboleth
ShibRequireSession On
ShibUseHeaders On
require valid-user
</Location>
<Location /Shibboleth.sso>
SetHandler shib
</Location>
# End of Shibboleth configuration
<Location /admin/media/>
SetHandler None
</Location>
Alias /admin/media /usr/share/pyshared/django/contrib/admin/media
Alias /media /usr/share/pyshared/django/contrib/admin/media
DocumentRoot /var/www
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
SSLProxyEngine off
ProxyErrorOverride off
ProxyTimeout 28800
ProxyPass /fod http://localhost:8081/fod retry=0
ProxyPassReverse /fod http://localhost:8081/fod
LogLevel warn
ErrorLog /var/log/apache2/ssl-error.log
CustomLog /var/log/apache2/ssl-access.log combined
Alias /fodstatic /srv/flowspy/static
</VirtualHost>
<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName fod.example.com
DocumentRoot /var/www
ErrorLog ${APACHE_LOG_DIR}/fod_error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel debug
CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
Alias /static /srv/flowspy/static
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^/(.*) https://fod.example.com/$1 [L,R]
</VirtualHost>
<VirtualHost *:443>
ServerName fod.example.com
ServerAdmin webmaster@localhost
ServerSignature On
SSLEngine on
SSLCertificateFile /etc/ssl/certs/fod.example.com.crt
SSLCertificateChainFile /etc/ssl/certs/example-chain.pem
SSLCertificateKeyFile /etc/ssl/private/fod.example.com.key
AddDefaultCharset UTF-8
IndexOptions +Charset=UTF-8
ShibConfig /etc/shibboleth/shibboleth2.xml
Alias /shibboleth-sp /usr/share/shibboleth
<Location /login>
AuthType shibboleth
ShibRequireSession On
ShibUseHeaders On
ShibRequestSetting entityID https://idp.example.com/idp/shibboleth
require valid-user
</Location>
# Shibboleth debugging CGI script
ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
<Location /shibboleth/test>
AuthType shibboleth
ShibRequireSession On
ShibUseHeaders On
require valid-user
</Location>
<Location /Shibboleth.sso>
SetHandler shib
</Location>
# Shibboleth SP configuration
#SetEnv proxy-sendchunked
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
SSLProxyEngine off
ProxyErrorOverride off
ProxyTimeout 28800
ProxyPass /static !
ProxyPass /shibboleth !
ProxyPass /Shibboleth.sso !
ProxyPass / http://localhost:8081/ retry=0
ProxyPassReverse / http://localhost:8081/
Alias /static /srv/flowspy/static
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/fod_error.log
CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
</VirtualHost>
Now, enable your site. You might want to disable the default site if fod is the only site you host on your server::
......@@ -517,18 +329,14 @@ Restart, gunicorn and apache::
Testing the platform
====================
Log in to the admin interface via https://<your ip>/fod/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 83.212.0.0/19
Log in to the admin interface via https://<hostname>/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 10.20.0.0/19
Go to Peers and add a new peer, eg. id: 1, name: Test, AS: 16503, tag: TEST and move the network you have crteated from Avalable to Chosen. From the admin front, go to User, and edit your user. From the bottom of the page, select the TEST peer and save.
Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need for Shibboleth. Go to https://<your ip>/fod/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need for Shibboleth. Go to https://<hostname>/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
Branding
========
Via the admin interface you can modify flatpages to suit your needs
Logos
-----
Inside the static folder you will find two empty png files: fod_logo.xcf (Gimp file) and shib_login.dist.png. Edit those two with your favourite image processing software and save them as fod_logo.png (under static/img/) and shib_login.png (under static/). Image sizes are optimized to operate without any other code changes. In case you want to incorporate images of different sizes you have to fine tune css and/or html as well.
Footer
------
Under the templates folder (templates), you can alter the footer.html file to include your own footer messages, badges, etc.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment