cyclades-upgrade.rst 12.4 KB
Newer Older
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
1
2
Upgrade
=======
3
4

This file documents the upgrade to newer versions of the Synnefo software.
5
For more information, please see deployment guide.
6

7
8
9

v0.7.4 -> v0.8
--------------
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
10

Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
11
Synnefo is now distributed packaged in python/debian packages. You should
12
13
consider the following when migrating from projects previously deployed from
source:
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
14
    
15
16
    * To keep consistency with future changes, change all ``python manage.py``
      calls to refer to the shipped ``snf-manage`` console script.
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
17
18
19
20
      This includes automation scripts, service scripts etc.

      Same applies for calls to ``dispatcher.py``, ``snf-tools/admin.py``,
      ``snf-tools/burnin.py`` and ``snf-tools/cloud.py`` which are replaced
21
22
      by ``snf-dispatcher``, ``snf-admin``, ``snf-burnin`` and ``snf-cloud``,
      respectively.
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
23

24
    * Copy custom settings files located in project's ``settings.d`` directory
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
25
26
27
28
      to ``/etc/synnefo/`` directory.

    * Migrate location from which :ref:`static files <static-files>` are served from.

29
30
31
32
33
34
35
36
.. warning::
   
   Synnefo settings have been refactored as part of the
   :ref:`snf-common <snf-common>` component. File locations may be inaccurate
   and the list of changed settings incomplete.  Please consult the actual
   files installed under ``/etc/synnefo`` as the authoritative source for new
   settings.

37
NEW APPS
38
39
40
41
    * The ``synnefo.ui.userdata`` application has been added in
      :file:`settings.d/00-apps.conf`. Application urls appended in
      :file:`ui/urls.py`.
      If no custom ``ROOT_URLCONF`` module is used, no url change is needed.
42
    * The new app has migrations defined.
43
      Use ``snf-manage migrate`` to migrate *all* apps.
44

45
NEW/UPDATED SETTINGS
46
47
48
49
    * ``BYPASS_AUTHENTICATION_TOKEN_SECRET`` replaces ``BYPASS_AUTHENTICATION_TOKEN``
      in :file:`settings/common/aai.py`.
    * New config file :file:`31-userdata.conf`, containing userdata app settings
    * ``USERDATA_SSH_KEY_LENGTH`` in :file:`31-userdata.conf`:
50
      Key length in bits for generated ssh keys
51
    * ``USERDATA_SSH_KEY_EXPONENT`` in :file:`31-userdata.conf`:
52
      Generated SSH key exponent
53
    * ``USERDATA_MAX_SSH_KEYS_PER_USER`` in :file:`31-userdata.conf`:
54
      Maximum number of ssh keys a user is allowed to have
55
    * ``UI_SUPPORT_SSH_OS_LIST``, in :file:`30-ui.conf`:
56
      A list of os names that support ssh public key assignment
57
    * ``UI_OS_DEFAULT_USER_MAP``, in :file:`30-ui.conf`:
58
      OS/username map to identify default user name for a specific os
59
    * ``VM_CREATE_NAME_TPL``, in :file:`30-ui.conf`:
60
61
62
      Template to be used for suggesting the user a default name for newly
      created VMs. String {0} gets replaced by the value of metadata key "os"
      for the Image.
63
    * ``UI_FLAVORS_DISK_TEMPLATES_INFO`` added in :file:`30-ui.conf`:
64
      Name/description metadata for the available flavor disk templates
65
    * ``VM_CREATE_SUGGESTED_FLAVORS`` in :file:`30-ui.conf`:
66
      Updated flavor data to include disk_template value.
67
    * ``GANETI_DISK_TEMPLATES`` and ``DEFAULT_GANETI_DISK_TEMPLATE`` in :file:`20-api.conf`:
68
69
70
      The list of disk templates supported by the Ganeti backend.
      The default template to be used when migrating Flavors with no value for
      disk_template (i.e., 'drbd').
71
    * ``UI_COMPUTE_URL``, ``UI_GLANCE_URL`` in ui app:
72
      Configurable API endpoints
73
    * ``UI_ENABLE_GLANCE`` in ui app:
74
      Whether or not UI should retrieve images from the Glance API endpoint
75
76
      set in ``UI_GLANCE_API_URL``. If setting is set to False, ui will request
      images using calls to the Cyclades API.
77
UI
78
    * ``synnefo.ui.userdata`` application has been added in ``INSTALLED_APPS``.
79
      Database migration is needed for the creation of application db tables.
80

81
API
82
83
84
85
    * A new ``disk_template`` attribute has been added to Flavors.
      ``GANETI_DISK_TEMPLATES`` and ``DEFAULT_GANETI_DISK_TEMPLATE`` have been
      added in :file:`20-api.conf` to control its value. A database migration is
      needed.
86

87
88
89
90
91
92
93
PLANKTON
    Plankton is a new image service that has been added as a separate app. The
    images are stored in backend of Pithos and thus it must be configured to
    access the DB and directory that Pithos uses to store its data. These
    settings are defined in the new plankton.py file in settings.
    
    Plankton stores and looks for images in the Pithos container named
94
    ``PITHOS_IMAGE_CONTAINER``.
95
96
    
    There is a Pithos account that is reserved for system images. This account
97
    is defined in ``SYSTEM_IMAGES_OWNER``.
98

99
ADMIN TOOLS
100
101
    * A new ``--disk-template`` flag has been added to snf-admin to choose a
      disk template when creating flavors. Similarly, ``disk_template`` support
102
103
      in flavors has beed added to the admin web interface.

104

105
v0.7.3 -> v0.7.4
106
----------------
107

108
109
110
OKEANOS INTRO
    * News section added. News content can be remotely retrieved using
      OKEANOS_NEWS_CONTENT_URL settings option.
111

112

113
v0.6.2 -> v0.7
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
114
115
--------------

116
117
118
119
HTML TEMPLATES
    * Included a generic service unavailable template based on
      generic_info_tpl.html, as ui/static/service_unavailable.html.

120
121
NEW DEPENDENCIES
    * python-unittest2, python-paramiko needed by the new integration
122
      test suite snf-tools/burnin. Paramiko shipped with Squeeze uses
123
      broken RandomPool, produces warnings, see #576697.
124
125
    * snf-image must be installed on all Ganeti hosts, see README.deploy,
      Step 13.
126

127
COMPONENTS
Vangelis Koukis's avatar
Vangelis Koukis committed
128
129
    * snf-admin has been updated with new functionality, be sure to upgrade any
      locally installed versions.
130
131
132
133
134
135
    * snf-image replaces snf-ganeti-instance-image as the Ganeti OS provider
      used by Synnefo, and can live alongside snf-ganeti-instance-image.
      Once snf-image has been deployed on all Ganeti nodes, be sure to modify
      the value of settings.d/10-backend.conf:GANETI_CREATEINSTANCE_KWARGS
      to use 'snf-image+default'.

136
137
138
139
140
141
142
143
144
145
UI STATIC FILES
    * A major reorganization of static files has been commited. All ui and
      invitations static files has been moved in their own separate directory
      (ui/static/snf, ui/static/invitations).
    * UI static files base url is now configurable via UI_MEDIA_URL.
    * A symlink named after the current version of the ui application has been
      committed in ui/static directory. The symlink will get updated after each
      version upgrade to allow us provide unique urls of static files between
      after each upgrade, see #1460.

146
NEW/UPDATED SETTINGS
147
148
149
    * EMAIL_SUBJECT_PREFIX:
      Prepended to automated emails, set to someting that uniquely identifies
      the deployment.
Vangelis Koukis's avatar
Vangelis Koukis committed
150
151
152
    * MAX_PERSONALITY and MAX_PERSONALITY_SIZE in 20-api.conf:
      Maximum number of files to be injected in newly created servers,
      maximum total size of encoded file contents.
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
    * Renamed SUGGESTED_FLAVORS to VM_CREATE_SUGGESTED_FLAVORS in 30-ui.conf
    * VM_CREATE_SUGGESTED_ROLES in 30-ui.conf:
      A list of suggested vm roles to display to user on create wizard.
    * UI_DELAY_ON_BLUR in 30-ui.conf:
      Whether to increase the intervals of recurrent requests (networks/vms 
      update) if window loses its focus.
    * UI_BLUR_DELAY in 30-ui.conf:
      The value of update intervals if window loses its focus.
      Considered only if `UI_DELAY_ON_BLUR` is set to True.
    * UI_UPDATE_HIDDEN_VIEWS in 30-ui.conf:
      Whether not visible vm views will update their content if vm changes.
    * UI_SKIP_TIMEOUTS in 30-ui.conf:
      After how many timeouts of reccurent ajax requests to display the timeout
      error overlay.
    * UI_HANDLE_WINDOW_EXCEPTIONS in 30-ui.conf:
      Whether UI should display error overlay for all Javascript exceptions.
169
170
171
172
173
174
175
176
    * UI_MEDIA_URL in 30-ui.conf:
      Base url for ui static files, 
      defaults to MEDIA_URL + 'snf-<latest_ui_version>/'.
    * MEDIA_URL changed in 00-site.conf:
      Changed to '/static/' since it is now used in ui app.
    * TEMPLATE_CONTEXT_PROCESSORS changed in 00-apps.conf:
      added 'django.core.context_processors.media' to allow access of MEDIA_URL
      in template files.
177
178
    * GANETI_CREATEINSTANCE_KWARGS in 10-backend.conf:
      Must be updated to use snf-image as the Ganeti OS provider.
179
      
180
    
Vangelis Koukis's avatar
Vangelis Koukis committed
181
DB MIGRATION
182
183
    * Uniqueness constraints have been added to the metadata models.
      A database migration is needed.
184

185
LOGGING
Vangelis Koukis's avatar
Vangelis Koukis committed
186
187
188
    * A new logging mechanism has been implemeted. Please see 00-logging.conf
      under settings.d/ and read the relevant section in README.admin for more
      info.
189

190

Giorgos Verigakis's avatar
Giorgos Verigakis committed
191
v0.6.1 -> v0.6.2
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
192
193
----------------

194
ADMIN INTERFACE
Giorgos Verigakis's avatar
Giorgos Verigakis committed
195
196
197
198
199
    * The models were changed so that Flavors and SynnefoUsers now have
      a deleted state. The admin tools were updated so that models are
      now marked as deleted instead of actually being deleted from the DB.
      A database migration is needed.

200
201
202
203
COMPONENTS
    * Only the Django webapp is affected, must restart the logic dispatcher
      due to DB migration taking place.

Giorgos Verigakis's avatar
Giorgos Verigakis committed
204

205
v0.5.5 -> v0.6
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
206
207
--------------

208
209
210
211
212
213
ADMIN INTERFACE
    * A new Web-based admin interface is available under /admin.
      It is accessible by users of type 'ADMIN' in the DB, with
      their authentication token. "snf-admin user modify" may be used
      to change the type of a specific user.

214
215
RECONCILIATION
    * Implemented new reconciliation management command, please see
216
      ./manage.py reconcile --help and README.admin for more info.
217
218
219
      Recommended to run ./manage.py reconcile --detect-all periodically,
      via cron.

220
221
222
GANETI-INSTANCE-IMAGE
    * A new version of ganeti-instance-image is required (v0.5.1-1-snf1).
      The new version is available for download as a debian package:
223
      https://code.grnet.gr/projects/gnt-instance-image/files
224

225
COMPONENTS
226
    * snf-cyclades-gtools must be rebuilt, see snf-cyclades-gtools/debian/
227
228
      for Debian packaging.

229
230
NEW SETTINGS
    * 30-ui.conf:SUGGESTED_FLAVORS
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
231
    * 30-ui.conf:VM_IMAGE_COMMON_METADATA
232

233

234
v0.5.4 -> v0.5.5
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
235
236
----------------

237
238
239
LOGGING
    * Changed the default logging settings for the dispatcher to also log
      to /var/log/synnefo/dispatcher.log, redirecting stderr and stdout there
240

Georgios Gousios's avatar
Georgios Gousios committed
241
242
243
244
QUEUES
    * Changed default routing key naming for queues. Queues must be redeclared.
      The suggested upgrade path is to delete ALL (even obsolete) existing
      queues and restart the dispatcher. To do so, download amqp-utils from
245

Georgios Gousios's avatar
Georgios Gousios committed
246
247
248
249
          https://github.com/dougbarth/amqp-utils

      and run the amqp-deleteq tool for each declared queue. Alternatively,
      amqp-utils can be installed with: sudo gem install amqp-utils.
250

251
252
253
254
UI
    * Feedback form now uses django native send_mail for sending emails.
      Proper django settings should be set for feedback mails to work
      (https://docs.djangoproject.com/en/dev/topics/email/)
255

Vangelis Koukis's avatar
Vangelis Koukis committed
256
COMPONENTS
257
    * snf-cyclades-gtools must be rebuilt, see snf-cyclades-gtools/debian/
Vangelis Koukis's avatar
Vangelis Koukis committed
258
      for Debian packaging.
259

Vangelis Koukis's avatar
Vangelis Koukis committed
260

261
v0.5.3 -> v0.5.4
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
262
263
----------------

264
REPOSITORY
265
    * Split Synnefo Ganeti tools to snf-cyclades-gtools, with Debian packaging
266
267
268
269
270

REMOVED APPS
    * The ganeti/ app has been removed from the Django project completely.
      Any explicit references to it in Django settings must be removed.

271
DJANGO SETTINGS
272
    * snf-cyclades-gtools is configured independently from Django,
273
      need to add proper /etc/synnefo/settings.conf
274
    * Removed 15-queues.conf: fix_amqp_settings (no need to call it anywhere)
275
    * Removed settings.d/98-ganeti-* due to split of snf-cyclades-gtools
276
277
    * ~okeanos intro: OKEANOS_VIDEO_URL: Changed from string to dict
    * ~okeanos intro: OKEANOS_VIDEO_POSTER_IMAGE_URL: New setting
278
    * ~okeanos intro: OKEANOS_VIDEO_FLOWPLAYER_URL: New setting
279
280
281
282

DB MIGRATIONS
    * 0018_auto__add_field_virtualmachine_buildpercentage

283
PACKAGING
284
    * Split Synnefo Ganeti tools to separate snf-cyclades-gtools Debian package
285

286
NEW DEPENDENCIES
287
    * python-prctl: Needed by the snf-progress-monitor,
288
      specified as a dependency of the snf-cyclades-gtools Debian package.
289

290
291
292
293
EXTERNAL COMPONENTS
    * Ganeti Instance Image must be upgraded to support progress monitoring,
      please see README.deploy.

294

Georgios Gousios's avatar
Georgios Gousios committed
295
v0.5.2 -> v0.5.3
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
296
297
----------------

Georgios Gousios's avatar
Georgios Gousios committed
298
299
300
301
302
NEW SETTINGS
    * 30-ui.conf:LOGOUT_URL
    * 00-admins.conf:DEFAULT_FROM_EMAIL
    * 90-okeanos.conf.sample:LOGOUT_URL

303
REMOVED SETTINGS
Georgios Gousios's avatar
Georgios Gousios committed
304
305
306
    * 00-admins.conf:SYSTEM_EMAIL_ADDR
    * 90-okeanos.conf.sample:APP_INSTALL_URL

Georgios Gousios's avatar
Georgios Gousios committed
307

308
v0.5.1 -> v0.5.2
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
309
310
----------------

311
312
313
NEW SETTINGS
    * 10-backend.py:GANETI_CREATEINSTANCE_KWARGS

314
REMOVED SETTINGS
Georgios Gousios's avatar
Georgios Gousios committed
315
316
    * 10-backend.conf:GANETI_OS_PROVIDER
    * 20-api.conf:GANETI_DISK_TEMPLATE
317
318
319
320
321
322

BACKEND CHANGES
    * Need to patch Ganeti, file:
      lib/python2.6/site-packages/ganeti/rapi/rlib2.py
      to honor the wait_for_sync flag, see Synnefo #835.
      Patch provided under contrib/patches/ganeti-rlib2.py-v0.5.2
Kostas Papadimitriou's avatar
Kostas Papadimitriou committed
323