Commit 81cf516b authored by Stavros Sachtouris's avatar Stavros Sachtouris
Browse files

Merge branch 'release-0.12'

Conflicts:
	Changelog
	docs/conf.py
	kamaki/cli/commands/cyclades.py
	kamaki/cli/commands/image.py
	kamaki/cli/commands/pithos.py
	setup.py
	version
parents 0baa648a 0db8331f
CHANGELOG for hotfix 0.11.4
CHANGELOG for version 0.12.rc6
Unicode pithos filenames bug fix
Zero file syncing bug fix
Add self.poolsize for kamaki.clients.Client
CHANGELOG for hotfix 0.11.3
CHANGELOG for version 0.12.rc5
Fix bug with image meta set
Add ip attach/detach (shortcuts for port create/delete)
CHANGELOG for hotfix 0.11.2
CHANGELOG for version 0.12.rc4
Fix an output bug wiuth file permissions get
Fix syntax errors in network commands
CHANGELOG for hotfix 0.11.1
CHANGELOG for version 0.12.rc3
Bug fixes:
Check clients unittests
- Check if personlity mode is always in octal
CHANGELOG for version 0.12rc3
CHANGELOG for version 0.11
Correct keys in HTTP bodies on network-related requests
CHANGELOG for version 0.12
Bug Fixes:
- Make authentication lazy (#4300)
- Stop shell from destroying the config instance
- Shell can manage all valid command line arguments [#3716]
- Restore 2nd level command syntax in shell [#3736]
- Allow copy of deleted objects by refering to older version [#3737]
- Add image.add_member missing content-length header
- Do not unpublish by default in all pithos low level requests [#3780]
- Unquote http respons headers
-In file list, the path or prefix was converted to boolean value. Fixed.
-Thread options did not work [#4616]
Changes:
- Logs do not contain kamaki.clients pids by default [#4242]
- Replace page_hold method with pydoc.pager [#4279]
- Remove commands.pithos.DelimiterArgument, replace with ValueArgument
wherever it is used
- Replace print methods with member print methods in _commands.* [#4292]
- kamaki.cli.config is now a directory package [#4058]
- Modify progress bar handler in cli.commands to allow count down and timeout
[#4352]
- Make wait progress bars in cyclades CLI to count down to timeout instead of
progressing forward [#4352]
- Rename IP commands to new namespace and better names [#4422]
1. Make astakosclient a mantatory requirement for kamaki [#4312]
2. Make post_user_catalogs obsolete, but keep for one more version [#4337]
3. Rename user commands for cached account requests as /user session [#4340]
4. Remove max_theads from config, move control to threaded commands [#4617]
5. Modify all commands [#4583]
New scheme for ALL <object> <verb> [object id] [--arguments]
e.g., file modidy --metadata-add=revier='Mr. Reviewer' /pithos/myfile.txt
6. Deprecate (with note) server stats/console/addr (move to server info)
7. A vanilla kamaki call now shows only the available commands instead of
loading a shell [#4612]
Features:
- New config option log_pid (default: off) can allow pid-logging [#4242]
- Expand kamaki.cli unitests for the following packages ( [#4058] ):
errors
- Modify print methods in cli utils to use arbitary stream objects [#4288]
- Implement wrapers for cli.utils print methods, in _commands [#4292]
- Implement unittests for kamaki.cli.config [#4058]
- Implement --type=SOFT/HARD argument for server reboot [#4338]
The old argument -f/--force <==> --type=HARD is kept for bw compatibility
A warning message guides users to stop using it
- Catch Keyboard Interupts in cli wait methods and exit gracefully [#4351]
- Add -R/--recursive in file listing, to list containers with their contents
[#4301]
- Add -w/--wait to server firewall set [#4298]
- Add --output-format where applicable, keep -j/--json too, for bw
compatibility [#4342]
- Allow metakeys in personality argument [#4312]
1. Cache user catalog calls [#4337]
2. Implement separate methods for uuid2usernames and v.v. [#4337]
3. Move all methods from snf-astakos.py to astakos.py, with some renaimings
astakos_quotas/uuid/username/authenticate --> user_quotas/uuid2username/username2uuid/info
astakos_... --> admin_...
e.g.,
astakos_services --> admin_service, astakos_commission --> admin_commission
4. Implement OpenStack Network API 2.0, with synnefo/cyclades extentions. New:
network info/list/create/delete/set
subnet info/list/create/set
port info/list/create/delete/set
5. Create a kamaki-shell for kamaki shell [#4612]
This document describes changes and steps to upgrade from kamaki 0.10 to
kamaki 0.11
This document describes changes and steps to upgrade from kamaki 0.11 to
kamaki 0.12
1. Update the kamaki configuration with these commands:
$ kamaki config delete network_cli
$ kamaki config delete ip_cli
2. Upgrade shell scripts to use the new syntax:
positional arguments are removed, except if refering to the objects
e.g., kamaki server info SERVER_ID
but not kamaki server create NAME IMAGE FLAVOR
Use mandatory non-positional arguments
e.g., kamaki server create --name=NAME --image-id=IMAGE_ID --flavor-id=FLAVOR_ID
kamaki image register --name=NAME --location=IMAGE_LOCATION
Config file
-----------
......
This diff is collapsed.
......@@ -67,7 +67,23 @@ except ImportError:
from objpool.http import PooledHTTPConnection
PooledHTTPConnection
except ImportError:
stderr.write("`objpool` package is required to build kamaki docs.\n")
stderr.write('`objpool` package is required to build kamaki docs.\n')
exit(1)
try:
from astakosclient import AstakosClient
AstakosClient
except ImportError:
path.insert(0, '%s/%s' % (
SITE_PACKAGES_PATH,
'astakosclient-0.14.5next_10224_6c10f58-py2.7.egg'))
try:
from astakosclient import AstakosClient
AstakosClient
except ImportError:
stderr.write(
'`astakosclient` package is required to build kamaki docs.\n')
exit(1)
# try:
......@@ -79,7 +95,8 @@ except ImportError:
# from progress.bar import ShadyBar
# ShadyBar
# except ImportError:
# stderr.write("`progress` package is suggested to build kamaki docs.\n")
# stderr.write(
# '`progress` package is suggested to build kamaki docs.\n')
path.insert(0, os.path.join(os.path.abspath(os.path.dirname(__file__)), '..'))
......@@ -112,7 +129,7 @@ copyright = u'2013, GRNET'
# built documents.
#
# The short X.Y version.
version = '0.11'
version = '0.12'
# The full version, including alpha/beta/rc tags.
try:
......
......@@ -11,10 +11,10 @@ command line interface of kamaki.
Since Synnefo services are build as OpenStack extensions, an inheritance
approach has been chosen for implementing clients for both. In specific,
the *compute*, *storage* and *image* modules are client implementations for the
OpenStack compute and OpenStack object-store APIs, respectively. The rest of the
modules implement the Synnefo extensions (i.e., *cyclades* and
OpenStack compute, OpenStack object-store and Image APIs respectively. The rest
of the modules implement the Synnefo extensions (i.e., *cyclades* and
*cyclades_rest_api* extents *compute*, *pithos* and *pithos_rest_api* extent
*storage*) or novel Synnefo services (*image* for *plankton*).
*storage*).
Setup a client instance
-----------------------
......@@ -45,7 +45,7 @@ Using endpoints to get the base_url
-----------------------------------
In OpenStack, each service (e.g., `compute`, `object-store`, etc.) has a number
of `endpoints`. These `endpoints` are actually URIs that are used by kamaki as
of `endpoints`. These `endpoints` are URIs that are used by kamaki as
prefixes to form the corresponding API calls. Client applications need just
one of these these `endpoints`, namely the `publicURL`, which is also referred
to as `base_url` in kamaki client libraries.
......@@ -56,7 +56,7 @@ Here are instructions for getting the base_url for a service::
(Example 1.2)
2. Use them to instantiate an AstakosClient
(Example 1.2)
3. Use AstakosClient instance to get the endpoints of the service of interest
3. Use AstakosClient instance to get endpoints for the service of interest
(Example 1.3)
4. The 'publicURL' endpoint is the base_url we are looking for
(Example 1.3)
......@@ -99,10 +99,11 @@ clients only need the one labeled as ``publicURL``.
The ``get_service_endpoints`` method is called with the service name as an
argument. Here are the service names for the kamaki clients::
storage.StorageClient, pithos.PithosClient --> object-store
compute.ComputeClient, cyclades.CycladesClient --> compute
image.ImageClient --> image
astakos.AstakosClient --> identity, account
storage.StorageClient, pithos.PithosClient --> object-store
compute.ComputeClient, cyclades.CycladesClient --> compute
network.NetworkClient, cyclades.CycladesNetworkClient --> network
image.ImageClient --> image
astakos.AstakosClient --> identity, account
Use client methods
------------------
......
......@@ -6,22 +6,27 @@ APIs code
Command Specifications
----------------------
user
^^^^
astakos
^^^^^^^
Features: user, project, quota, resource, commission, endpoint, service
.. automodule:: kamaki.cli.commands.astakos
:members:
:undoc-members:
cyclades (server, flavor, network)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cyclades
^^^^^^^^
Features server, flavor
.. automodule:: kamaki.cli.commands.cyclades
:members:
:undoc-members:
pithos (file)
^^^^^^^^^^^^^
pithos
^^^^^^
Features file, container, sharer, group
.. automodule:: kamaki.cli.commands.pithos
:members:
......@@ -30,129 +35,68 @@ pithos (file)
image
^^^^^
.. automodule:: kamaki.cli.commands.image
:members:
:undoc-members:
Kamaki commands
^^^^^^^^^^^^^^^
config
""""""
.. automodule:: kamaki.cli.commands.config
:members:
:undoc-members:
history
"""""""
Features (image, imagecompute)
.. automodule:: kamaki.cli.commands.history
:members:
:undoc-members:
Command Line Interfaces
-----------------------
argument
^^^^^^^^
.. automodule:: kamaki.cli.argument
.. automodule:: kamaki.cli.commands.image
:members:
:show-inheritance:
:undoc-members:
command_shell
^^^^^^^^^^^^^
.. automodule:: kamaki.cli.command_shell
:members:
:show-inheritance:
:undoc-members:
network
^^^^^^^
command_tree
^^^^^^^^^^^^
Features network, port, subnet, ip
.. automodule:: kamaki.cli.command_tree
.. automodule:: kamaki.cli.commands.network
:members:
:show-inheritance:
:undoc-members:
Kamaki commands
^^^^^^^^^^^^^^^
config
^^^^^^
""""""
.. automodule:: kamaki.cli.config
.. automodule:: kamaki.cli.commands.config
:members:
:show-inheritance:
:undoc-members:
errors
^^^^^^
.. automodule:: kamaki.cli.errors
:members:
:show-inheritance:
:undoc-members:
history
^^^^^^^
.. automodule:: kamaki.cli.history
:members:
:show-inheritance:
:undoc-members:
utils
^^^^^
.. automodule:: kamaki.cli.utils
.. automodule:: kamaki.cli.commands.errors
:members:
:show-inheritance:
:undoc-members:
.. _the-client-api-ref:
The clients API
---------------
Modules list
^^^^^^^^^^^^
compute REST API
^^^^^^^^^^^^^^^^
compute
^^^^^^^
.. automodule:: kamaki.clients.compute.rest_api
:members:
:show-inheritance:
:undoc-members:
compute
^^^^^^^
.. automodule:: kamaki.clients.compute
:members:
:show-inheritance:
:undoc-members:
cyclades REST API
^^^^^^^^^^^^^^^^^
cyclades
^^^^^^^^
.. automodule:: kamaki.clients.cyclades.rest_api
:members:
:show-inheritance:
:undoc-members:
cyclades
^^^^^^^^
.. automodule:: kamaki.clients.cyclades
:members:
:show-inheritance:
......@@ -167,17 +111,14 @@ storage
:show-inheritance:
:undoc-members:
pithos_rest_api
^^^^^^^^^^^^^^^
pithos
^^^^^^
.. automodule:: kamaki.clients.pithos.rest_api
:members:
:show-inheritance:
:undoc-members:
pithos
^^^^^^
.. automodule:: kamaki.clients.pithos
:members:
:show-inheritance:
......@@ -191,20 +132,21 @@ image
:show-inheritance:
:undoc-members:
astakos
network
^^^^^^^
.. automodule:: kamaki.clients.astakos
.. warning:: For synnefo, the suggested network implementation is in
kamaki.clients.cyclades.CycladesNetworkClient extension
.. automodule:: kamaki.clients.network
:members:
:show-inheritance:
:undoc-members:
astakos
^^^^^^^
utils
^^^^^
.. automodule:: kamaki.clients.utils
.. automodule:: kamaki.clients.astakos
:members:
:show-inheritance:
:undoc-members:
......@@ -27,23 +27,31 @@ configuration types can also be created and managed in the same fashion as the
Kamaki preset global options, as they appear in the configuration file::
[global]
default_cloud=
colors=off
log_file=${HOME}/.kamaki.log
log_token=off
log_data=off
log_pid=off
max_threads=7
history_file=${HOME}/.kamaki.history
user_cli=astakos
file_cli=pithos
server_cli=cyclades
flavor_cli=cyclades
network_cli=cyclades
ip_cli=cyclades
image_cli=image
config_cli=config
history_cli=history
project_cli = astakos
default_cloud = my_cloud
quota_cli = astakos
file_cli = pithos
subnet_cli = network
history_cli = history
group_cli = pithos
server_cli = cyclades
container_cli = pithos
imagecompute_cli = image
user_cli = astakos
network_cli = network
resource_cli = astakos
config_cli = config
flavor_cli = cyclades
sharer_cli = pithos
image_cli = image
port_cli = network
ip_cli = network
history_file = /home/someuser/.kamaki.history
colors = off
log_pid = off
log_token = off
log_data = off
log_file = /home/someuser/.kamaki.log
A cloud configuration is required to make kamaki run. The
`setup guide <../setup.html>`_ can help when setting one or more cloud
......
......@@ -169,6 +169,7 @@ and we want to register it to the Plankton *image* service.
raise
# 3.2 Register the image
properties = dict(osfamily='linux', root_partition='1')
try:
image = plankton.image_register(IMAGE_NAME, IMAGE_LOCATION)
except ClientError:
......@@ -580,12 +581,12 @@ logging more. We also added some command line interaction candy.
raise
def register_image(plankton, name, user_id, container, path):
def register_image(plankton, name, user_id, container, path, properties):
image_location = (user_id, container, path)
print(' Register the image')
try:
return plankton.register(name, image_location)
return plankton.register(name, image_location, properties)
except ClientError:
log.debug('Failed to register image %s' % name)
raise
......@@ -691,7 +692,9 @@ logging more. We also added some command line interaction candy.
plankton = init_plankton(endpoints['plankton'], token)
image = register_image(
plankton, 'my image', user_id, opts.container, opts.imagefile)
plankton, 'my image', user_id, opts.container, opts.imagefile,
properties=dict(
osfamily=opts.osfamily, root_partition=opts.rootpartition))
print('4. Create virtual cluster')
cluster = Cluster(
......@@ -778,6 +781,16 @@ logging more. We also added some command line interaction candy.
help='Where to store information on created servers '
'including superuser passwords',
default='')
parser.add_option('--image-osfamily',
action='store', type='string', dest='osfamily',
metavar='OS FAMILY',
help='linux, windows, etc.',
default='linux')
parser.add_option('--image-root-partition',
action='store', type='string', dest='rootpartition',
metavar='IMAGE ROOT PARTITION',
help='The partition where the root home is ',
default='1')
opts, args = parser.parse_args(argv[1:])
......
Configuration
=============
The following refers to the configuration version 0.9 or better. There is also
The following refers to the configuration version 0.12 or better. There is also
information on how to convert from older configuration files.
In this scenario, we start with an old configuration file at
......@@ -22,8 +22,8 @@ Now, let kamaki do the conversion
.. code-block:: console
$ kamaki user authenticate
. Config file format version >= 9.0 is required
$ kamaki user info
. Config file format version >= 0.12 is required
. Configuration file: /home/someuser/.kamakirc
. Attempting to fix this:
. Calculating changes while preserving information
......@@ -37,6 +37,7 @@ Now, let kamaki do the conversion
. ... rescue server.cli => global.server_cli
. ... rescue history.file => global.history_file
. ... rescue history.cli => global.history_cli
. ... change network_cli value: cyclades => network
. ... DONE
. The following information will NOT be preserved:
. global.account = AccountName
......@@ -80,7 +81,7 @@ Ask kamaki to load from a non-existing configuration file
.. code-block:: console
$ kamaki -c nonexisting.cnf user authenticate
$ kamaki -c nonexisting.cnf user info
. No cloud is configured
. | To configure a new cloud "<cloud name>", find and set the
. | single authentication URL and token:
......@@ -106,7 +107,7 @@ Try to connect
.. code-block:: console
$ kamaki -c nonexisting.cnf user authenticate
$ kamaki -c nonexisting.cnf user info
. No authentication token provided for cloud "mytest"
. | Set a token for cloud mytest:
. | kamaki config set cloud.mytest.token <token>
......@@ -149,7 +150,7 @@ Check for clouds and attempt to authenticate
$ kamaki config get cloud
. cloud.default.token = my0ld70k3n==
$ kamaki user authenticate
$ kamaki user info
. No authentication URL provided for cloud "mytest"
. | Set a URL for cloud mytest:
. | kamaki config set cloud.mytest.url <URL>
......@@ -160,7 +161,7 @@ Set a non-existing URL for cloud.default and attempt authentication
.. code-block:: console
$ kamaki config set cloud.default.url https://nonexisting.example.com
$ kamaki user authenticate
$ kamaki user info
. Failed while http-connecting to https://nonexisting.example.com
$
......@@ -169,7 +170,7 @@ Set the URL from the previous example and attempt authentication
.. code-block:: console
$ kamaki config set cloud.default.url https://accounts.example.com/identity/v2.0/
$ kamaki user authenticate
$ kamaki user info
. (401) Authorization failed for token gZH99orgkfYHmGksZKvHJw==
. | UNAUTHORIZED unauthorized (Invalid token)
$
......@@ -186,7 +187,7 @@ Set up the correct values and attempt authentication
$ kamaki config set cloud.default.url https://accounts.deploymentexample.com/identity/v2.0
$ kamaki config set cloud.default.token myd3pl0ym3nt70k3n==
$ kamaki user authenticate
$ kamaki user info
. ...
. user:
. id: my-d3pl0ym3nt-u53r-1d
......@@ -242,19 +243,19 @@ Authenticate against different clouds
.. code-block:: console
$ kamaki user authenticate
$ kamaki user info
. ...
. <response from deploymentexample.com>
. ...
$ kamaki --cloud=mytest user authenticate
$ kamaki --cloud=mytest user info
. ...
. <response from example.com>
. ...
$ kamaki --cloud=default user authenticate
$ kamaki --cloud=default user info
. ...
. <response from deploymentexample.com, same as default behavior>
. ...
$ kamaki --cloud=nonexistingcloud user authenticate
$ kamaki --cloud=nonexistingcloud user info
. No cloud "nonexistingcloud" is configured
. | To configure a new cloud "nonexistingcloud", find and set the
. | single authentication URL and token:
......@@ -267,14 +268,14 @@ Confuse kamaki by removing the default_cloud option, set mytest as default
.. code-block:: console
$ kamaki config delete default_cloud
$ kamaki user authenticate
$ kamaki user info
. Found 2 clouds but none of them is set as default
. | Please, choose one of the following cloud names:
. | default, mytest
. | To set a default cloud:
. | kamaki config set default_cloud <cloud name>
$ kamaki config set default_cloud mytest
$ kamaki user authenticate
$ kamaki user info
. ...
. <response from example.com>