Commit 9f75385a authored by Stavros Sachtouris's avatar Stavros Sachtouris
Browse files

Merge branch 'release-0.11'

Conflicts:
	Changelog
	docs/examplesdir/imageregister.rst
	docs/installation.rst
	kamaki/cli/__init__.py
	kamaki/cli/commands/__init__.py
	kamaki/cli/commands/astakos.py
	kamaki/cli/commands/cyclades.py
	kamaki/cli/commands/image.py
	kamaki/cli/commands/pithos.py
	kamaki/cli/errors.py
	kamaki/clients/__init__.py
	version
parents 47c2bf2a 864767eb
CHANGELOG for hotfix 0.10.2
CHANGELOG for version 0.11
Bug Fixes:
- Fix command error when creating server with personality [#4339]
CHANGELOG for hotfix 0.10.1
Bug Fixes:
- Fix error causing typo in kamaki.commands.astakos
CHANGELOG for version 0.10
Bug Fixes:
- Avoid loading optional cmd groups specified in same pkg with mantatory
comand groups, except if user requests otherwise ( #4114 )
- Ignore subdir output in container_get with prefix+delimiter [#4134]
- Fix the split-input bug in repeatable k=v args [#4232]
- Make authentication lazy (#4300)
- Stop shell from destroying the config instance
- Shell can manage all valid command line arguments [#3716]
- Restore 2nd level command syntax in shell [#3736]
......@@ -26,96 +12,33 @@ Bug Fixes:
Changes:
- Change context: kamaki-image-addproperty --> kamaki-image-compute-property
- http logger names: clients.recv/send --> kamaki.clients.recv/send
- Move logger creation and control (except of the actuall logging) to cli
- These PithosClient methods return the metadata of created object [#3656]:
upload_object, upload_object_unchunked, create_object_by_manifestation
- Cache container info per upload session [#3735]
- Disallow moving deleted objects by version [#3737]
This operation was implemented by accident, due to the symetry between
move and copy
- Rename file/server-meta commands to file/server-metadata
- Rename image-[add|del]member commands to members-[add|delete]
- Remove update option from image-register
- In image-compute split properties to properties-list and properties-get
- Add optional output to methods[#3756, #3732]:
- file:
mkdir, touch, create, move, copy, move, append, truncate, overwrite,
manifest, upload, delete, purge, unpublish, permissions set/delete, info,
metadata set/delete, containerlimit set, versioning set, group set/delete,
upload, overwrite
- image:
unregister, members add/delete/set
-image compute:
delete, properties delete
- server: rename, delete, reboot, start, shutdown, firewall-set
- network: rename, delete, connect
- Add optional json for methods with output [#3732]
- file:
list, hashmap, permissions-get, info, metadata-get, quota,
containerlimit-get, group-list, sharers, versions
- server: list, info, create, console, addr, metadata-list/set, stats
- image: list, meta, register, shared, list
- image compute: list, info, properties-list/get/add/set
- flavor: list, info
- network: info, list, create
- astakos: authenticate
- Transliterate methods to list-get-set-delete command groups:
- file: permissions, versioning, group and metadata
- image: members, member
- image compute: properties
- server: firewall, metadata
- Add a _format_image_headers method and use it in image_register and get_meta
for uniform image meta output [#3797]
- Rename meta-->metadata and remove values @lib [#3633]
- Adjust authenticate to new url schema of synnefo>=0.14 [#3832, #3874]
as a side effect, some renamings in astakos.AstakosClient:
info --> user_info, user --> list_users
- Simplify listings (kamaki.cli.utils: print methods)
- Enrich client API docs with examples and astakos endpoint information [#4135]
- Show user names in /file sharers [#4203]
- Show user name in image/server/network list [#4228]
- Remove kamaki compute image properties add [#4231]
- Image register file location format is changed:
pithos://<uuid>/<container>/<image path> --> <container>:<image path>
but Backwards compatibility is preserved for this version
- Logs do not contain kamaki.clients pids by default [#4242]
- Replace page_hold method with pydoc.pager [#4279]
- Remove commands.pithos.DelimiterArgument, replace with ValueArgument
wherever it is used
- Replace print methods with member print methods in _commands.* [#4292]
- kamaki.cli.config is now a directory package [#4058]
- Modify progress bar handler in cli.commands to allow count down and timeout
[#4352]
- Make wait progress bars in cyclades CLI to count down to timeout instead of
progressing forward [#4352]
- Rename IP commands to new namespace and better names [#4422]
Features:
- Add unittests in the folowing kamaki.cli packages: [#4058]
command_tree, argument, history, logger, utils
- Implement an optional astakosclient cli exposed as "astakos", with the following methods:
authenticate, uuid, username, quotas, service uuid/username/quotas
- Add some astakos/keystone kamaki-lib api calls [#3874], used to access
astakos-calls cache:
get_services, get_service_details, get_service_endpoints
- Implement floating ip methods for compute and cyclades clients [#3862]
ComputeRestClient: floating_ip_pools_get, floating_ips_get/post/delete
CycladesRestClient: floating_ip_pools_get, floating_ips_get/post/delete
ComputeClient: get_floating_ip_pools, get_floating_ips,
alloc/get_delete_floating_ip
CycladesClient: get_floating_ip_pools, get_floating_ips,
alloc/get_delete_floating_ip, dis/assoc_floating_ip_to_server
- Add Examples to documentation
- Implement floating IP commands
server ip list/create/delete/info/pools/attach/detach
- Implement --wait for server_create/delete/start/reboot/shutdown [#3867]
- Implement network_wait [#3862]
- Implement user_list/set/whoami to switch between session users [#3632, #3660]
- Expose more astakosclient methods in a 1to1 fashion [#4115]:
resources, feedback, endpoints,
commission_pending/info/action/accept/reject/resolve/issuejson/issue
- Implement resize_server and expose it as server resize [#4153]
- Expose the project API of astakosclient to kamaki CLI [#4155]
- Full support for compute v2 basic API [#4139]
- Use container:path format in register [#3778]
- Guess mimetype when uploading and use it in upload [#4196]
- Add upload local image abilities to register [#4206]
New register args: --update-image-file=/local/path, --no-progress-bar
- Implement an astakos.post_user_catalogs call for getting usernames from a
list of user uuids [#4203]
- Use multiformed ranges in file/pithos [#4059]
- Implement more filters in image/network/flavor/server/file list [#4220]
- Implement /image meta list/set/delete [#4231] metadata and properties
- New config option log_pid (default: off) can allow pid-logging [#4242]
- Expand kamaki.cli unitests for the following packages ( [#4058] ):
errors
- Modify print methods in cli utils to use arbitary stream objects [#4288]
- Implement wrapers for cli.utils print methods, in _commands [#4292]
- Implement unittests for kamaki.cli.config [#4058]
- Implement --type=SOFT/HARD argument for server reboot [#4338]
The old argument -f/--force <==> --type=HARD is kept for bw compatibility
A warning message guides users to stop using it
- Catch Keyboard Interupts in cli wait methods and exit gracefully [#4351]
- Add -R/--recursive in file listing, to list containers with their contents
[#4301]
- Add -w/--wait to server firewall set [#4298]
- Add --output-format where applicable, keep -j/--json too, for bw
compatibility [#4342]
- Allow metakeys in personality argument [#4312]
This document describes changes and steps to upgrade from kamaki 0.9.X to
kamaki 0.10
This document describes changes and steps to upgrade from kamaki 0.10 to
kamaki 0.11
Config file
-----------
......
......@@ -143,16 +143,16 @@ server (Compute/Cyclades)
.. code-block:: text
addr : List a server's nic address
addr : List a server nic address
console : Get a VNC console
create : Create a server
delete : Delete a server
firewall: Manage server's firewall profile
set : Set the server's firewall profile
get : Get the server's firewall profile
firewall: Manage server firewall profile
set : Set the server firewall profile
get : Get the server firewall profile
ip : Manage floating IPs for the servers
attach: Attach a floating ip to a server with server_id
info : A floating IPs' details
info : A floating IP details
detach: Detach floating ip from server
list : List all floating ips
create: Create a new floating IP
......@@ -165,7 +165,7 @@ server (Compute/Cyclades)
set : Add / update server metadata
delete: Delete a piece of server metadata
reboot : Reboot a server
rename : Update a server's name
rename : Update a server name
shutdown: Shutdown a server
start : Start a server
stats : Get server statistics
......@@ -246,6 +246,41 @@ Showcase: Create a server
.. Note:: In kamaki shell, / is used to access top-level command groups while working in command group contexts
ip (Compute/Cyclades)
---------------------
.. code-block:: text
info: Details for an IP
list: List reserved floating IPs
attach: Attach a floating IP to a server
pools: List pools of floating IPs
release: Release a floating IP
detach: Detach a floating IP from a server
reserve: Reserve a floating IP
Showcase: Reserve and attach IP to server
-----------------------------------------
.. code-block:: text
* Enter IP context *
[kamaki]: ip
* Reserve an IP and see servers *
[kamaki]: reserve
123.456.78.9
[kamaki]: /server list
42 My Windows Server
43 My Linux Server
* Attach IP to server
[kamaki]: attach 43 123.456.78.9
.. Note:: In kamaki shell, / is used to access top-level command groups while
working in command group contexts
network (Compute/Cyclades)
--------------------------
......@@ -445,12 +480,12 @@ Showcase: Upload and download a file
-rw-rw-r-- 1 ******** ******** 20M Nov 26 15:42 rndm_remote.file
[file]: !diff rndm_local.file rndm_remote.file
.. Note:: In kamaki shell, ! is used to execute OS shell commands (e.g. bash)
.. Note:: In kamaki shell, ! is used to execute OS shell commands (e.g., bash)
.. warning:: The container:object/path syntax does not function if the
container and / or the object path contain one or more : characters. To use
containers and objects with : use the --container and --dst-container
arguments, e.g. to copy test.py object from grnet:dev container to
grnet:deploy ::
arguments, e.g., to copy test.py object from example:dev container to
example:deploy ::
$ kamaki file copy --container=grnet:dev test.py --dst-container=grnet:deploy
$ kamaki file copy --container=example:dev test.py --dst-container=example:deploy
......@@ -54,17 +54,32 @@
from sys import path, stderr
import os
SITE_PACKAGES_PATH = os.path.expanduser(
'~/src/kamaki/docsenv/lib/python2.7/site-packages')
try:
from objpool.http import PooledHTTPConnection
PooledHTTPConnection
except ImportError:
stderr.write("`objpool` package is required to build kamaki docs.\n")
try:
from progress.bar import ShadyBar
ShadyBar
except ImportError:
stderr.write("`progress` package is required to build kamaki docs.\n")
path.insert(0, SITE_PACKAGES_PATH)
try:
from objpool.http import PooledHTTPConnection
PooledHTTPConnection
except ImportError:
stderr.write("`objpool` package is required to build kamaki docs.\n")
exit(1)
# try:
# from progress.bar import ShadyBar
# ShadyBar
# except ImportError:
# path.insert(0, SITE_PACKAGES_PATH)
# try:
# from progress.bar import ShadyBar
# ShadyBar
# except ImportError:
# stderr.write("`progress` package is suggested to build kamaki docs.\n")
path.insert(0, os.path.join(os.path.abspath(os.path.dirname(__file__)), '..'))
......@@ -97,7 +112,7 @@ copyright = u'2013, GRNET'
# built documents.
#
# The short X.Y version.
version = '0.10'
version = '0.11'
# The full version, including alpha/beta/rc tags.
try:
......
......@@ -2,29 +2,31 @@ Adding Commands
===============
Kamaki commands are implemented as python classes, decorated with a special
decorator called *command*. This decorator is a method of kamaki.cli that adds
a new command in a CommandTree structure (kamaki.cli.commant_tree). The later
is used by interfaces to manage kamaki commands.
decorator called *command*. This decorator is a method of *kamaki.cli* that
adds a new command in a *CommandTree* structure. A *CommandTree* (package
*kamaki.cli.commant_tree*) is a data structure used by kamaki to manage command
namespaces.
In the following, a set of kamaki commands will be implemented::
For demonstration purposes, the following set of kamaki commands will be
implemented in this document::
mygrp1 list all //show a list
mygrp1 list details [--match=<>] //show list of details
mygrp2 list all [regular expression] [-l] //list all subjects
mygrp2 info <id> [name] //get information for subject with id
There are two command sets to implement, namely mygrp1 and mygrp2. The first
will contain two commands, namely list-all and list-details. The second one
will also contain two commands, list-all and info. To avoid ambiguities,
command names should rather be prefixed with the group they belong to, e.g.
mygrp1-list-all and mygrp2-list-all.
There are two command groups to implement i.e., *mygrp1* and *mygrp2*,
containing two commands each (*list_all*, *list_details* and *list_all*, *info*
respectively). To avoid ambiguities, command names are prefixed with the
command group they belong to, e.g., *mygrp1_list_all* and *mygrp2_list_all*.
The underscore is used to separate command namespaces.
The first command has the simplest possible syntax: no parameters, no runtime
arguments. The second accepts an optional runtime argument with a value. The
third features an optional argument and an optional runtime flag argument. The
last is an example of a command with an obligatory and an optional argument.
The first command (*mygrp1_list_all*) has the simplest possible syntax: no
parameters, no runtime arguments. The second accepts an optional runtime argument with a value. The third features an optional parameter and an optional
runtime flag argument. The last is an example of a command with an obligatory
and an optional parameter.
Samples of the expected behavior in one-command mode are following:
Examples of the expected behavior in one-command mode:
.. code-block:: console
......@@ -35,34 +37,32 @@ Samples of the expected behavior in one-command mode are following:
- - - -
list
$ kamaki mygrp1 list
Syntax Error
Options
- - - -
all show a list
details show a list of details
$ kamaki mygrp1 list all
... (mygrp1 client method is called) ...
... (a mygrp1_list_all instance runs) ...
$ kamaki mygrp2 list all 'Z[.]' -l
... (mygrp2 client method is called) ...
... (a mygrp2_list_all instance runs) ...
$
The above example will be used throughout the present guide.
The CommandTree structure
-------------------------
CommandTree manages a command by its path. Each command is stored in multiple
nodes on the tree, so that the last term is a leaf and the route from root to
that leaf represents the command path. For example the commands *file upload*,
*file list* and *file info* are stored together as shown bellow::
CommandTree manages a command by its namespace. Each command is stored in
a tree path, where each node is a name. A leaf is the end term of a namespace and contains a pointer to the command class to be executed.
Here is an example from the actual kamaki command structure, where the commands
*file upload*, *file list* and *file info* are represented as shown bellow::
- file
''''''''|- info
|- list
|- upload
The example used in the present, should result to the creation of two trees::
Now, let's load the showcase example on CommandTrees::
- mygrp1
''''''''|- list
......@@ -74,12 +74,12 @@ The example used in the present, should result to the creation of two trees::
'''''''|- all
|- info
Each command group should be stored on a different CommandTree. For that
reason, command specification modules should contain a list of CommandTree
objects, named *_commands*
Each command group should be stored on a different CommandTree.
For that reason, command specification modules should contain a list of CommandTree objects, named *_commands*. This mechanism allows any interface
application to load the list of commands from the *_commands* array.
A command group information (name, description) is provided at CommandTree
structure initialization:
The first name of the command path and a description (name, description) are needed to initializeg a CommandTree:
.. code-block:: python
......@@ -88,15 +88,19 @@ structure initialization:
_commands = [_mygrp1_commands, _mygrp2_commands]
The command decorator
---------------------
The *command* decorator mines all the information necessary to build a command
specification which is then inserted in a CommanTree instance::
All commands are specified by subclasses of *kamaki.cli.commands._command_init*
These classes are called "command specifications".
The *command* decorator mines all the information needed to build a namespace
from a command specification::
class code ---> command() --> updated CommandTree structure
Kamaki interfaces make use of this CommandTree structure. Optimizations are
Kamaki interfaces make use of the CommandTree structure. Optimizations are
possible by using special parameters on the command decorator method.
.. code-block:: python
......@@ -108,14 +112,16 @@ possible by using special parameters on the command decorator method.
:param prefix: of the commands allowed to be inserted ('' for all)
:param descedants_depth: is the depth of the tree descedants of the
:param descedants_depth: is the depth of the tree descendants of the
prefix command.
"""
Creating a new command specification set
----------------------------------------
A command specification developer should create a new module (python file) with as many classes as the command specifications to be offered. Each class should be decorated with *command*.
A command specification developer should create a new module (python file) with
one command specification class per command. Each class should be decorated
with *command*.
.. code-block:: python
......@@ -129,10 +135,10 @@ A command specification developer should create a new module (python file) with
...
A list of CommandTree structures must exist in the module scope, with the name
_commands, as shown above. Different CommandTree objects correspond to
different command groups.
*_commands*. Different CommandTree objects correspond to different command
groups.
Get command description
Set command description
-----------------------
The description of each command is the first line of the class commend. The
......@@ -143,25 +149,50 @@ subject with id*" description.
...
@command(_mygrp2_commands)
class mygrp2_info()
"""get information for subject with id"""
class mygrp2_info():
"""get information for subject with id
Anything from this point and bellow constitutes the long description
Please, mind the indentation, pep8 is not forgiving.
"""
...
Description placeholders
------------------------
There is possible to create an empty command, that can act as a description
placeholder. For example, the *mygrp1_list* namespace does not correspond to an
executable command, but it can have a helpful description. In that case, create
a command specification class with a command and no code:
.. code-block:: python
@command(_mygrp1_commands)
class mygrp1_list():
"""List mygrp1 objects.
There are two versions: short and detailed
"""
.. warning:: A command specification class with no description is invalid and
will cause an error.
Declare run-time argument
-------------------------
The argument mechanism allows the definition of run-time arguments. Some basic
argument types are defined at the
A special argument mechanism allows the definition of run-time arguments. This
mechanism is based on argparse and is designed to simplify argument definitions
when specifying commands.
Some basic argument types are defined at the
`argument module <code.html#module-kamaki.cli.argument>`_, but it is not
uncommon to extent these classes in order to achieve specialized type checking
and syntax control (e.g. at
`pithos cli module <code.html#module-kamaki.cli.commands.pithos>`_).
a bad idea to extent these classes in order to achieve specialized type
checking and syntax control. Still, in most cases, the argument types of the
argument package are enough for most cases.
To declare a run-time argument on a specific command, the object class should
initialize a dict called *arguments* , where Argument objects are stored. Each
argument object is a possible run-time argument. Syntax checking happens at
client level, while the type checking is implemented in the Argument code
(thus, many different Argument types might be needed).`
To declare a run-time argument on a specific command, the specification class
should contain a dict called *arguments* , where Argument objects are stored.
Each argument object is a run-time argument. Syntax checking happens at client
level, while the type checking is implemented in the Argument code (e.g.,
IntArgument checks if the value is an int).
.. code-block:: python
......@@ -190,15 +221,15 @@ or more usually and elegantly:
arguments = dict(
match=ValueArgument(
'Filter output to match string', ('-m', --match'))
'Filter output to match string', ('-m', --match'))
)
Accessing run-time arguments
----------------------------
To access run-time arguments, users can use the _command_init interface, which
implements __item__ accessors to handle run-time argument values. In specific,
an instance of _command_init can use brackets to set or read <argument>.value .
To access run-time arguments, users can use the *_command_init* interface,
which implements *__item__* accessors to handle run-time argument values. In
other words, one may get the value of an argument with *self[<argument>]*.
.. code-block:: python
......@@ -222,9 +253,8 @@ an instance of _command_init can use brackets to set or read <argument>.value .
The main method and command parameters
--------------------------------------
The command behavior for each command / class is coded in *main*. The
parameters of *main* method defines the command parameters part of the syntax.
In specific::
The command behavior for each command class is coded in *main*. The
parameters of *main* method affect the syntax of the command. In specific::
main(self, param) - obligatory parameter <param>
main(self, param=None) - optional parameter [param]
......@@ -236,8 +266,8 @@ In specific::
main(self, *args) - arbitary number of params [...]
main(self, param1____param2, *args) - <param1:param2> [...]
The information that can be mined by *command* for each individual command is
presented in the following:
Let's have a look at the command specification class again, and highlight the
parts that affect the command syntax:
.. code-block:: python
:linenos:
......@@ -245,23 +275,32 @@ presented in the following:
from kamaki.cli.argument import FlagArgument
...
_commands = [_mygrp1_commands, _mygrp2=commands]
_commands = [_mygrp1_commands, _mygrp2_commands]
...
@command(_mygrp2_commands)
class mygrp2_list_all():
"""List all subjects"""
"""List all subjects
Refers to the subject accessible by current user
"""
arguments = dict(FlagArgument('detailed list', '-l'))
def main(self, reg_exp=None):
...
This will load the following information on the CommandTree:
The above lines contain the following information:
* Namespace and name (line 8): mygrp2 list all
* Short (line 9) and long (line 10) description
* Parameters (line 15): [reg exp]
* Runtime arguments (line 13): [-l]
* Runtime arguments help (line 13): detailed list
* Syntax (from lines 8,12,19): mygrp list all [reg exp] [-l]
* Description (form line 9): List all subjects
* Arguments help (from line 13,14): -l: detailed list
.. tip:: It is suggested to code the main functionality in a member method
called *_run*. This allows the separation between syntax and logic. For
example, an external library may need to call a command without caring
about its command line behavior.
Letting kamaki know
-------------------
......@@ -270,11 +309,11 @@ Kamaki will load a command specification *only* if it is set as a configurable
option. To demonstrate this, let the command specifications coded above be
stored in a file named *grps.py*.
The developer should move file *grps.py* to kamaki/cli/commands, the default
place for command specifications, although running a command specification from
a different path is also a kamaki feature.
The developer should move the file *grps.py* to *kamaki/cli/commands*, the
default place for command specifications
The user has to use a configuration file where the following is added:
These lines should be contained in the kamaki configuration file for a new
command specification module to work:
::
[global]
......@@ -288,9 +327,11 @@ or equivalently:
$ kamaki config set mygrp1_cli grps
$ kamaki config set mygrp2_cli grps
Command specification modules don't need to live in kamaki/cli/commands,
although this is suggested for uniformity. If a command module exist in another
path::
.. note:: running a command specification from a different path is supported.
To achieve this, add a *<group>_cli = </path/to/module>* line in the
configure file under the *global* section
An example::
[global]
mygrp_cli = /another/path/grps.py
......@@ -319,13 +360,23 @@ Summary: create a command set
# Define command specifications
@command(_mygrp1_commands)
class mygrp1_list(_command_init):
"""List mygrp1 objects.
There are two versions: short and detailed
"""
@command(_mygrp1_commands)
class mygrp1_list_all(_command_init):
"""show a list"""
def main(self):
def _run():
...
def main(self):
self._run()
@command(_mygrp1_commands)
class mygrp1_list_details(_command_init):
......@@ -336,11 +387,16 @@ Summary: create a command set