Commit d1be0836 authored by Stavros Sachtouris's avatar Stavros Sachtouris
Browse files

Merge branch 'develop' into feature-store

parents 58a7ac27 2aca7f5e
......@@ -59,26 +59,27 @@ Showcase: show details for flavor with id 43
name : C4R2048D10
ram : 2048
image (Compute/Cyclades + Plankton)
-----------------------------------
image (Plankton commands + Compute Image subcommands)
-----------------------------------------------------
.. code-block:: text
addmember : Add a member to an image
addproperty: Add an image property
delete : Delete image
delmember : Remove a member from an image
delproperty: Delete an image property
info : Get image details
list : List images
list : List images accessible by user
members : Get image members
meta : Get image metadata
properties : Get image properties
public : List public images
register : (Re)Register an image
setmembers : Set the members of an image
setproperty: Update an image property
shared : List shared images
compute : Compute Image API commands
list : List images
delete : Delete image
info : Get image details
properties : Get image properties
delproperty: Delete an image property
setproperty: Update an image property
Showcase: Pick an image and list the properties
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
......@@ -91,20 +92,33 @@ Showcase: Pick an image and list the properties
* list all available images *
[image]:list
1395fdfb-51b4-419f-bb02-f7d632860611 Ubuntu Desktop LTS
1580deb4-edb3-4496-a27f-7a246c4c0528 Ubuntu Desktop
18a82962-43eb-4b32-8e28-8f8880af89d7 Kubuntu LTS
6aa6eafd-dccb-422d-a904-67fe2bdde87e Debian Desktop
6b5681e4-7502-46ae-b1e9-9fd837932095 maelstrom
78262ee7-949e-4d70-af3a-85360c3de57a Windows Server 2012
86bc2414-0fb3-4898-a637-240292243302 Fedora
926ab1c5-2d85-49d4-aebe-0fce712789b9 Windows Server 2008
b2dffe52-64a4-48c3-8a4c-8214cc3165cf Debian Base
baf2321c-57a0-4a69-825d-49f49cea163a CentOS
c1d27b46-d875-4f5c-b7f1-f39b5af62905 Kubuntu
* Get properties of image with id b2dffe52-64a4-48c3-8a4c-8214cc3165cf *
[image]:properties b2dffe52-64a4-48c3-8a4c-8214cc3165cf
1. Windows Server 2008
container_format: bare
disk_format : diskdump
id : 926ab1c5-2d85-49d4-aebe-0fce712789b9
size : 11917066240
status : available
2. Windows Server 2012
container_format: bare
disk_format : diskdump
id : 78262ee7-949e-4d70-af3a-85360c3de57a
size : 11697913856
status : available
3. ubuntu
container_format: bare
disk_format : diskdump
id : 5ed5a29b-292c-4fe0-b32c-2e2b65628635
size : 2578100224
status : available
4. Debian_Wheezy_Base
container_format: bare
disk_format : diskdump
id : 1f8454f0-8e3e-4b6c-ab8e-5236b728dffe
size : 795107328
status : available
* Get properties of image with id 1f8454f0-8e3e-4b6c-ab8e-5236b728dffe *
[image]:compute properties 1f8454f0-8e3e-4b6c-ab8e-5236b728dffe
description : Debian 6.0.6 (Squeeze) Base System
gui : No GUI
kernel : 2.6.32
......@@ -164,7 +178,7 @@ Showcase: Create a server
-s, --silent Do not output anything
* List all available images *
[server]:/image list
[server]:/image compute list
1395fdfb-51b4-419f-bb02-f7d632860611 Ubuntu Desktop LTS
1580deb4-edb3-4496-a27f-7a246c4c0528 Ubuntu Desktop
18a82962-43eb-4b32-8e28-8f8880af89d7 Kubuntu LTS
......
......@@ -133,7 +133,7 @@ Modules list
compute_rest_api
^^^^^^^^^^^^^^^^
.. automodule:: kamaki.clients.compute_rest_api
.. automodule:: kamaki.clients.compute.rest_api
:members:
:show-inheritance:
:undoc-members:
......@@ -177,7 +177,7 @@ pithos
pithos_rest_api
^^^^^^^^^^^^^^^
.. automodule:: kamaki.clients.pithos_rest_api
.. automodule:: kamaki.clients.pithos.rest_api
:members:
:show-inheritance:
:undoc-members:
......
......@@ -5,9 +5,7 @@ An http connection package with connection pooling.
Since version 0.6 it is safe to use threaded connections.
The Connection package uses httplib, standard python threads and a connection pooling mechanism.
.. note:: Since version 0.6.2 the underlying pooling mechanism is packed in a new GRNET package called *objpool*.
The Connection package uses httplib, standard python threads and a connection pooling mechanism, which is imported from the *objpool* package.
.. automodule:: kamaki.clients.connection
:members:
......
......@@ -94,4 +94,166 @@ Kamaki clients may handle multiple requests at once, using threads. In that case
event = SilentEvent(self._single_threaded_method, **args)
event.start()
thread_list.append(event)
thread_list = self._watch_thread_limit(thread_list)
\ No newline at end of file
thread_list = self._watch_thread_limit(thread_list)
Going agile
-----------
The kamaki.clients package contains a set of fine-grained unit-tests for all its packages.
.. note:: unit tests require the optional python-mock package, version 1.X or better
Using the tests
^^^^^^^^^^^^^^^
To run the tests, the kamaki source code has to be downloaded.
.. code-block:: console
$ git clone https://code.grnet.gr/git/kamaki
$ cd kamaki/kamaki/clients
In each package under kamaki.clients, there is a test module (test.py) where the tests are implemented. To run all tests, run the test.py file from kamaki.clients
.. code-block:: console
$ python test.py
To test a specific class, add the class name as an argument. E.g. for the Client class:
.. code-block:: console
$ python test.py Client
To test a specific method in a class, apply an extra argument, e.g. for the request method in the Client class:
.. code-block:: console
$ python test.py Client request
Each package contains a test module (test.py) which is also runnable from the command line. E.g. in the pithos package there is a test module which contains, among others, the **download** sub-test:
.. code-block:: console
$ cd pithos
# Run all kamaki.clients.pithos tests
$ python test.py
# Run all kamaki.clients.pithos.PithoClient tests
$ python test.py Pithos
# Test kamaki.clients.pithos.PithosClient.download
$ python test.py Pithos download
To fully test a specific package, run test.py from the package location. E.g. to test everything in kamaki.clients.pithos package:
.. code-block:: console
$ cd pithos
$ python test.py
Mechanism
^^^^^^^^^
Each folder / package contains a test.py file, that represents the test module of this package. All test modules contain a set of classes that extent the TestCase class. They also contain a main method to run the tests.
By convention, testing classes are named as <Tested Class> where <Test Class> is the name of the tested class or module. Methods not grouped in classes are tested by classes named after their respective module.
For example, the kamaki.clients.pithos.PithosClient class is tested by the kamaki.clients.pithos.test.PithosClient class, while the methods in kamaki.clients.utils module are tested by the kamaki.clients.utils.test.Utils testing class.
Adding unit tests
^^^^^^^^^^^^^^^^^
After modifying or extending kamaki.clients method, classes, modules or packages, it is a good practice to also modify or extend the corresponding unit tests. What's more, it is recommended to modify or implement the testing of new behavior before implementing the behavior itself. The aim for kamaki.clients package is an 1 to 1 mapping between methods and their tests.
Modifying an existing method
""""""""""""""""""""""""""""
In case of an existing method modification, the programmer has to modify the corresponding test as well. By convention, the test method is located in the test module under the same package, in a TestCase subclass that is named with a name similar to the package or class that contains the tested method.
Example 1: to modify the kamaki.clients.utils.filter_in method, the programmer has to also adjust the kamaki.clients.utils.test.Utils.test_filter_in method.
Example 2: to modify the kamaki.clients.pithos.PithosRestClient.object_get, the programmer has to also adjust the kamaki.clients.pithos.test.PithosRestClient.test_object_get method.
Adding a new method
"""""""""""""""""""
Programmers who want to implement a new method in an existing class, are encouraged to implement the corresponding unit test first. In order to do that, they should find the testing class that is mapped to the class or module they need to extend.
Example 1: To add a **list_special** method to kamaki.clients.astakos.AstakosClient, extend the kamaki.clients.astakos.test.AstakosClient class, as shown bellow:
.. code-block:: python
# file: ${kamaki}/kamaki/clients/astakos/__init__.py
class AstakosClient(TestCase):
...
def test_list_special(self):
"""Test the list_special method"""
...
Example 2: To add a **get_random_int** method in kamaki.clients.utils module, extend the kamaki.clients.utils.test.Utils test class, as shown bellow:
.. code-block:: python
# file: ${kamaki}/kamaki/clients/utils/__init__.py
class Utils(TestCase):
...
def test_get_random_int(self):
"""Test the get_random_int method"""
...
Implementing a new class or module
""""""""""""""""""""""""""""""""""
Each class or module needs a seperate test sub-module. By convention, each class or module under the kamaki.clients should be located in a separate directory.
Example 1: To add a NewService class that implements the kamaki.clients.Client interface:
* create a new_service package and implement the unit tests in the kamaki.clients.new_service.test module:
.. code-block:: console
$ mkdir new_service && touch new_service/test.py
* create the file that will hold the package code and implement the module there:
.. code-block:: console
$ touch new_service/__init__.py
* Create the test class and methods in kamaki.clients.new_service.test
.. code-block:: python
# file: ${kamaki}/kamaki/clients/new_service/test.py
from unittest import TestCase
class NewService(TestCase):
def test_method1(self):
...
* Create the NewService and its actual functionality in kamaki.clients.new_service
.. code-block:: python
# file: ${kamaki}/kamaki/clients/new_service/__init__.py
from kamaki.clients import Client
class NewService(Client):
def method1(self, ...):
...
* Expose the new tests to top test module, by importing the test class to kamaki.clients.test
..code-block:: python
# file: ${kamaki}/kamaki/clients/test.py
from kamaki.clients.new_service.test import NewService
.. note:: If the new class or module is part of an existing sub-package, it is acceptable to append its testing class in the existing test.py file of the sub-package it belongs to. For example, the kamaki.clients.pithos.PithosClient and kamaki.clients.pithos.rest_api.PithosRestClient classes are tested by two different classes (PithosClient and PithosRestClient respectively) in the same module (kamaki.clients.pithos.test).
......@@ -6,7 +6,9 @@ This guide describes the standard installation process for kamaki, with the aspi
* Kamaki repository: `http://code.grnet.gr/git/kamaki <http://code.grnet.gr/git/kamaki>`_
* Synnefo Linux packages: `http://apt.dev.grnet.gr <http://apt.dev.grnet.gr>`_, `http://apt2.dev.grnet.gr <http://apt2.dev.grnet.gr>`_
* Kamaki at pypi: `http://pypi.python.org/pypi/kamaki <https://pypi.python.org/pypi/kamaki>`_
* Synnefo Linux packages: `http://apt2.dev.grnet.gr <http://apt2.dev.grnet.gr>`_
Linux and Unix-like enviroments
-------------------------------
......@@ -18,7 +20,6 @@ The following steps describe a command-line approach, but any graphic package ma
* As root, append the following to */etc/apt/sources.list* ::
deb http://apt.dev.grnet.gr/ squeeze main
deb http://apt2.dev.grnet.gr stable/
* Make sure the GPG public key for the GRNET dev team is added:
......@@ -65,18 +66,28 @@ The following steps describe a command-line approach, but any graphic package ma
$ sudo apt-get install kamaki
Install ansicolors and/or progress (Optional but recommended)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Install ansicolors (optional but recommended)
"""""""""""""""""""""""""""""""""""""""""""""
.. code-block:: console
$ sudo apt-get install python-ansicolors
$ sudo apt-get install python-progress
.. _installing-from-source-ref:
Install mock (for developers only)
""""""""""""""""""""""""""""""""""
.. code-block:: console
$ sudo apt-get install python-mock
.. warning:: kamaki.clients unit-tests need python-mock 1.X or better. e.g.::
Installing from source (git repos.)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
$ sudo apt-get install python-mock=1.0.1
.. _installing-from-pypi-ref:
Installing from pypi
^^^^^^^^^^^^^^^^^^^^
Requirements
""""""""""""
......@@ -102,83 +113,68 @@ With virtualenv users can setup kamaki and synnefo services in a sandbox environ
A more detailed example of using virtual env can be found at the `snf-image-creator setup guide <http://docs.dev.grnet.gr/snf-image-creator/latest/install.html#python-virtual-environment>`_
Install objpool (was: snf-common)
"""""""""""""""""""""""""""""""""
Kamaki is based on python-objpool. The objpool package is easy to install from source, even on windows platforms:
.. code-block:: console
$ git clone http://code.grnet.gr/git/objpool
$ cd objpool
$ ./setup build install
$ cd -
Install kamaki
""""""""""""""
Kamaki can be downloaded from `this location <https://code.grnet.gr/projects/kamaki/files>`_, where users can pick the version they prefer and unzip it locally:
.. code-block:: console
$ tar xvfz kamaki-0.7.tar.gz
or it can be downloaded directly from the git repository:
$ pip install kamaki
.. code-block:: console
$ git clone http://code.grnet.gr/git/kamaki
Install ansicolors
""""""""""""""""""
and then installed by the setup script:
The **ansicolors** package is not required for running kamaki, but it is
recommended as a user experience improvement. In specific, ansicolors
adds colors to kamaki responses.
.. code-block:: console
$ cd kamaki
$ ./setup build install
Install progress and/or ansicolors (optional)
"""""""""""""""""""""""""""""""""""""""""""""
$ pip install ansicolors
progress: command-line progress bars (in some commands)
Install mock
""""""""""""
ansicolors: color kamaki output (can switched on and off in `setup <setup.html>`_)
The **mock** package is needed for running the prepared unit-tests in the kamaki.clients
package. This feature is useful when extendnig / debugging kamaki functionality and is
aimed to kamaki developers and contributors. Therefore, users can enjoy the full kamaki
user experience without installing mock.
.. code-block:: console
$ pip install progress
$ pip install ansicolors
$ pip install mock
.. warning:: mock version >= 1.X
Mac OS X
--------
Kamaki can be installed on Mac OS X systems from source, by following the steps at :ref:`installing-from-source-ref`.
Kamaki can be installed on Mac OS X systems from source, by following the steps at :ref:`installing-from-pypi-ref`.
Windows
-------
Kamaki can run on Windows, either on standard Windows console, or inside an improved command line shell. The present guide presents a tested method for setting up kamaki in windows
Kamaki can be installed on Windows by following the pypi method. Installing the requirements is a bit different than in other systems.
The full process is detailed in the following:
Requirements
^^^^^^^^^^^^
* Python 2.7 or better (`Official versions <http://www.python.org/getit>`_)
* Git (download `windows version <http://git-scm.com/download/win>`_)
* Setuptools (`Official versions and workarounds <http://pypi.python.org/pypi/setuptools>`_)
Installation from source
^^^^^^^^^^^^^^^^^^^^^^^^
Users who have already set up python and setuptools (e.g. for another project) may skip python and / or setup tools installation.
Install python
""""""""""""""
^^^^^^^^^^^^^^
Download and run the Windows installer from `here <http://www.python.org/getit>`_
Users should pick the installer that fits their windows version and architecture.
Add python to windows path
""""""""""""""""""""""""""
^^^^^^^^^^^^^^^^^^^^^^^^^^
The following will allow users to run python and python scripts from command line.
......@@ -193,7 +189,7 @@ The following will allow users to run python and python scripts from command lin
.. warning:: C:\\Python should be replaced with the actual python path in the system, e.g. C:\\Python27
Install setuptools
""""""""""""""""""
^^^^^^^^^^^^^^^^^^
According to the corresponding `python org page <http://pypi.python.org/pypi/setuptools>`_, the setuptools installer doesn't currently work on 64bit machines.
......@@ -207,41 +203,9 @@ According to the corresponding `python org page <http://pypi.python.org/pypi/set
Installation finished
C:\Downloads\>
Install GIT
"""""""""""
`Download GIT <http://git-scm.com/download/win>`_ and run the graphic installer. During the installation, users will be able to modify some installation options. The present guide is tested with the default selections.
After the installation is completed, a GIT standalone shell will be installed (a desktop shortcut is created, by default). Users are advised to run kamaki through this shell.
Install kamaki
""""""""""""""
* Run the GIT standalone shell
* Enter the location where kamaki will be installed, e.g. **C:\\**
^^^^^^^^^^^^^^
.. code-block:: console
$ cd /c/
* Download source from GRNET repository
.. code-block:: console
$ git clone http://code.grnet.gr/git/kamaki
Cloning into 'kamaki'...
Receiving objects: ...
Resolving Deltas: ...
* Enter source and install kamaki
.. code-block:: console
$ cd kamaki
$ python setup.py install
running install
...
Finished processing dependencies for kamaki==0.7
.. code-block:: console
$ kamaki --version
$ easy_setup kamaki
......@@ -54,7 +54,7 @@ network
image
Manage compute API and Plankton images.
Manage images on Plankton (and Compute).
store
......@@ -143,8 +143,8 @@ flavor commands
* info get flavor details
image commands and options
**************************
image commands
**************
* addmember Add a member to an image
* addproperty Add an OS-related property to an image
......@@ -152,15 +152,21 @@ image commands and options
* delmember Remove a member from an image
* delproperty Delete a property of an image
* info Get detailed information on an image
* list List images
* members Get image members
* meta Get image metadata
* properties Get properties related to OS installation in an image
* public List public images
* list List images accessible by user
* register (Re)Register an image
* setmembers Set the members of an image
* setproperty Update an existing property in an image
* shared List images shared by a member
* compute Compute Image API commands
* list List images
* delete Delete image
* info Get image details
* properties Get image properties
* delproperty Delete an image property
* setproperty Update an image property
network commands
......
......@@ -16,20 +16,22 @@ Kamaki interfaces rely on a list of configuration options. Be default, they are
Example 1.1: Set user token to myt0k3n==
$ kamaki set token myt0k3n==
$ kamaki config set token myt0k3n==
Optional features
-----------------
For installing any all of the following, consult the `kamaki installation guide <installation.html#install-progress-and-or-ansicolors-optional>`_
For installing any or all of the following, consult the `kamaki installation guide <installation.html#install-ansicolors>`_
* ansicolors
* Make command line / console user interface responses prettier with text formating (colors, bold, etc.)
* Can be switched on/off in kamaki configuration file: colors=on/off
* Has not been tested on non unix / linux based platforms
* progress
* Attach progress bars to various kamaki commands (e.g. kamaki store upload)
* If desired, progress version should be 1.0.2 or better
* mock
* For kamaki contributors only
* Allow unittests to run on kamaki.clients package
* Needs mock version 1.X or better
Any of the above features can be installed at any time before or after kamaki installation.
......@@ -140,9 +142,9 @@ The [global] group is treated by kamaki as a generic group for arbitrary options
a special package that is used to load cyclades virtual network commands to kamaki UIs. Don't touch this unless you know what you are doing.
* image.url <Plankton image service url>
the url of the Plankton service. Set to Okeanos.grnet.gr Plankton service be default. Users should set a different value if they need to use a different service.
the url of the Plankton service. Set to Okeanos.grnet.gr Plankton service by default. Users should set a different value if they need to use a different service. Note that the *image compute* commands are depended on the compute.url instead.
* image.cli <UI command specifications for Plankton and Cyclades image service>
* image.cli <UI command specifications for Plankton (and Compute) image service>
a special package that is used to load image-related commands to kamaki UIs. Don't touch this unless you know what you are doing.
* astakos.url <Astakos authentication service url>
......@@ -257,3 +259,8 @@ A quotaholder client is introduced as an advanced feature. Quotaholder client is
url=<URL of quotaholder service>
Quotaholder is not tested in livetest
The unit testing system
"""""""""""""""""""""""
Kamaki container a set of finegrained unit tests for the kamaki.clients package. This set is not used when kamaki is running. Instead, it is aimed to developers who debug or extent the kamaki clients library. For more information, check the `Going Agile <developers/extending-clients-api.html#going-agile>`_ entry at the `developers section <developers/extending-clients-api.html>`_.
......@@ -118,7 +118,7 @@ To see the command groups, use -h or --help like in example 1.3.1. In the same w
config : Configuration commands
flavor : Compute/Cyclades API flavor commands
history: Command history
image : Compute/Cyclades or Plankton API image commands
image : Plankton (and Compute) Image API commands
network: Compute/Cyclades API network commands
server : Compute/Cyclades API server commands
store : Pithos+ storage commands
......@@ -456,7 +456,7 @@ An example (4.3.1) that showcases how top-level access improves user experience
* the flavor id
* the image id
It is often the case that a user who works in the context command, needs to create a new VM, but hasn't selected a flavor or an image id, or cannot recall the id of that flavor or image. Therefore, it is necessary to list all available flavors (flavor-list) or images (image-list). Both commands belong to different contexts.
It is often the case that a user who works in the context command, needs to create a new VM, but hasn't selected a flavor or an image id, or cannot recall the id of that flavor or image. Therefore, it is necessary to list all available flavors (flavor-list) or images (image-compute-list). Both commands belong to different contexts.
.. code-block:: console
:emphasize-lines: 1
......@@ -475,7 +475,7 @@ It is often the case that a user who works in the context command, needs to crea
disk : 10
ram : 2048
[server]: /image list
[server]: /image compute list
1580deb4-edb3-7a246c4c0528 (Ubuntu Desktop)
18a82962-43eb-8f8880af89d7 (Windows 7)
531aa018-9a40-a4bfe6a0caff (Windows XP)
......
......@@ -390,6 +390,9 @@ def main():
_init_session(parser.arguments)
from kamaki.cli.utils import suggest_missing
suggest_missing()
if parser.unparsed:
run_one_cmd(exe, parser)
elif _help:
......
......@@ -169,7 +169,7 @@ class ConfigArgument(Argument):
def get_groups(self):
return self.value.apis()
_config_arg = ConfigArgument(1, 'Path to configuration file', '--config')
_config_arg = ConfigArgument(1, 'Path to configuration file', '-c, --config')
class CmdLineConfigArgument(Argument):
......
......@@ -95,7 +95,7 @@ class Shell(Cmd):
def greet(self, version):
print('kamaki v%s - Interactive Shell\n' % version)
print('\t\exit \tterminate kamaki')
print('\t/exit \tterminate kamaki')