Doc updates and minor improvements

docs/installation.rst

@@ -11,65 +11,62 @@ This guide describes the standard installation process for kamaki, with the aspi
 Linux and Unix-like enviroments
 -------------------------------
 
-Ubuntu and Debian packages
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+Debian:
+^^^^^^^
 
 The following steps describe a command-line approach, but any graphic package manager can be used instead.
 
-Add the following to apt sources list
-"""""""""""""""""""""""""""""""""""""
+* As root, append the following to */etc/apt/sources.list* ::
 
-As root, append one of the following to */etc/apt/sources.list*:
+    deb http://apt.dev.grnet.gr/ squeeze main
+    deb http://apt2.dev.grnet.gr stable/
 
+* Make sure the GPG public key for the GRNET dev team is added:
 
-* Debian Stable::
+    .. code-block:: console
 
-    deb http://apt.dev.grnet.gr/ squeeze main
-    deb http://apt2.dev.grnet.gr stable/
+        $ sudo curl https://dev.grnet.gr/files/apt-grnetdev.pub|apt-key add -
 
-* Ubuntu::
+    otherwise *apt-get update* will produce GPG warnings.
 
-    deb http://ppa.launchpad.net/grnet/synnefo/ubuntu <UBUNTU_VERSION> main
-    deb-src http://ppa.launchpad.net/grnet/synnefo/ubuntu <UBUNTU_VERSION> main
+* Update the Debian sources:
 
-where <UBUNTU_VERSION> is the system Ubuntu version (e.g. precise)
+    .. code-block:: console
 
-Update
-""""""
+        $ sudo apt-get update
 
-* In Debian, make sure the GPG public key for the GRNET dev team is added:
+* Install kamaki:
 
     .. code-block:: console
 
-        $ curl https://dev.grnet.gr/files/apt-grnetdev.pub|apt-key add -
+        $ sudo apt-get install kamaki
 
-    otherwise *apt-get update* will produce GPG warnings.
+Ubuntu
+^^^^^^
+
+The following steps describe a command-line approach, but any graphic package manager can be used instead.
 
-* In Ubuntu, make sure the correct PPA signing key is installed:
+* Let ppa take care of the repository configuration:
 
     .. code-block:: console
 
         $ sudo apt-get install python-software-properties
         $ sudo add-apt-repository ppa:grnet/synnefo
 
-Finally:
-
-.. code-block:: console
-
-    $ sudo apt-get update
+* Update the Debian sources:
 
+    .. code-block:: console
 
-Install kamaki
-""""""""""""""
+        $ sudo apt-get update
 
-.. note:: The objpool package is available at synnefo repositories and is automatically installed as a dependency.
+* Install kamaki:
 
-.. code-block:: console
+    .. code-block:: console
 
-    $ sudo apt-get install kamaki
+        $ sudo apt-get install kamaki
 
-Install ansicolors and/or progress (Optional)
-"""""""""""""""""""""""""""""""""""""""""""""
+Install ansicolors and/or progress (Optional but recommended)
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
 
 .. code-block:: console
 

kamaki/clients/pithos.py

@@ -179,7 +179,7 @@ class PithosClient(PithosRestAPI):
 
     def _put_block(self, data, hash):
         #from random import randint
-        #if not randint(0, 3):
+        #if randint(0, 3):
         #    raise ClientError('BAD GATEWAY STUFF', 503)
         r = self.container_post(update=True,
             content_type='application/octet-stream',
@@ -265,16 +265,12 @@ class PithosClient(PithosRestAPI):
             r = self._put_block_async(data, hash, upload_gen)
             flying.append(r)
             unfinished = []
+            current_fails = 0
             for thread in flying:
                 unfinished = self._watch_thread_limit(unfinished)
-                if upload_gen:
-                    for i in range(len(unfinished), len(flying)):
-                        try:
-                            upload_gen.next()
-                        except:
-                            pass
                 if thread.exception:
                     failures.append(thread)
+                    current_fails += 1
                     if isinstance(thread.exception, ClientError)\
                     and thread.exception.status == 502:
                         self.POOLSIZE = self._thread_limit
@@ -285,6 +281,12 @@ class PithosClient(PithosRestAPI):
                         upload_gen.next()
                     except:
                         pass
+            if upload_gen:
+                for i in range(len(unfinished), len(flying) - current_fails):
+                    try:
+                        upload_gen.next()
+                    except:
+                        pass
             flying = unfinished
 
         for thread in flying:
@@ -367,9 +369,10 @@ class PithosClient(PithosRestAPI):
         else:
             upload_gen = None
 
-        retries = 3
+        retries = 7
         try:
             while retries:
+                sendlog.info('%s blocks missing' % len(missing))
                 num_of_blocks = len(missing)
                 missing = self._upload_missing_blocks(
                     missing,