Skip to content
  • Michael Hanselmann's avatar
    ensure-dirs: Speed up when using big queues · 196d70fa
    Michael Hanselmann authored
    
    
    The “ensure-dirs” script as included in Ganeti 2.3 is very slow when
    working with big queues requiring a change of permissions on many or all
    files.
    
    $ find /var/lib/ganeti/queue/ | wc -l
    52354
    
    Before this change:
    $ time /usr/local/lib/ganeti/ensure-dirs -f
    real    16m4.739s
    
    While not adressed in this patch, I'd like to record the overall
    ineffiency of the “ensure-dirs” script, even after this change:
    
    $ time /usr/local/lib/ganeti/ensure-dirs -f
    real    5m57.362s
    […]
    $ strace -e clone,execve -f -c /usr/local/lib/ganeti/ensure-dirs -f
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
     50.08    5.147090          49    104774           clone
     49.92    5.131094          49    104739           execve
    
    More changes will be needed. Just for comparision, a small Python
    snippet changing permissions on all files (“ensure-dirs” changes the
    owner too):
    
    $ time python -c 'import os; from ganeti import utils;
    [os.chmod(i, 0644) for i in
    utils.ListVisibleFiles("/var/lib/ganeti/queue/archive/big")]'
    real    0m0.605s
    […]
    
    Signed-off-by: default avatarMichael Hanselmann <hansmi@google.com>
    Reviewed-by: default avatarIustin Pop <iustin@google.com>
    196d70fa