• Michael Hanselmann's avatar
    rpc: Disable HTTP client pool and reduce memory consumption · 05927995
    Michael Hanselmann authored
    
    
    We noticed that “ganeti-masterd” can use large amounts of memory,
    especially on large clusters. Measurements showed a single PycURL client
    using about 500 kB of heap memory (the actual usage depends on versions,
    build options and settings).
    
    The RPC client uses a per-thread HTTP client pool with one client per
    node. At this time there are 41 non-main threads (25 for the job queue
    and 16 for client requests). This means the HTTP client pools use a lot
    of memory (ca. 200 MB for 10 nodes, ca. 1 GB for 50 nodes).
    
    This patch disables the per-thread HTTP client pool. No cleanup of
    unused code is done. That will be done in the master branch only.
    Signed-off-by: default avatarMichael Hanselmann <hansmi@google.com>
    Reviewed-by: default avatarIustin Pop <iustin@google.com>
    05927995
rpc.py 46.5 KB