Commit 05927995 authored by Michael Hanselmann's avatar Michael Hanselmann
Browse files

rpc: Disable HTTP client pool and reduce memory consumption



We noticed that “ganeti-masterd” can use large amounts of memory,
especially on large clusters. Measurements showed a single PycURL client
using about 500 kB of heap memory (the actual usage depends on versions,
build options and settings).

The RPC client uses a per-thread HTTP client pool with one client per
node. At this time there are 41 non-main threads (25 for the job queue
and 16 for client requests). This means the HTTP client pools use a lot
of memory (ca. 200 MB for 10 nodes, ca. 1 GB for 50 nodes).

This patch disables the per-thread HTTP client pool. No cleanup of
unused code is done. That will be done in the master branch only.
Signed-off-by: default avatarMichael Hanselmann <hansmi@google.com>
Reviewed-by: default avatarIustin Pop <iustin@google.com>
parent fd121c8e
......@@ -372,7 +372,7 @@ class Client:
"""
if not http_pool:
http_pool = _thread_local.GetHttpClientPool()
http_pool = http.client.HttpClientPool(_ConfigRpcCurl)
http_pool.ProcessRequests(self._request.values())
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment