1. 20 Feb, 2013 1 commit
    • Iustin Pop's avatar
      Add a Ganeti-specific implementation of Curl Multi · cc40185c
      Iustin Pop authored
      
      
      As we want to be able to run queries against multiple nodes in
      parallel, and furthermore in parallel with other work, we need to
      implement the Curl Multi interface (see libcurl-multi(3)).
      
      This patch adds a Ganeti-specific such implementation, to be used
      until upstream Curl bindings provide it. The implemented interface
      (there are two 'multi' interfaces) is the older curl_multi_perform(3).
      
      It has one downside (which is also, somewhat, and advantage): we use
      polling instead of more properly select() or poll. This is due to how
      waiting for FDs is implemented in Haskell: currently, it's not
      possible to wait for multiple FDs at once nicely, so we'd have to fork
      many threads for each FD to be watched, or alternatively one could use
      FFI select, but that would block the entire runtime.
      
      With the current poll method, the implementation achieves consistent
      ~100 RPC/s per second (with 10 multi interfaces running in parallel,
      each with 10 easy handles), and over ~1 hour of runtime the memory
      usage is stable, so memory allocation/deallocation (manual when
      dealing with FFI) _seems_ to be well handled.
      
      Future optimisations could be to move to curl_multi_socket_action(3),
      which might allow better integration with the Haskell runtime.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichele Tartara <mtartara@google.com>
      cc40185c