GitLab

All other exceptions are named after the error name in RFC2616 (HTTP/1.1).

Reviewed-by: amishchenko

Reviewed-by: amishchenko

Reviewed-by: amishchenko

Reviewed-by: amishchenko

Reviewed-by: iustinp

Reviewed-by: iustinp

Also add a comment next to the place where the SSL connection is shut
down.

Reviewed-by: iustinp

It'll be queried from other classes.

Reviewed-by: iustinp

It's more appropriate.

Reviewed-by: iustinp

Reviewed-by: iustinp

This is a leftover from old code.

Reviewed-by: iustinp

The combination of the current buffer splitting method and (4KB) buffer
size is very inefficient when writing big amounts of data. Just walking
over a 16 megabyte string using a 4K buffer takes (on a random computer)
1m06s, whereas using slices will decrease this to 0.080s, and slicing
with 32 KB size decreases this to 0.073s.

This means that uploading a big config file (it nears 1MB for big
clusters) will take more and more time per the number of nodes, since it
needs lots of slicing.

I happened upon this by accidentally setting all nodes as master
candidates, at which point just uploading the config file to all nodes
took 40s. Applying the patch decreases this to 15s (this probably can
still be optimized).

The patch also removes a duplicate constant (the one actually used is in
http/client.py), and changes the receive buffer size to use the same
constant.

Reviewed-by: imsnah

Reviewed-by: amishchenko

Reviewed-by: amishchenko

This includes a large rewrite of the HTTP server code. The handling of
OpenSSL errors had some problems that were hard to fix with its
structure. When preparing all of this, I realized that actually HTTP
is a message protocol and that the same code can be used on both the
server and client side to parse requests/responses, with only a few
differences. There are still a few TODOs in the code, but none should
be a show stopper. Many pylint warnings have been fixed, too.

The old code will be removed once all users have been migrated.

Reviewed-by: amishchenko

It should be consistent.

Reviewed-by: amishchenko

This is a preparation step for splitting the HTTP client and server code
into two separate modules.

Reviewed-by: amishchenko

This is a preparation step for splitting the HTTP client and server code
into two separate modules.

Reviewed-by: amishchenko

Reviewed-by: ultrotter

This is a leftover from removed code.

Reviewed-by: ultrotter

Reviewed-by: ultrotter

This is a preparation step to move the HTTP server class to the
same model as the HTTP client (polling, non-blocking I/O, better
OpenSSL error handling).

Reviewed-by: ultrotter

Reviewed-by: ultrotter

We don't need it anymore and it wouldn't work as it is, anyway.

Reviewed-by: ultrotter

Reusing the private key and certificate objects gave us problems. This
patch changes the code to only cache the PEM data, but the objects
themselves. For every socket, the private key and certificate objects
are created again.

Reviewed-by: iustinp

Otherwise we would read them for every request the HTTP client
makes against a server and this is not needed.

Reviewed-by: iustinp

This allows a single HttpClientManager to be used from more than one
thread at the same time. We discussed having one HttpClientManager
per job queue thread. Assuming there should be one HTTP thread per
node, this would mean quadratic growth with the number of nodes. By
being able to reuse the manager, this problem is a defused a bit.

This patch also fixes a typo in HttpClientRequestExecutor.

Reviewed-by: iustinp

Reviewed-by: imsnah

Implementing SSL for the HTTP client required more work than I expected.
For correct error handling, quite a lot of code is needed. To avoid
code duplication, I moved a lot of the socket handling code into a
single function named _SocketOperation. It takes care of the polling
and error handling for OpenSSL. At the same time, the functions calling
it become a lot smaller.

This patch brings a few more changes:
- Error handling for socket.connect_ex (e.g. name resolution failures)
- Rename "ResponseError" exception class to "_HttpClientError" --
  it's an internal class anyway

Reviewed-by: iustinp

The same code will be used by the HTTP client.

Reviewed-by: iustinp

This is an expected error message and will be used in two places.

Reviewed-by: iustinp

It is based on the WorkerPool class which is already used for the job
queue and master daemon. Each request must be encapsulated in an
instance of HttpClientRequest, which will then be passed to
HttpClientManager for processing. Upon completion, the request object
contains the response or error message.

While not yet enabled, the HTTP client implementation has been designed
with HTTP/1.1 and keep-alive in mind. For now it only uses HTTP/1.0,
though.

HttpClientManager will likely need more changes when integrating with
the RPC layer. SSL will also be integrated in a second step.

Reviewed-by: iustinp, ultrotter

Reviewed-by: iustinp

As a preparation for the new HTTP client class, add more constants
to ganeti.http.

Reviewed-by: iustinp

Reviewed-by: killerfoxi

At the same time, simplify the interface a bit by not using a tuple.

Reviewed-by: killerfoxi, ultrotter

It turned out that clients not sending a full request will stop
us from responding to further requests. This patch leverages the
situation a bit by always forking before handling the request,
but we still have DoS situations.

Reviewed-by: ultrotter

Reviewed-by: imsnah

All users of this code have been migrated to the new and shiny HttpServer
class.

It also fixes a typo in the ApacheLogfile unittests. It has not yet been
decided whether we should keep ApacheLogfile or not, hence leaving it in.

Reviewed-by: ultrotter

Reviewed-by: imsnah