Skip to content
Snippets Groups Projects
Select Git revision
0 results
Search by author
  • Any Author
0 authors
  1. Mar 14, 2022
  3. Mar 13, 2022
  5. Mar 12, 2022
  7. Mar 10, 2022
  9. Mar 05, 2022
  11. Mar 04, 2022
  13. Feb 26, 2022
  15. Feb 25, 2022
  17. Feb 24, 2022
  19. Feb 23, 2022
    • Andrej Shadura's avatar
      Set Content-Type/Length headers for the payload · 2e7d1fd6
      Andrej Shadura authored 
      
If Content-Type or Content-Length headers were supplied with the
original upload, pass them through to the upstream. Otherwise,
set Content-Length to the actual size of the payload; this also
disables chunked encoding of the payload.

Signed-off-by: default avatarAndrej Shadura <andrew.shadura@collabora.co.uk>
      2e7d1fd6
    • Andrej Shadura's avatar
      Add upload buffering, enabled by default on clients · 736305ad
      Andrej Shadura authored 
      Quart doesn’t yet support backpressure to handle situations when an
application cannot handle the incoming data as fast as it’s received.

This happens, for example, when the proxy client run on the same machine
as the worker, and a worker attempts to upload a multi-gigabyte build
artifact. In this case, the artifact is consumed almost immediately,
as the local connection is fast enough to allow this, but the upload to
the upstream proxy doesn’t keep up and the artifact ends up being
buffered in memory. If there’s not enough RAM and swap space, the upload
terminates with an out of memory exception.

To work around this until this missing feature is implemented in Quart,
we buffer incoming uploads at clients in a temporary files, as disk
space is still cheaper than RAM at the time of writing.

Uploads on the server are not buffered on the server, as typically the
upload speed isn’t high enough to cause excessive buffering.

This brings a new dependency on aiofiles which implements async temporary
files, as using a normal tempfile functionality would block on I/O.

While httpx supports passing file-like objects as content directly, the
current implementation doesn’t use them in an optimal way (despite a fix in
https://github.com/encode/httpx/pull/1948

), so instead of that, we read
the temporary file chunk by chunk manually.

Signed-off-by: default avatarAndrej Shadura <andrew.shadura@collabora.co.uk>
      736305ad
  21. Feb 22, 2022
    • Andrej Shadura's avatar
      Use ASGI scope instead of Host header to reliably detect server port · 81fea97b
      Andrej Shadura authored 
      
Host header can be spoofed, making it possible to bypass authentication.
Since proxy exposes one port per worker for the backend to connect to,
it cannot require authentication on these ports. The port exposed to the
outside though, needs to be authenticated to prevent information leaks,
so the proxy server needs to be able to reliably distinguish clients
connecting to the proxy from the outside and the backend services trying
to talk to (emulated) workers.

Signed-off-by: default avatarAndrej Shadura <andrew.shadura@collabora.co.uk>
      81fea97b
  23. Feb 16, 2022
Apertis Website Terms of Use Privacy Policy