json-rpc-test-server: add a pure HTTP listener #5

New Issue

p · 2020-10-15T01:55:56+02:00

p commented

2020-10-15 01:55:56 +02:00

It will transform HTTP request headers to the CGI format and process CGI result headers ("Status" needs to be read out; we control the data, so the parser may be a trivial split on : or even a filter into a strv and re-join with CRLF).

Reuse experience from the shell's co-process backend and code from the other listeners. It should be possible to refactor the code so that the new listener can at least pass down the request to the WebSocket handler when we see an upgrade. It even seems that the HTTP part wants to be split out entirely--as soon as we see an upgrade request, initialize and run an internal ws_handler to which all further push and shutdown invocations will be forwarded. It just needs initial access to parsed out headers and a few bits from http-parser.

We only claim to be a JSON-RPC 2.0 server, so a few crazy HTTP statuses don't seem to matter a whole lot. After all, the CGI part already isn't very standards-compliant. And given that we're a test server, DoS concerns are insignificant. While the daemon was meant to be part of a full CI system originally, I've lost all intent to use C for this sort of problem. Go does it better.

That being said: "Expect: 100-continue" just means we need to send a "HTTP/1.1 100 Continue" and that's that, unless it's an HTTP/1.0 request, in which case we ignore this. Implementation of methods such as OPTIONS and HEAD is an entirely separate concern, shared with CGI. Respond to unknown Transfer-Encodings with "501 Not Implemented"--the last one is always "chunked", or http-parser complains. Respond to unknown Content-Encoding with "415 Unsupported Media Type". Given that CGI requires CONTENT_LENGTH for request data, it is necessary to dechunk it first and compute it later.

https://tools.ietf.org/html/rfc2145 is an older document supporting the use of HTTP/1.1 in responses even for HTTP/1.0 requests, we only must ensure that we don't rely on headers unsupported by that version. This includes the "Connection" header. Never use "keep-alive" for an HTTP/1.0 client, send it "Connection: close". We're told to query http_should_keep_alive() in on_{headers,message}_complete--it is at the end of a message that we'll process it, because of "keep-alive" support. on_message_complete is always called, even with a missing body.

This really concludes the (sub)project.

Stage 1: make normal HTTP requests work
Stage 2: replace client_ws

It will transform HTTP request headers to the CGI format and process CGI result headers ("Status" needs to be read out; we control the data, so the parser may be a trivial split on `: ` or even a filter into a `strv` and re-join with CRLF). Reuse experience from the shell's co-process backend and code from the other listeners. It should be possible to refactor the code so that the new listener can at least pass down the request to the WebSocket handler when we see an upgrade. It even seems that the HTTP part wants to be split out entirely--as soon as we see an upgrade request, initialize and run an internal `ws_handler` to which all further `push` and `shutdown` invocations will be forwarded. It just needs initial access to parsed out headers and a few bits from http-parser. We only claim to be a JSON-RPC 2.0 server, so a few crazy HTTP statuses don't seem to matter a whole lot. After all, the CGI part already isn't very standards-compliant. And given that we're a test server, DoS concerns are insignificant. While the daemon was meant to be part of a full CI system originally, I've lost all intent to use C for this sort of problem. Go does it better. That being said: "Expect: 100-continue" just means we need to send a "HTTP/1.1 100 Continue" and that's that, unless it's an HTTP/1.0 request, in which case we ignore this. Implementation of methods such as OPTIONS and HEAD is an entirely separate concern, shared with CGI. Respond to unknown `Transfer-Encoding`s with "501 Not Implemented"--the last one is always "chunked", or http-parser complains. Respond to unknown `Content-Encoding` with "415 Unsupported Media Type". Given that CGI requires CONTENT_LENGTH for request data, it is necessary to dechunk it first and compute it later. https://tools.ietf.org/html/rfc2145 is an older document supporting the use of HTTP/1.1 in responses even for HTTP/1.0 requests, we only must ensure that we don't rely on headers unsupported by that version. This includes the "Connection" header. Never use "keep-alive" for an HTTP/1.0 client, send it "Connection: close". We're told to query `http_should_keep_alive()` in `on_{headers,message}_complete`--it is at the end of a message that we'll process it, because of "keep-alive" support. `on_message_complete` is always called, even with a missing body. This really concludes the (sub)project. - Stage 1: make normal HTTP requests work - Stage 2: replace `client_ws`

p self-assigned this 2020-10-15 01:55:57 +02:00

p added this to the v1.2.0 milestone 2020-10-15 03:37:19 +02:00

p added a new dependency 2020-10-15 03:50:07 +02:00

#7 Clean up HTTP and JSON-RPC parsing code

p added a new dependency 2020-10-18 03:42:51 +02:00

#8 Lose the dependency on cURL

p added the

WIP

label 2020-10-27 11:28:26 +01:00

p added this to the v1.2.0 project 2021-01-01 18:11:25 +01:00

Sign in to join this conversation.