Continuation of https://github.com/nim-lang/Nim/pull/19262
Fixes https://github.com/nim-lang/Nim/issues/19261
The parsing code is still too lenient (e.g. it will happily parse header
names with spaces in them, which is outright invalid by the spec), but I
didn't want to touch it beyond the simple changes to make sure that
`std/httpclient` won't throw `IndexDefect`s like it does now on those
cases:
- Multiline header values
- No colon after the header name
- No value after the header name + colon
One question remains - should I keep `toCaseInsensitive` exported in
`httpcore` or just copy-paste the implementation?
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Per https://datatracker.ietf.org/doc/html/rfc9110#name-user-agent a
User-Agent is defined as follows:
```
User-Agent = product *( RWS ( product / comment ) )
```
Where
```
product = token ["/" product-version]
product-version = token
```
In this case, `token` is defined in RFC 7230 -
https://datatracker.ietf.org/doc/html/rfc7230#section-3.2.6:
```
token = 1*tchar
tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*"
/ "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
/ DIGIT / ALPHA
; any VCHAR, except delimiters
```
or, in the original RFC 2616 -
https://datatracker.ietf.org/doc/html/rfc2616#section-2.2 (next page):
```
token = 1*<any CHAR except CTLs or separators>
separators = "(" | ")" | "<" | ">" | "@"
| "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
```
which means that a `token` cannot have whitespace. Not sure if this
should be in the breaking changelog section - theoretically, some
clients might've relied on the old Nim user-agent?
For some extra info, some other languages seem to have adopted the same
hyphen user agent to specify the language + module, e.g.:
-
https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L1679
(`Python-urllib/<version>`)
Fixes#22862.
* Add docs informing about gotcha of clients
It is a hidden problem that AsyncHttpClient-instances *can not* deal with multiple requests at once.
Similar to normal HttpClients they can only deal with sending one request at a time.
This is not told anywhere in the documentation, but critical information that should be available to the user.
* Update lib/pure/httpclient.nim
fixes typo
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* make more standard libraries work with `nimPreviewSlimSystem`
* typo
* part two
* Delete specutils.nim
* fixes more tests
* more fixes
* fixes tests
* fixes three more tests
* add formatfloat import
* fix
* last
Without this, the httpclient examples are essentially setting you up for failure. I was bitten by this when my app became unable to open any more sockets.
I'm not entirely sure how long this will relevant, as I hope destructors will be added to an upcoming version of the stdlib. But figured it was worth submitting anyways!
* Fixed missing newline after bound marker in mulipart post (#14794) and a problem where calling postContent with multipart data does not set content-length header.
* Update lib/pure/httpclient.nim
* Added comment outlining the reason for changes to httpclient.nim and added tests to ensure that multipart post has a newline at the end of the body, and that the content-length header is present.
* Fixed typo in comments.
* Removed redundant blank lines in thttpclient_standalone.nim.
Co-authored-by: Mark Pointing <mark@futurepoint.com.au>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* A new request should always have a new content-length
In [my last PR](https://github.com/nim-lang/Nim/pull/16618) I made a mistake by assuming that the client.headers were cleared on every request, like the Python version. So, due to the fact that Nim keeps the client headers, we need to clear the Content-Length header on each request (which makes sense because you almost never want to use the same Content-Length twice, but you may want to reuse other headers)
* Move content-length to newHeaders instead of in the global client headers
* Use single backticks
* Allow passing Uri instead of strings
* Teach httpclient about 308
* Deprecate request proc where httpMethod is string
* More use of HttpMethod enum
Also fix handling of 308, I forgot to add the hunk to the previous
commit.
* Well behaved redirect handler
* Also remove Transfer-Encoding
* Removed unused proc
* Secure redirection rules
Strip sensitive headers for cross-domain redirects.
* Allow httpMethod to be a string again
This way unknown http verbs can be used without any problem.
* Respect user-specified Host header
* Missed multipart argument.
* Try another method
* add changelog
* Fix hidden deprecation warning, parseEnum failing
* This is wrong
* Have to do it manually, parseEnum is not suitable
* Review comments
* update
Co-authored-by: LemonBoy <thatlemon@gmail.com>
Co-authored-by: Dominik Picheta <dominikpicheta@googlemail.com>
* DELETE requests should always have a content-length header
Not having DELETE in this list is causing hanging when trying to close webdriver sessions in [halonium](https://github.com/halonium/halonium/issues/10) and likely any other implementation of the webdriver protocol. Both at least chromedriver and geckodriver are affected by this issue.
* Change the content length calculation to match the http spec
For reference:
https://www.w3.org/Protocols/HTTP/1.0/draft-ietf-http-spec.html#Entity-Body
This commit indents the contents of a `code-block` in `httpclient.nim`
so that it displays correctly. The bug was introduced by 42a64245f8.
I did a quick search for other `code-block`s that are broken in the same
way, but the only other one I found (in `pegs.nim`) is not included in
the generated documentation.
* Error -> Defect for defects
The distinction between Error and Defect is subjective,
context-dependent and somewhat arbitrary, so when looking at an
exception, it's hard to guess what it is - this happens often when
looking at a `raises` list _without_ opening the corresponding
definition and digging through layers of inheritance.
With the help of a little consistency in naming, it's at least possible
to start disentangling the two error types and the standard lib can set
a good example here.
As the async httpclient is almost certainly the first async example beginners will want to try, we OWE it to them to give them a real example.
Example repeated here for clarity:
```nim
import asyncdispatch, httpclient
proc asyncProc(): Future[string] {.async.} =
var client = newAsyncHttpClient()
return await client.getContent("http://example.com")
echo waitFor asyncProc()
```
This is my first Nim contribution, please let me know if the code is right. (it runs on my machine, but may not be the best example)
