[8.x] Backport http2 changes from 9.x#20456
Closed
kjin wants to merge 77 commits intonodejs:v8.x-stagingfrom
Closed
[8.x] Backport http2 changes from 9.x#20456kjin wants to merge 77 commits intonodejs:v8.x-stagingfrom
kjin wants to merge 77 commits intonodejs:v8.x-stagingfrom
Conversation
Instead of having per-request callbacks, always call a callback on the `StreamBase` instance itself for `WriteWrap` and `ShutdownWrap`. This makes `WriteWrap` cleanup consistent for all stream classes, since the after-write callback is always the same now. If special handling is needed for writes that happen to a sub-class, `AfterWrite` can be overridden by that class, rather than that class providing its own callback (e.g. updating the write queue size for libuv streams). If special handling is needed for writes that happen on another stream instance, the existing `after_write_cb()` callback is used for that (e.g. custom code after writing to the transport from a TLS stream). As a nice bonus, this also makes `WriteWrap` and `ShutdownWrap` instances slightly smaller. PR-URL: nodejs#17564 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]>
PR-URL: nodejs#17665 Fixes: nodejs#17636 Refs: nodejs#16482 Refs: nodejs#16860 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Timothy Gu <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]>
Currently, when configured --without-ssl tests that use
process.binding('crypto') fail with the following error:
=== release test-accessor-properties ===
Path: parallel/test-accessor-properties
node/test/parallel/test-accessor-properties.js:16
const crypto = process.binding('crypto');
^
Error: No such module: crypto
at Object.<anonymous> (test-accessor-properties.js:16:24)
at Module._compile (module.js:660:30)
at Object.Module._extensions..js (module.js:671:10)
at Module.load (module.js:577:32)
at tryModuleLoad (module.js:517:12)
at Function.Module._load (module.js:509:3)
at Function.Module.runMain (module.js:701:10)
at startup (bootstrap_node.js:194:16)
at bootstrap_node.js:645:3
This commit adds a hasCrypto check.
PR-URL: nodejs#17867
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Remove reliance on V8-specific error messages in test/parallel/test-tls-external-accessor.js. Check that the error is a `TypeError`. The test should now be successful without modification using ChakraCore. PR-URL: nodejs#16272 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Yuta Hiroto <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Tobias Nießen <[email protected]>
Refactor and simplify the perf_hooks native internals. PR-URL: nodejs#17822 Reviewed-By: Anna Henningsen <[email protected]>
`OnCallbackPadding` on the native side already clamps the return value into the right range, so there’s not need to also do that on the JS side. Also, use `>>> 0` instead of `| 0` to get an uint32, since the communication with C++ land happens through an Uint32Array. PR-URL: nodejs#17717 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
PR-URL: nodejs#17908 Refs: nodejs#17746 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>
Adds the possibility to keep a strong persistent reference to a JS object while a `SetImmediate()` call is in effect. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
Calling into JS land from GC is not allowed, so delay the resolution of pending pings when a session is destroyed. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
Introduce an `Http2Scope` class that, when it goes out of scope, checks whether a write to the network is desired by nghttp2. If that is the case, schedule a write using `SetImmediate()` rather than a custom per-session libuv handle. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
PR-URL: nodejs#17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Backport-PR-URL: nodejs#18050
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> This is a significant cleanup and refactoring of the cleanup/close/destroy logic for Http2Stream and Http2Session. There are significant changes here in the timing and ordering of cleanup logic, JS apis. and various related necessary edits.
`nghttp2_stream_write_t` was not a necessary redirection layer and came with the cost of one additional allocation per stream write. Also, having both `nghttp2_stream_write` and `nghttp2_stream_write_t` as identifiers did not help with readability. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17718 Reviewed-By: James M Snell <[email protected]>
- Only finish outgoing `WriteWrap`s once data has actually been passed to the underlying socket. - This makes HTTP2 streams respect backpressure - Use `DoTryWrite` as a shortcut for sending out as much of the data synchronously without blocking as possible - Use `NGHTTP2_DATA_FLAG_NO_COPY` to avoid copying DATA frame contents into nghttp2’s buffers before sending them out. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17718 Reviewed-By: James M Snell <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17763 Refs: nodejs#17746 Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17863 Fixes: nodejs#17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
Keep a local handle as a reference to the JS `Http2Session` object so that it will not be garbage collected when inside an `Http2Scope`, because the presence of the latter usually indicates that further actions on the session object are expected. Strictly speaking, storing the `session_handle_` as a property on the scope object is not necessary, but this is not very costly and makes the code more obviously correct. Fixes: nodejs#17840 Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17863 Fixes: nodejs#17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17620 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
Collect and report basic timing information about `Http2Session` and `Http2Stream` instances. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17906 Refs: nodejs#17746 Reviewed-By: Matteo Collina <[email protected]>
This is the portion of be2cbcc that is not in dea44b9. Update performance_state to use AliasedBuffer and update usage sites. PR-URL: nodejs#18300 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>
Strictly limit the number of concurrent streams based on the current setting of the MAX_CONCURRENT_STREAMS setting Backport-PR-URL: nodejs#18050 PR-URL: nodejs#16766 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]>
This commit also includes prerequisite error definitions from c75f87c and 1698c8e. Add support for sending and receiving ALTSVC frames. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17917 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17935 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Add new properties to `Http2Session` to identify alpnProtocol, and indicator about whether the session is TLS or not, and initial support for origin set (preparinng for `ORIGIN` frame support and the client-side `Pool` implementation. The `originSet` is the set of origins for which an `Http2Session` may be considered authoritative. Per the `ORIGIN` frame spec, the originSet is only valid on TLS connections, so this is only exposed when using a `TLSSocket`. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17935 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Add a padding strategy option that makes a best attempt to ensure that total frame length for DATA and HEADERS frames are aligned on multiples of 8-bytes. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17938 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17942 Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17942 Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17954 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
The maxSessionMemory is a cap for the amount of memory an Http2Session is permitted to consume. If exceeded, new `Http2Stream` sessions will be rejected with an `ENHANCE_YOUR_CALM` error and existing `Http2Stream` instances that are still receiving headers will be terminated with an `ENHANCE_YOUR_CALM` error. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17967 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>
Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17968 Reviewed-By: Anna Henningsen <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18134 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18151 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Michael Dawson <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18166 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Add a C++ variant of `Buffer.from(arrayBuffer, offset, length)`. Backport-PR-URL: #20456 PR-URL: #18030 Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Refactor the read mechanism to completely avoid copying. Instead of copying individual `DATA` frame contents into buffers, create `ArrayBuffer` instances for all socket reads and emit slices of those `ArrayBuffer`s to JS. Backport-PR-URL: #20456 PR-URL: #18030 Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Verify that server close callbacks are being called Backport-PR-URL: #20456 PR-URL: #18182 Refs: #18176 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Improve documentation of callback signature of http2Stream.pushStream() function to align with the changes made in #17406. Backport-PR-URL: #20456 PR-URL: #18258 Fixes: #18198 Refs: #17406 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18407 Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18439 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18602 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
This adds the Http1IncomingMessage and Http1ServerReponse options to http2.createServer(). Backport-PR-URL: #20456 PR-URL: #15752 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Evan Lucas <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
This adds the optional options argument to `http.createServer()`. It contains two options: the `IncomingMessage` and `ServerReponse` option. Backport-PR-URL: #20456 PR-URL: #15752 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Evan Lucas <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. Backport-PR-URL: #20456 PR-URL: #15560 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18609 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Upcoming changes to move away from synchronous I/O on the main thread will imply that using the same file descriptor to respond on multiple HTTP/2 streams at the same time is invalid, because at least on Windows `uv_fs_read()` is race-y. Therefore, warn against such usage. Backport-PR-URL: #20456 PR-URL: #18762 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Currenlty when configuring --debug-http2 /test/parallel/test-http2-getpackedsettings.js will segment fault: $ out/Debug/node test/parallel/test-http2-getpackedsettings.js Segmentation fault: 11 This is happening because the settings is created with the Environment in PackSettings: Http2Session::Http2Settings settings(env); This will cause the session to be set to nullptr. When the init function is later called the expanded DEBUG_HTTP2SESSION macro will cause the segment fault when the session is dereferenced. Backport-PR-URL: #20456 PR-URL: #18815 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18872 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michael Dawson <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Previously, we performed casts that are considered undefined behavior. Instead, just define `ssize_t` for nghttp2 the same way we define it for the rest of Node. Also, remove a TODO comment that would probably also be *technically* correct but shouldn’t matter as long as nobody is complaining. Backport-PR-URL: #20456 PR-URL: #18565 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Send a human-readable HTTP/1 response in case of an unexpected ALPN protocol. This helps with debugging this condition, since previously the only result of it would be a closed socket. Backport-PR-URL: #20456 PR-URL: #18986 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Previously, if `session.destroy()` was called with an error object, the information contained in it would be discarded and a generic `ERR_HTTP2_STREAM_CANCEL` would be used for all pending streams. Instead, make the information from the original error object available. Backport-PR-URL: #20456 PR-URL: #18988 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #18924 Fixes: #18169 Refs: #18673 Refs: https://github.com/nodejs/node/blob/v9.5.0/src/node_http2.cc#L1481-L1484 Refs: https://github.com/nodejs/node/blob/v9.5.0/lib/_http_outgoing.js#L659-L661 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Khaidi Chu <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Refs: #18673 Backport-PR-URL: #20456 PR-URL: #18924 Refs: #18673 Refs: https://github.com/nodejs/node/blob/v9.5.0/src/node_http2.cc#L1481-L1484 Refs: https://github.com/nodejs/node/blob/v9.5.0/lib/_http_outgoing.js#L659-L661 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
The test was flaky because it relied on a specific order of asynchronous operation that were fired paralellely. This was true on most platform and conditions, but not all the time. See: #18986 Backport-PR-URL: #20456 PR-URL: #19093 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
This fixes a crash that occurred when a `Http2Stream` write is completed after it is already destroyed. Instead, don’t destroy the stream in that case and wait for GC to take over. Backport-PR-URL: #19185 Backport-PR-URL: #20456 PR-URL: #19002 Fixes: #18973 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #19476 Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
May 2, 2018
Backport-PR-URL: #20456 PR-URL: #19451 Reviewed-By: Matteo Collina <[email protected]>
Contributor
|
I had to back out 5ea29ef as it is a semver-minor change to HTTP |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a backport of commits related to http2 changes from the 9.x branch to 8.x, as based off of the list of commits in this gist. I took the liberty of adding a few other changes which seemed like they needed to be backported as well as pre-requisites -- this fork of the earlier mentioned gist illustrates how I deviated from that list. Concerning this list:
I guess the list is slightly outdated so this PR doesn't completely cover every commit, but I think it's a start.
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes/cc @MylesBorins @nodejs/http2 @jasnell