Motivation:
Our HTTP code handles only HTTP/1.X. There is no reason to support
HTTP/0.9, and we cannot safely handle a major protocol higher than 1 in
this code, so we should simply treat requests/responses claiming to be
of those protocols as errors.
Modifications:
HTTPDecoder now checks the major version is equal to 1 before it
continues with parsing. If it hits an error, that error will be propagated
out to the user.
Result:
Better resilience against bad HTTP messages.
Motivation:
Add primitive for folding futuristic values.
Modifications:
Add a new EventLoopFuture extension containing the implementation of
`fold<U>(futures:combiningFunction)`.
Result:
An API for foding futures.
Motivation:
The current test function does not wait for the futures from close to
return, and so they may in principle be leaked. This obscures errors that
the test might hit with precondition failures.
Modifications:
Changed the defer statements to wait for the futures.
Result:
Fewer spurious test failures.
Motivation:
Adding @convention(c) is beneficial for performance of variables bound
to C functions so we should do that in ByteBuffer.
Modifications:
Adds @convention(c) to the malloc/realloc/free constants in ByteBuffer
Result:
Slightly better performance.
Motivation:
The channel leak detection test currently gates its progress based on
the firing of the promise passed to close(). That promise fires way, way
before the channel stops scheduling work, which means that it's highly
likely the event loop will still be holding a ref to the channel shortly
after that promise fires.
That causes the test to fail. Failing tests are bad.
Modifications:
Wait for the closeFuture instead. This fires at the end of the last
callback a socket channel ever schedules, so will fire just before the
channel drops out of scope.
Result:
Less flaky tests.
Motivation:
When calling Selector.whenReady(...) and using a SelectorStrategy.blockUntil(...) we may end up re-arm the timerfd each time even if the wakeup time did not change.
This is expensive and can be avoided in most situations.
Modifications:
Keep track of the earliest scheduled timer and only schedule a new one if it would fire sooner.
Result:
Less overhead when using epoll and timers.
Motivation:
We had a lot of problems with the Channel lifecycle statemachine as it
wasn't explicit, this fixes this. Additionally, it asserts a lot more.
Modifications:
- made Channel lifecycle statemachine explicit
- lots of asserts
Result:
- hopefully the state machine works better
- the asserts should guide our way to work on in corner cases as well
Motivation:
We only use our GetaddrinfoResolver for SocketChannels (TCP stream channels), so we should set some hints for getaddrinfo.
Modifications:
Set proto and socktype hints.
Result:
More correct usage of getaddrinfo. Fixes https://github.com/apple/swift-nio/issues/201.
Motivation:
We triggered extra allocations for the generics on PriorityQueue/Heap.
They unfortunately can't be solved using `@_specialize` as the element
type is `ScheduledTask`.
Fortunately we never exposed those types externally so we can just
inline without breaking the public API.
Modifications:
Moved everything from the `NIOPriorityQueue` module into the `NIO`
module.
Result:
Less allocations, more happiness.
Motivation:
We need to update isActive before notify the promises of connect and close to correctly refect Channel.isActive in callbacks.
Modifications:
- Correctly update isActive before notify promises.
- Add unit test.
Result:
Correct Channel state in callbacks.
Motiviation:
When the Selector is closed we also close the registered Channels and notify the future once all the close operations complete. The problem here is that while the close operation may be complete already there may still be events flowing through the ChannelPipeline. The only way to ensure this not happens anymore is to take the closeFuture into account of each Channel (as the closeFuture will only be notified once all is done for a Channel).
Modification:
- Also take the closeFuture into account per Channel
- Add test
Result:
Safer shutdown of EventLoopGroup.
Motivation:
ServerSocketChannel should just fail the EventLoopFuture for unsupported operations and not crash.
Modifications:
- Correctly fail the EventLoopFuture for unsupported operations
- Add unit test.
Result:
No more crashes when users invoke unsupported operations.
Motivation:
The Head and Tail ChannelHandlers should not be accessible as these are implementation details of the ChannelPipeline and removal of these will even make the whole ChannelPipeline not work anymore.
Modifications:
- Make it impossible to access / removal the Head and Tail ChannelHandlers
- Exclude the Head and Tail ChannelHandler from the debug String
- Add unit tests.
Result:
More robust ChannelPipeline.
Motivation:
Fix inconsistency of <Void> and <()> occurances in the codebase.
Modifications:
Changed EventLoop{Future,Promise}<()> occurances to EventLoop{Future,Promise}<Void>
Result:
Consistent code style for EventLoopPromise and EventLoopFuture.
Motivation:
b8055f7467 added some code to guard against the case when a fd is deregistered while processing ready events. On Linux it also added an assert which tried to verify that the fd in question is not registered anymore via epoll. The problem here is that it may also be closed which will have epoll_ctl fail with EBADF
Modifications:
Remove assert and add test-case which would have triggered the assert.
Result:
No more assertion error on linux.
Motivation:
`readIfNeeded` was triggered straight after the channel was registered
but _before_ `channelActive` was called which is wrong.
Modifications:
Moved the calls to `readIfNeeded` into `becomeActive0` and removed them
everywhere else (except some error path).
Result:
handle ChannelHandler lifecycle more correctly.
Motivation:
It's possible someone deregistered a Selectable while Selector.whenReady(...) is processed and an even for the now deregistered Selectable is ready. In this case we crashed (due a force-unwrap) on linux.
Modifications:
- Skip the processing of the Selectable if deregistered (by replacing force-unwrap with an if let).
- Add unit test.
Result:
No more crashes caused by this on Linux
Motivation:
If connectSocket() completes directly we failed to mark the Channel active by calling becomeActive0(...).
Modifications:
Correctly call becomeActive0(...)
Result:
Mark Channel active if connect succeed directly.
Motivation:
We should return a failed EventLoopFuture when getOption(...) / setOption(...) is called on a closed Channel as otherwise it may not be safe to modify the Channel ioptions after its closed.
Modifications:
- Add guards that check if the Channel is still open and if not fail the operation.
- Add testcase
Result:
Calling getOption(...) / setOption(...) on a closed Channel fails.
Motivation:
The HappyEyeballs state machine incorrectly assumed that it would always
be able to cancel any scheduled task that would execute on the event loop
that the state machine itself was running on. This is not accurate: if
the scheduled task is scheduled to execute at roughly the same time as
another callback into the state machine then it may be uncancellable.
Modifications:
Allowed the three scheduled tasks (connect timeout, connect delay, and
resolver delay) to fire in completed.
Result:
No fatalErrors in window conditions under high load
Motivation:
If we don't wait for the promise to succeed it may not be succeded
before we tear the loop down. This causes precondition failures.
Modifications:
We now always wait for the promise to succeed.
Result:
My tests will run.
Motivation:
fireChannelActive(...) must always be called before fireChannelRead(...) which was not the cause all of the times as once ChannelOptions were applied to the accepted SocketChannel it was possible that read0() was called during this. This would then lead to an assert error as it tried to reregister on the Selector before we ever registered.
Modifications:
- Ensure we only try to reregister once we have registered to the Selector
- Only trigger read0() or pauseRead0() if the autoRead value actually changed
- Add unit test
Result:
Correct order of events and no crashes.
Motivation:
When running swift test two warnings were printed.
Modifications:
- Signal that we want to ignore the return value
- Use boolean test in the while loop
Result:
Cleaner build
Motivation:
Changes the Boolean `fulfilled` to `isFulfilled` to make it more clear it is in fact a Boolean.
Also changes a `func isRecoverable()` to `var isRecoverable`.
Both of these were non-public, so this is no breaking change.
Modifications:
Renamed one Boolean and changed one function to a variable.
Result:
No use of unncessary (and potentially misinforming) functions, as well as adhering to the Swift boolean naming guidelines.
Motivation:
There are quite a few `switch`es that cover just two cases. Explicitly state this via an `if case` + `else` structure.
Modifications:
Changes something like:
switch a {
case .b:
return c
default:
return d
}
to
if case .b = a {
return c
} else {
return d
}
Result:
This should prevent misinterpretation and thus potential bugs in the future.
Motivation:
This changes two things:
- it removes the negation of non-used associated types (the removal of `(_)`).
- it prefers exhausting `switch`es over `default`ing.
This change does not impact bahavior; it is pure refactoring.
Modifications:
Omitting needless `(_)`'s and explicitly exhausting switches.
Result:
This change is pure refactoring.
Motivation:
We not always ensure correct ordering which made it quite hard to reason about things. We should always first notify the promise before we call the fire* method that belongs to the event. Beside this we sometimes fired events or notified promised before correctly update the active state / addresses of a Channel which could result in unexpected results when query the Channel during execution of callbacks and handlers.
Modifications:
- Ensure we always notify promise first
- Always correctly update channel state before notify promise
- Add test to verify notification order.
Result:
Correct ordering of events which makes things easier to reason about and which follows what netty is doing.
Motivation:
Testing specific ordering behaviours (e.g. cancelling tasks) requires
that EmbeddedEventLoop behave as similarly as possible to the full
event loop.
Modifications:
Adopt the batched scheduled task dispatch logic from the regular
event loop.
Result:
Easier to test subtle behaviours around task dispatch.
Motivation:
320561f437 added a few tests which missed to correctly shutdown the EventLoopGroup
Modifications:
Add code to shutdown group
Result:
Correctly release resources during tests.
Motivation:
When the SelectableEventLoop is shutdown it will process a defer block in which we tried to fail all scheduled tasks that were not processed yet. The problem here was that we incorrectly not dropped the processed task and so ended in an endless loop which prevent the defer block to complete.
Beside this how we processed the scheduled tasks that were ready for execution was quite in-efficient as we always called removeFirst() which will most likely case a memory copy to move the elements around.
Modifications:
- Correctly fail all scheduled tasks without end up in an endless loop
- Use a for each loop to reduce overhead of processing ready tasks.
Result:
It's always possible to shutdown a SelectableEventLoop and less overhead when processing ready tasks.
Motivation:
A somewhat common requirement when working with chains of futures
is needing to hop from one event loop to another. This is particularly
common when relying on the fact that EventLoopFuture will synchronize
with the event loop that created it: you often want to rely on that
implicit locking, rather than use Dispatch or Lock to achieve the
same effect.
While doing this hop requires relatively little code, it's not
necessarily totally apparent to new users how they would do it.
Additionally, the most naive implementation incurs the overhead of
allocations and reference counting in cases where it's not necessary
(e.g. when you have only one event loop, or when both work items are
being scheduled on the same event loop).
For this reason, we should have a nice concise way for a user to
request this behaviour and get a relatively performant implementation
of the behaviour.
Modifications:
Added EventLoopFuture<T>.on(eventLoop:).
Changed AcceptHandler to use the new method rather than its (slower)
alternative.
Result:
Users will have an easier time working with EventLoopFutures.
Motivation:
EventLoopFuture.and had serious threading issues if the EventLoops
weren't the same.
Modifications:
Fixed the threading issues and tested them properly.
Result:
Hopefully `and` and `andAll` now don't crash if you use them across
EventLoops.
Motivation:
We did not verify that fireErrorCaught(...) was called for recoverable errors while accept new sockets.
Modifications:
Verify errorCaught is called.
Result:
Better test.
Motivation:
We should not close the DatagramChannel when recvfrom fails with ECONNREFUSED / ENOMEM. The former may be caused by a previous sendto and the other may be recoverable by just stop reading for some time.
Modifications:
- Special handle ECONNREFUSED and ENOMEN to not close the channel.
- Add unit tests
Result:
More robust datagram support.
Motivation:
For servers NIO had a race where when you start tearing down the
`ServerSocketChannel` and at roughly the same time a client connects, we
would not fully bring that channel up, nor tear it down. That lead to
open channels being deallocated which leads to a crash as that's an
illegal state.
Modifications:
Made sure newly accepted channels get closed if the
`ServerSocketChannel` is already clsing.
Result:
No crashes anymore when a client connects to a closing server.
Motivation:
We returned port numbers directly from the sockaddr structures but
they're in network byte order. Our code only read them correctly when
presenting, oops.
Modifications:
Corrected the port number reading and tested that.
Result:
Port numbers should now be correct.
Motivation:
Often accept errors are recoverable over time as for example you may just run out of file descriptors. Because of this it may be useful to just "delay" accepts a bit to recover.
Modifications:
Add ChannelHandler which will backoff accepts.
Result:
Be able to recover from accept errors gracefully.
Motivation:
At the moment its possible that channelRead is triggered even if autoRead is never changed to true and no read() is triggered.
Modifications:
- Always register with .none on the Selector and then just call readIfNeeded0()
- Only register for EPOLLRDHUP if .read, .write or .all is used.
- Add test case.
Result:
Fixes https://github.com/apple/swift-nio/issues/160.
Motivation:
The current HTTP server helpers are not modular, meaning that each
time we add a new first-party HTTP handler we need to add a brand new
method. Additionally, they do not currently compose, so if you want
assistance with HTTP pipelining *and* to support HTTP upgrade you are
out of luck.
Modifications:
Deprecate the two existing server helpers.
Add a new server helper that can be extended to support all of the
possible features that users may want in their HTTP pipelines.
Result:
Users will have a single place to go that can be used to configure
their HTTP server pipeline, and that understands how the different
handlers fit together in a complete pipeline.
Motivation:
In certain cases it would be useful to be able to access the current EventLoop (which is tied to a thread).
Modifications:
- Add MultiThreadEventLoopGroup.currentEventLoop
- Add tests
Result:
Fixes [#59]
Motivation:
The HTTP pipeline may contain other HTTP protocol specific handlers,
beyond just the ones that the upgrade handler knows explicitly. It should
be possible to get the upgrade handler to remove those at the right time.
Modifications:
Add extra support for removing handlers.
Result:
It'll be possible for the HTTP upgrade logic to remove things like the
pipelining handler at upgrade time.
Motivation:
It is often useful to find a handler of a given type even when you
aren't holding a reference to that handler. This patch allows us to
do that.
Modifications:
Add context(handlerType:) function to the channel pipeline.
Result:
Users can find handlers of a given type in a pipeline.
Motivation:
In a few instances, we have code that ignores a `()` argument in a
closure with this syntax:
{ (_: Void) in
whilst that's a bit odd, it shouldn't be wrong and Swift 4.0.x is also
totally happy with it. The latest Swift 4.1 candidates however are not
and also there's a better way of spelling this:
{ () in
and in some cases even just
{
Modifications:
Removed instances of `{ (_: Void) in`
Result:
Compiles with `swift-4.1-DEVELOPMENT-SNAPSHOT-2018-03-12-a`
Motivation:
Websockets is a major protocol in use on the web today, and is
particularly valuable in applications that use asynchronous I/O
as it allows servers to keep connections open for long periods of
time for fully-duplex communication.
Users of NIO should be able to build websocket clients and servers
without too much difficulty.
Modifications:
Provided a WebsocketFrameEncoder and Decoder that can serialize and
deserialize Websocket frames.
Result:
Easier use of websockets.
Motivation:
HTTP pipelining can be tricky to handle properly on the server side.
In particular, it's very easy to write out of order or inconsistently
mutate state. Users often need help to handle this appropriately.
Modifications:
Added a HTTPServerPipelineHandler that only lets one request through
at a time.
Result:
Better servers that are more able to handle HTTP pipelining
Motivation:
We should not close the server socket when accept fails with ECONNABORTED / EMFILE / ENFILE, ENOBUFS. ENOMEM as generally speaking these may be "recoverable" once some existing connections were closed for example. It's possible to implement a backoff strategy for these with a ChannelHandler.
Modifications:
- Special handle of ECONNABORTED / EMFILE / ENFILE, ENOBUFS. ENOMEM for ServerSocketChannel.
Result:
Not close servet socket for errors from which it is possible to recover.
Cory Benfield