Motivation:
We left a doc comment on EventLoopFuture.map that caused confusion by
misleadingly claiming that it could handle throwing functions. The
operation that handles throwing functions is flatMapThrowing.
Modifications:
- Removed the misleading comment and pointed users to flatMapThrowing
instead.
Result:
Less user confusion.
Motivation:
We still had a bunch of docker files for NIO on Swift versions that NIO2
doesn't actually work on. Also we had allocation counter limits in
places where they don't mean anything.
Modifications:
Prune all the useless information.
Result:
Less configuration.
Motivation:
In #957 we accidentally broke the API just a little bit. We turned the
`ByteToMessageHandler.init` from
public init(_ decoder: Decoder)
into
public init(_ decoder: Decoder, maximumBufferSize: Int? = nil)
This looks totally benign and in almost all cases because of the default
argument, you wouldn't notice the API breakage. However, it's possible
to write code that works in NIO 2.0.0 but breaks in NIO 2.1.0. For
example
let g: (Dummy) -> ByteToMessageHandler<Dummy> = ByteToMessageHandler<Dummy>.init
let h: (Dummy) -> ByteToMessageHandler<Dummy> = ByteToMessageHandler<Dummy>.init(_:)
Modifications:
According to SemVer, this releases a patch that unbreaks the public API.
Result:
No API breakages (checked with `swift-api-digester` which also found the
breakage in the first place).
Motivation:
In many unit tests, it's important to count the events. So far we have
many many ad-hoc handlers to do so but having one that counts all events
would be handy.
Modifications:
Add `EventCounterHandler` to count all events travelling the
`ChannelPipeline`.
Result:
Easier unit testing
* Allow adding a sequence of headers to HTTPHeaders
Motivation:
When combining two collections of headers, adding each pair one-by-one
using add(name:value:) can cause multiple unnecessary reallocations of
the underlying array.
Modifications:
Provide two additional add methods to HTTPHeaders: one to add a sequence
of name/value pairs and another to add the headers from another
HTTPHeaders, both of which use `append(contentsOf:)` on the underlying
array to avoid additional reallocations.
Result:
Fewer reallocations when adding multiple headers.
* Fix incorrect documentation, make add<S: Sequence>(contentsOf:) @inlineable
* Reserve capacity and add sequentially instead of using append(contentsOf:)
* Add tests and missing @usableFromInline
Add an `always` method on EventLoopFuture
Motivation:
https://github.com/apple/swift-nio/issues/665
Modifications:
* Add `always` in EventLoopFuture.swift
* Add tests
Result:
NIO Users will be able to use `.always` to, well, always run an action
wether the future succeed or not.
Motivation:
When writing B2MDs, there are a couple of scenarios that always need to
be tested: firehose feeding, drip feeding, many messages, ...
It's tedious writing those tests over and over again for every B2MD.
Modifications:
- Add a simple B2MD verifier that users can use in unit tests.
- Add a new, public `NIOTestUtils` module which contains utilities
mostly useful for testing. Crucially however, it does not depend on
`XCTest` so it can be used it all targets.
Result:
Hopefully fewer bugs in B2MDs.
Motivation:
The allocation counter tests should default to using the integration
tests' tmp dir.
Modifications:
Propagate the right tmp dir.
Result:
fewer orphaned tmp dirs lying around in case of error.
Motivation:
The new allocation counter suite hardcoded a few things:
- the NIO and NIOHTTP1 modules
- no extra dependencies apart from the repo you're in
Modifications:
remove these hardcoded things.
Result:
allocation counter script works for nio-http2
Motivation:
New public NIO types must be nested in existing ones or `NIO` prefixed.
This wasn't the case for ByteToMessageDecoderPayloadTooLargeError
Modifications:
nest PayloadTooLargeError into ByteToMessageDecoderError
Result:
we stick to our rules.
Motivation:
With HTTP/1.0 and EOF framing it's possible to have a HTTP response that
doesn't have any headers. HTTPDecoder unfortunately fell flat on its
face and crashed the program if that happened.
Modifications:
- don't expect headers on responses
- add tests for that
- rename the state machine cases to reflect a request and response
parsing
Result:
fewer crashes.
Motivation:
Due to https://bugs.swift.org/browse/SR-10614, assigning an array containing a tuple type
to a variable that expects an array containing a different-but-compatible tuple type will cause
an allocation and copy of that array storage.
In some cases this is necessary, but we were doing it in the HTTPHeaders constructors, which meant
that the swift-nio-http2 code was hitting a hilariously over the top number of allocations.
Modifications:
- Change the internal storage type of HTTPHeaders to match the public constructor.
Result:
Fewer allocations when constructing HTTPHeaders
Motivation:
Swift 5.1 shows much fewer allocations, let's tighten the limits.
Modifications:
Tighten the allocation limits on 5.1.
Result:
We'll catch allocation regressions much more easily.
Motivation:
While CircularBuffer has an inlinable implementation to improve performance, MarkedCircularBuffer
never got enhanced to have it. This harms performance in many core NIO libraries which heavily use
MCB, forcing heap allocations on almost all MCB operations.
Modifications:
- Make basically everything on MarkedCircularBuffer inlinable.
Result:
Faster code.
Motivation:
We should test all available Swift versions, including snapshots.
Modifications:
Add docker compose file for Swift 5.1.
Result:
More testing.
Motivation:
The new file RW opening facilities didn't have any tests and it didn't
let you specify arbitrary POSIX flags.
Modifications:
- Add tests.
- Let users specify both POSIX flags & mode.
Result:
Better code quality.
Motivation:
To make the web socket upgrader naming clearer, particularly once we add a client version.
Modifications:
Rename WebSocketUpgrader too WebSocketServerUpgrader.
Result:
Improved clarity of naming on the web socket upgrader.
Motivation:
Structures that users can't create don't give them much value.
Modifications:
- Make the initializer public.
- Remove that godforsaken @testable import.
Result:
ChannelOptions.Storage sparks joy.
Motivation:
https://github.com/apple/swift-nio/issues/966
Modifications:
- Add a `handler(type:)` method on ChannelPipeline
- Add a test in ChannelPipelineTest
Result:
Provides a nicer way to get a typed handler directly from the pipeline
Closes#966
Motivation:
https://github.com/apple/swift-nio/issues/701
Modifications:
* Add `init(rawValue:)`
* Add `rawValue`
* Add tests in HTTPTypesTest.swift
Result:
Users no longer have to write their own conversion methods.
Motivation:
Because of #600, many users will start leaking Channels because they
don't close on error.
Modifications:
* Add ChannelHandlerError
* Add ChannelHandlerErrorTest
Result:
User will have access to an handler to use to automatically close
the channel in case of unhandled errors
Motivation:
The HTTPDecoder is a complex object that has very careful state management goals. One source of this
complexity is that it is fed a stream of bytes with arbitrary chunk sizes, but needs to produce a
collection of objects that are contiguous in memory. For example, each header field name and value
must be turned into a String, which requires a contiguous sequence of bytes to do.
As a result, it is quite common to have a situation where the HTTPDecoder has only *part* of an
object that must be emitted atomically. In this situation, the HTTPDecoder would like to instruct
its ByteToMessageHandler to keep hold of the bytes that form the beginning of that object. To avoid
asking http_parser to parse those bytes twice, the HTTPDecoder uses a value called httpParserOffset
to keep track.
As an example, consider what would happen if the "Connection: keep-alive\r\n" header field was delivered
in two chunks: first "Connection: keep-al", and then "ive\r\n". The header field name can be emitted in
its entirety, but the partial field value must be preserved. To achieve this, the HTTPDecoder will store
an offset internally to keep track of which bytes have been parsed. In this case, the offset will be set
to 7: the number of bytes in "keep-al". It will then tell the rest of the code that only 12 bytes of the
original 19 byte message were consumed, causing the ByteToMessageHandler to preserve those 7 bytes.
However, when the next chunk is received, the ByteToMessageHandler will *replay* those bytes to
HTTPDecoder. To avoid parsing them a second time, HTTPDecoder keeps track of how many bytes it is
expecting to see replayed. This is the value in httpParserOffset.
Due to a logic error in the HTTPDecoder, the httpParserOffset field was never returned to zero.
This field would be modified whenever a partial field was received, but would never be returned
to zero when a complete message was parsed. This would cause the HTTPDecoder to unnecessarily keep
hold of extra bytes in the ByteToMessageHandler even when they were no longer needed. In some cases
the number could get smaller, such as when a new partial field was received, but it could never drop
to zero even when a complete HTTP message was receivedincremented.
Happily, due to the rest of the HTTPDecoder logic this never produced an invalid message: while
ByteToMessageHandler was repeatedly producing extra bytes, it never actually passed them to http_parser
again, or caused any other issue. The only situation in which a problem would occur is if the HTTPDecoder
had a RemoveAfterUpgradeStrategy other than .dropBytes. In that circumstance, decodeLast would not
consume any extra bytes, but those bytes would have remained in the buffer passed to decodeLast, which
would then incorrectly *forward them on*. This is the only circumstance in which this error manifested,
and in most applications it led to surprising and irregular crashes on connection teardown. In all
other applications the only effect was unnecessarily preserving a few tens of extra bytes on
some connections, until receiving EOF caused us to drop all that memory anyway.
Modifications:
- Return httpParserOffset to 0 when a full message has been delivered.
Result:
Fewer weird crashes.
* Add ability to open a file for writing
Motivation:
Users can use NonBlockingFileIO for reading files, but if one wants to
write a file, there is no way to achieve that, apart from opening file
manually. I think having to inits to open files for reading and writing
would be convinient.
Modifications:
Allow to open files for writing.
Result:
Users can now open files for writing and reading in the same way.
Motivation:
The link to swift.org linked to the Swift 5 development versions but now
it's released. We also mentioned swift-nio-{http2,ssl,extras,transport-services}
but we already mention them in a more prominent place.
Modifications:
Remove outdated/duped info.
Result:
Better readme
Motivation:
CircularBuffer.Index was asserting that its size is the same as a word.
Whilst that's true on 64-bit architectures, the assert itself is bogus
and a left-over from development.
Modifications:
Remove bogus assert.
Result:
Better 32-bit support.
* optional max payload size for ByteToMessageDecoder
Motivation:
ByteToMessageDecoder aggregate data in memory as part of their normal operation. the ability to limit how much they aggregate is critical in many real-life applications
Modifications:
* add optional maximumBufferSize argument to ByteToMessageDecoder initializer
* test for buffer size when maximumBufferSize is set and throw ByteToMessageDecoderError.payloadTooLarge error
Result:
users can limit how much memory ByteToMessageDecoder takes and handle the exception on their end
Motivation:
The file-io implementation in the example HTTP1 server just crashed on
empty file ;).
Modifications:
- fix the crash
- add integration test
Result:
fewer crashes
Motivation:
Our original Heap implementation used a bunch of static functions
in order to make it clear how it related to textbook heap
implementations. This was primarily done to ensure that it was easier
to see the correctness of the code.
However, we've long been satisfied with the correctness of the code,
and the Heap is one of the best tested parts of NIO. For this reason,
we should be free to refactor it.
Given https://bugs.swift.org/browse/SR-10346, we've been given an
incentive to do that refactor right now. This is because the Heap
is causing repeated CoW operations each time we enqueue new work
onto the event loop. That's a substantial performance cost: well
worth fixing with a small rewrite.
Modifications:
- Removed all static funcs from Heap
- Rewrote some into instance funcs
- Merged the implementation of insert into append.
- Added an allocation test to avoid regressing this change.
Result:
Faster Heap, happier users, sadder textbook authors.
Motivation:
As Johannes previously said in (#946), there's no reason not be able to
get a ByteBuffer from a ByteBufferView.
Modifications:
I added a public initializer to ByteBuffer taking a ByteBufferView to
create a slice of the view's buffer from the same range.
Result:
`ByteBuffer` will have a public initializer to create a ByteBuffer from
a ByteBufferView.
Motivation:
The documentation popup for viewBytes did not correctly
show what the method returns.
Modifications:
I added a colon after the return doc-block keyword.
Result:
The documentation popup for viewBytes will now correctly show
what the method returns.
Motivation:
B2MDs must be robust regarding data after EOF, just in case. Let's add
test cases for that.
Modifications:
add two tests that send a B2MD data after EOF.
Result:
we're sure we're okay with that.
Motivation:
When first viewing the example classes, coming to the ‘read’ method first, leaves the subject unclear as to what the method is ‘reading’.
It is preferable to view something being sent first, and then to view the reading of the response.
It also matches the call order against the protocol making it a little easier for those unfamiliar with the protocol to see which methods have been implemented.
Modifications:
Moved channel active calls to be top of the class.
Despite the diff there are no actual code modifications.
UDP Client changed to indent using spaces to match rest of project. Incidental change.
Result:
The examples are slighter clearer to read, particularly for newcomers to swift-no as the calls are in a logical chronological order.
Motivation:
To provide a simple example of the HTTP1 client pipeline.
To create a comparable example to the TCP and UDP demos to show what extra steps are needed for a HTTP layer.
Modifications:
Adds a NIOHTTP1Client example in a main file.
Adds a readme to the NIOHTTP1Client example.
Result:
There is now a simple example of how to send a HTTP1 client request to the server.
Motivation:
@tomerd learned the hard way that there were some subtle changes in
B2MDs around intercepting arbitrary channel events. Previously,
intercepting those was illegal but possible. In NIO 2, it's impossible.
Modifications:
add a description for what changed
Result:
easier migrations
Motivation:
B2MD (in debug mode) has some checking that we never move out of a final
state in the state machine. Both .done and .error are final states.
Previously however, the code set .done in a defer block which meant that
even if the decodeLoop throws (which should bring it to .error), it goes
via .done first. That's an error because .done is already a final state
so we shouldn't move to .error after that.
Modifications:
Only move to .done if there was no error. If there was an error, we
should go straight to .error, not via .done.
Result:
- more correct B2MD
- fewer crashes (in debug mode)
* Significantly improve the performance of `EventLoopFuture.{and,when}AllSucceed`.
* PR improvements.
* Add the fast track for `AllComplete` as well.
* Use `future.inEventLoop` instead of pointer comparison.
Add missing server readme docs for WebSocket server and UDP echo server.
Motivation:
With the exception of these two samples, all client and server examples have helpful readme docs.
Provides brief description of purpose of the sample code and how to run it.
Modifications:
Adds a readme to NIOUDPEchoServer.
Adds a readme to NIOWebSocketServer.
Result:
The samples for NIOUDPEchoServer and NIOWebSocketServer have a little explanation.
Motivation:
There is a UDP echo client which is very similar to the NIOEchoClient.
The styles of these two examples should match closely so ‘diff’ comparing will show the pertinent API differences.
Modifications:
Add self tags to member variables in the TCP echo client.
Add a self tag to a member variable in the HTTP1 server.
Remove an assert from the echo client.
Result:
The code in the TCP and UDP examples now matches and fits the SwiftNIO style.
* UDP echo client example.
Motivation:
There is a UDP echo server example but it does not currently have a machine client example.
The TCP echo server has both sample client and server examples and it would be preferable to match.
The UDP client connection also differs from both the UDP server and the TCP client, so it is good to provide and example of this difference.
Modifications:
Added a UDP client example and a matching readme file.
The main swift file is a similar to the TCP echo example as possible.
Result:
There is now a clear example of how to create a simple UDP client side connection.
Motivation:
Our podspecs should also correctly require Swift 5.
Modifications:
require Swift 5 in the podspec generator.
Result:
better podspecs for NIO2.
tomer doron