* Small changes for the `NIOAsyncSequenceProducer`
# Motivation
In the PR for the `NIOAsyncWriter`, a couple of comments around naming of `private` properties that needed to be `internal` due to inlinability and other smaller nits came up.
# Modification
This PR includes two things:
1. Fixing up of the small nits like using `_` or getting the imports inside the `#if` checks
2. Changing the public API of the `makeSequence` to be aligned across the throwing and non-throwing one.
# Result
Cleaner code and alinged APIs.
* Fix refactoring left-overs
Motivation
Testing versions of NIO code that involve interfacing with Swift
Concurrency is currently a difficult business. In particular,
EmbeddedChannel is not available in Swift concurrency, making it
difficult to write tests where you fully control the I/O.
To that end, we should provide a variation of EmbeddedChannel that makes
testing these things possible.
Modifications
Provide an implementation of NIOAsyncTestingChannel.
Results
Users can write tests confidently with async/await.
* Add throwing version of `NIOAsyncSequenceProducer`
# Motivation
We recently introduced a `NIOAsyncSequenceProducer` to bridge a stream of elements from the NIO world into the async world. The introduced type was a non-throwing `AsyncSequence`. To support all use-cases we also need to offer a throwing variant of the type.
# Modification
- Introduce a new `NIOThrowingAsyncSequenceProducer` that is identical to the `NIOAsyncSequenceProducer` except that it has a `Failure` generic parameter and that the `next()` method is throwing.
- Extract the `StateMachine` from both `AsyncSequenceProducer`s and unify them.
- There is one modification in behaviour: `didTerminate` is now only called after `nil` or the error has been consumed from the sequence.
# Result
We now have a throwing variant of the `NIOAsyncSequenceProducer`.
* Code review and fix CI
* Remove duplicated code
Co-authored-by: Cory Benfield <lukasa@apple.com>
* initial adoption of DocC based documentaiton
motivation: use DocC for docs
change:
* add DocC catalog to NIO target, as the "landing page"
* remove jazzy doc generation script
* add Package.swift with tools-version 5.6 so that DocC can be used, and add a dependency on the DocC plugin
* udpate soundness script
* * add packge.swift files for older vrsions
* adjust waning-as-error and enable-test-discovery to the different permutations
* fixup
* remove jazzy
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
One of the `NIOAsyncSequenceProducer` extensions was missing a
availability requirements.
Modifications:
Add missing requirement.
Result:
Fewer bugs.
* Implement a back-pressure aware `AsyncSequence` source
# Motivation
We ran into multiple use-cases (https://github.com/apple/swift-nio/pull/2067, https://github.com/grpc/grpc-swift/blob/main/Sources/GRPC/AsyncAwaitSupport/PassthroughMessageSource.swift) already where we want to vend an `AsyncSequence` where elements are produced from the sync world while the consumer is in the async world. Furthermore, we need the `AsyncSequence` to properly support back-pressure.
Since we already identified that this is something fundamental for our ecosystem and that current `AsyncSequence` sources are not providing the proper semantics or performance, it would be great to find a single solution that we can use everywhere.
Before diving into the code, I think it is good to understand the goals of this `AsyncSequence`:
- The `AsyncSequence` should support a single unicast `Subscriber`
- The `AsyncSequence` should allow a pluggable back-pressure strategy
- The `AsyncSequence` should allow to yield a sequence of elements to avoid aquiring the lock for every element.
- We should make sure to do as few thread hops as possible to signal the producer to demand more elements.
# Modification
This PR introduces a new `AsyncSequence` called `NIOBackPressuredAsyncSequence`. The goal of that sequence to enable sync to async element streaming with back-pressure support.
# Result
We can now power our sync to async use-cases with this new `AsyncSequence`.
# Future work
There are couple of things left that I wanna land in a follow up PR:
1. An adaptive back-pressure strategy that grows and shrinks depending on the speed of consumption
2. A throwing version of this sequence
3. Potentially an async version that suspends on `yield()` and resumes when more elements should be demanded.
* Fix cancellation handling
* Review
* Add init helper
* Add return types to help the type checker
* Fix 5.7 CI
* Rename delegate method and update docs
* Review
* Switch to Deque, rename the type and change the behaviour of yielding an empty sequence
* Review comments from George
* Review comments by Konrad
* Review and add tests for the high low strategy
* Fix 5.4 tests
* Code review
Co-authored-by: Cory Benfield <lukasa@apple.com>
* Enhance and rename AsyncEmbeddedEventLoop
Motivation
AsyncEmbeddedEventLoop is an important part of our ongoing testing story
for NIO. However, it suffers from two problems.
The first is a usability one. As we discovered during the original
implementation, following EmbeddedEventLoop's pattern of not having any
EventLoop "Tasks" execute until run() meant that simple constructs like
`EventLoopFuture.wait` and `EventLoopFuture.get` didn't work at all,
forcing us to add an annoying `awaitFuture` method.
When playing with implementing EmbeddedChannel, this got worse, as I/O
methods also don't run immediately if called from the testing thread. We
couldn't easily work around this issue, and it meant that common
patterns (like calling `channel.writeAndFlush`) would deadlock!
This is unacceptable, so a change had to be made.
While we're here, we received feedback that the name is unclear to
users. Given that this particular event loop is in no sense "embedded",
we no longer need the name, so we can take this opportunity to use a
better one.
Modifications
Changed `func execute` to immediately execute its task body, and to
dequeue all pending tasks at this time. Essentially, it's the equivalent
to run(). This is a major change in its behaviour.
Renamed the loop to `NIOAsyncTestingEventLoop`.
Result
Better names, easier to use.
* Make soundness happy
* Remove awaitFuture
Workaround for https://github.com/apple/swift/issues/59911
```swift
/src/Tests/NIOHTTP1Tests/HTTPTest.swift:148:21: warning: 'buf' mutated after capture by sendable closure
buf.writeString("\(c)")
```
The variable `buf` is mutated but only before being capture by the sendable closure.
All other warnings/errors are fixed with the latest Swift 5.7 toolchain
We would previously attempt to convert an error to a description without
consideration for the provenance on Windows. This would result in a
failure if the error code was not from `errno`. Account for the source
of the error and translate it appropriately. We can not retrieve
descriptions on errors that may originate from Windows or WinSock as
well as the C library.
* Adopt `Sendable` for closures in `HTTPPipelineSetup.swift`
* make `UnsafeTransfer` and `UnsafeMutableTransferBox` available in Swift 5.4 too
* fix code duplication
* Copy `UnsafeTransfer` to `NIOHTTP1Tests` to be able to remove `@testable` from `NIOCore` import
* Adopt `Sendable` for closures in `HTTPServerUpgradeHandler.swift`
* Adopt `Sendable` for closures in `HTTPClientUpgradeHandler.swift`
* Adopt `Sendable` for closures in `HTTPPipelineSetup.swift`
* make `UnsafeTransfer` and `UnsafeMutableTransferBox` available in Swift 5.4 too
* fix code duplication
* Copy `UnsafeTransfer` to `NIOHTTP1Tests` to be able to remove `@testable` from `NIOCore` import
* Adopting `Sendable` in `NIOTestUtils`
The parameter `decoderFactory` of the static methods `ByteToMessageDecoderVerifier.verifyDecoder` do not need to be `@escaping`. I have made them non-escaping as part of `Sendable` adoption because we would otherwise need to think about if they should be `@Sendable` too.
`VerificationError` is interesting, see the code comment for more information.
* Adopting `Sendable` in `NIOTestUtils`
The parameter `decoderFactory` of the static methods `ByteToMessageDecoderVerifier.verifyDecoder` do not need to be `@escaping`. I have made them non-escaping as part of `Sendable` adoption because we would otherwise need to think about if they should be `@Sendable` too.
`VerificationError` is interesting, see the code comment for more information.
* Clarify the reason `VerificationError` already conforms to `Sendable`
`EmbeddedChannel` and `EmbeddedEventLoop` should not be `Sendable`. However, they inherit from `Channel` and `EventLoop` respectively. Therfore, they need to be `Sendable` (and are already implcitily `Sendable`. Given these constraints, we can’t mark `EmbeddedChannel` and `EmbeddedEventLoop` as non-Sendable.
This extension does not port cleanly to Windows as the time structures
on Windows are different. This happens to be unused, so simply remove
the extension on Windows.
* Provide conversion APIs between TimeAmount and Swift.Duration
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Remove unnecessary overflow check
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Move into new _NIOBeta57 module
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Rename _NIOBeta57 module to _NIOBeta and move tests to own target
Signed-off-by: Si Beaumont <beaumont@apple.com>
* soundness: copyright years for some reason
Signed-off-by: Si Beaumont <beaumont@apple.com>
* Throw fatalError when scheduling on shutdown EL if SWIFTNIO_STRICT is set
Signed-off-by: Si Beaumont <beaumont@apple.com>
* Add CrashTest for SWIFTNIO_STRICT crash
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Extract env var parsing to static let
Signed-off-by: Si Beaumont <beaumont@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
Add an import of `NIOCore` on Windows which mirrors the other platforms.
This greatly reduces the noise in the error list.
Co-authored-by: Cory Benfield <lukasa@apple.com>
* implement the _failEarlyRangeCheck methods as no-ops
* mark _failEarlyRangeCheck with @inlinable
* add a comment on why the _failEarlyRangeCheck methods are implemented as no-ops
Co-authored-by: Cory Benfield <lukasa@apple.com>
The member names are not identical across platforms. Add a case to
handle the name difference on Windows.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Use Win32 APIs to properly validate if a file is a pipe on Windows.
This enables providing the same semantics without leaking additional
Windows specifics.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Reorganise the ECN constants to colocate the definitions for the
different platforms. Define the constants for Windows as the platform
does not provide them in the system headers.
Franz Busch