* Adopt `Sendable` for types in `BSDSocket` namespace
* Adopt `Sendable` for `CircularBuffer` and `MarkedCircularBuffer`
and also `Endianness`
* move `NIOSendable` into its own file and copy it over to syscall integration tests
Motivation
The rise of Swift concurrency has meant that a number of our APIs need
to be recontextualised as async/await capable. While generally this is a
straightforward task, any time those APIs were tested using
EmbeddedChannel we have a testing issue. Swift Concurrency requires the
use of its own cooperative thread pool, which is completely incapable of
safely interoperating with EmbeddedChannel and EmbeddedEventLoop. This
is becuase those two types "embed" into the current thread and are not
thread-safe, but our concurrency-focused APIs want to enable users to
use them from any Task.
To that end we need to develop new types that serve the needs of
EmbeddedChannel and EmbeddedEventLoop (control over I/O and task
scheduling) while remaining fully thread-safe. This is the first of a
series of patches that adds this functionality, starting with the
AsyncEmbeddedEventLoop.
Modifications
- Define AsyncEmbeddedEventLoop
Result
A required building block for AsyncEmbeddedChannel exists.
Co-authored-by: Franz Busch <privat@franz-busch.de>
* Move tests of NIOCore types from NIOPosixTests to NIOCoreTests
* fixup: LinuxTest
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Port some test utils to not use NIOPosix
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: No swift-system, duh
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: read-write for temporary file in test helper
Signed-off-by: Si Beaumont <beaumont@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
Stop Sendable related warnings when using types defined inside NIOConcurrencyHelpers. All the code should actually already be thread safe, so no changes needed other than labelling some classes as either Sendable or @unchecked Sendable.
* Use unbuffered IO for stdout in NIOPerformanceTester
Signed-off-by: Si Beaumont <beaumont@apple.com>
* CRASH PERF TESTS: Up task count to 1M to test logging change
* Revert "CRASH PERF TESTS: Up task count to 1M to test logging change"
This reverts commit d8fa50eb15.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Cocoapods appears to have a different idea of the dependency graph to
SPM which has led to a handful of build failures (e.g.
https://github.com/apple/swift-nio/issues/2073).
This appears to have originated when we dropped the explicit dependency
on `CNIOAtomics` from `NIO` (https://github.com/apple/swift-nio/pull/1719).
We can work around this by listing all transitive dependencies as
requirements in the generated podspecs.
Modifications:
- Add a script to list transitive dependencies for a given module.
- Fix a bug in build_podspecs.sh which did a `pod repo update` even if
the pods were not being uploaded.
- Update build_podspecs.sh to use transitive dependencies.
- Use Python3 in 'dev/stackdiff-dtrace.py' and update soundness.sh
Result:
Podspecs should include all transitive dependencies.
* Drop support for 5.2 and 5.3
As outlined in a [Swift forums post in November ’21](https://forums.swift.org/t/swiftnio-swift-version-support/53232), SwiftNIO will only support the latest non-patch Swift release and the 2 immediately prior non-patch versions.
In this commit we drop support for Swift 5.2 and 5.3. We update CI for Swift 5.4 to run on bionic instead of focal to ensure that we still test bionic.
* Added Versions paragraph to Security document
* Apply suggestions from code review
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
ByteBuffer uses a synthetic UInt24 as the lower bound of the slice of
memory that it can access. This necessitates a slow path when we create
slices of ByteBuffers that would force the lower bound to be larger than
a UInt24.
This slow path has a bunch of math within it, and it turns out that some
of it is invalid. This means in rare cases slicing a ByteBuffer can
cause a crash. That's not great: the code should only crash if we broke
an internal invariant, not because we did some invalid math.
In particular, the relevant check is calculating the upper bound of the
storage to be copied. This was:
```
min(storageRebaseAmount + _toCapacity(self._slice.count), self._slice.upperBound, storageRebaseAmount + capacity)
```
In this case, `storageRebaseAmount` is either `self._slice.lowerBound` or
`self._slice.lowerBound + self._readerIndex`: that is,
`storageRebaseAmount >= self._slice.lowerBound`. Given that
`self._slice.count == self._slice.upperBound - self._slice.lowerBound`,
we know that `storageRebaseAmount + self._slice.count` is always
strictly greater than or equal to `self._slice.upperBound`. Calculating
this is unnecessary. Unfortunately, it can also overflow: if
`self._slice.upperBound + self._readerIndex` is greater than UInt32 then
this will crash.
Note that the other math, `storageRebaseAmount + capacity`, has been
kept. `capacity` is always computed by us and this should only ever
overflow if we are increasing the size of the buffer past UInt32.max,
which we cannot handle anyway.
Modifications:
- Restructured and heavily commented _copyStorageAndRebase.
- Removed the redundant crashing check.
Result:
Less trapping!
* Improve the performance of copying CirularBuffer
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Drop consuming
* fixup: whitespace
* fixup: endIndex instead of count
* fixup: Add missing self and fix test name
Signed-off-by: Si Beaumont <beaumont@apple.com>
* Avoid allocations by using @inline(__always) function
Signed-off-by: Si Beaumont <beaumont@apple.com>
* Manually inline the copy for each range
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Linux test names after test rename
Signed-off-by: Si Beaumont <beaumont@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
We regressed allocs in nightly, probably for a Swift bug. Fixing it is
tracked in #2070, for now we're going to raise the limits so CI isn't
moaning all the time.
# Motivation
`HTTPResponseStatus` is currently not conforming to `Hashable`.
# Modification
Add `Hashable` conformance to `HTTPResponseStatus`
# Result
`HTTPResponseStatus` is `Hashable`
### Motivation:
SPM has built in functionality to check the API of modules against a target git treeish. We can use this to simplify our `check_no_api_breakages.sh` script. Closes https://github.com/apple/swift-nio/issues/1239
### Modifications:
This PR, exchanges the direct calls to Swift's API checker with the new SPM `diagnose-api-breaking-changes` tool. This allows us to get rid of the manual module parsing, build invocations and result comparisons.
### Result:
We are now using SPMs `diagnose-api-breaking-changes` to check for breaking changes.
Motivation:
We currently expect two runs of a repeated task in a test to happen with
relatively little breathing room: the minimum delay is 10ms, and we
won't wait longer than 50ms. We should widen this a bit.
Modifications:
- Double the deadline from 50ms to 100ms.
Result:
This test passes consistently.
Resolves#2053.
The nightly builders have started automatically adding a struct with
@main to packages generated using swift package init. This is
interfering with our syscallwrapper tests, so we should delete that
file.
Motivation:
We've held off on doing this for a while on the theory that function
calls are cheap, and we're happy to make them where there is no
particular need to specialize. Unfortunately, while this is true in
general, Swift is capable of optimizing ARC and exclusivity checking
when it can better understand what a method actually does.
Additionally, we'd hoped that cross-module-optimization would be a
useful addition here. Sadly, right now this still hasn't rolled out as a
default, and even if it had, it tends to be limited to smaller methods
and generic functions, all of which we've already annotated.
Modifications:
- Add @inlinable to essentially everything.
Result:
Better codegen around ByteBuffer.
Motivation:
When turning a BBV into a Collection, several of the collection
algorithms can do better if you implement the semi-private
`_copyContents` hook. We can easily implement it, so we should. While
I'm here we should clean up some other bits.
Modifications:
- Implement _copyContents.
- Implement a faster `.count` for `ByteBufferView`.
- Make `ByteBuffer.capacity` `@inlinable` to reduce ARC traffic
Result:
Copying a BBV into an Array is faster.
David Nadoba