Motivation:
SwiftNIO periodically drops support for older Swift versions. Now that
5.7 has been released, 5.4 will be dropped.
Modifications:
- Remove 5.4 specific Package.swift and docker-compose
- Update the 5.7 docker-compose to use the released 5.7 and move from
focal (2004) to jammy (2204)
- Remove unused swiftformat from Dockerfile
- Update tools version in syscall wrapper tests to 5.5
- Update docs
Results:
Minimum Swift version is 5.5
Motivation:
The node.js HTTP parser library that we use has been unmaintained for some time. We should move to the maintained replacement, which is llhttp. This patch will update our dependency and bring us over to the new library, as well as make any changes we need.
Modifications:
This patch comes in 4 parts, each contained in a separate commit in the PR.
The first commit drops the existing http_parser code and updates some of the repo state for using llhttp.
The second commit rewrites the update script to bring in llhttp instead of http_parser.
The third runs the actual script. You can skip reviewing this except to sanity check the outcome.
The fourth commit updates the NIO code and the tests to get everything working.
In general the substance of the product modifications was minimal. The logic around keeping track of where we are in the buffer and how upgrades work has changed a bit, so that required some fiddling. I also had to add an error reporting path for the delegates to be able to throw specific errors that llhttp no longer checks for. Finally, I removed two tests that were a little overzealous and that llhttp does not police.
Result:
Back on the supported path.
* initial adoption of DocC based documentaiton
motivation: use DocC for docs
change:
* add DocC catalog to NIO target, as the "landing page"
* remove jazzy doc generation script
* add Package.swift with tools-version 5.6 so that DocC can be used, and add a dependency on the DocC plugin
* udpate soundness script
* * add packge.swift files for older vrsions
* adjust waning-as-error and enable-test-discovery to the different permutations
* fixup
* remove jazzy
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Cocoapods appears to have a different idea of the dependency graph to
SPM which has led to a handful of build failures (e.g.
https://github.com/apple/swift-nio/issues/2073).
This appears to have originated when we dropped the explicit dependency
on `CNIOAtomics` from `NIO` (https://github.com/apple/swift-nio/pull/1719).
We can work around this by listing all transitive dependencies as
requirements in the generated podspecs.
Modifications:
- Add a script to list transitive dependencies for a given module.
- Fix a bug in build_podspecs.sh which did a `pod repo update` even if
the pods were not being uploaded.
- Update build_podspecs.sh to use transitive dependencies.
- Use Python3 in 'dev/stackdiff-dtrace.py' and update soundness.sh
Result:
Podspecs should include all transitive dependencies.
* Drop support for 5.2 and 5.3
As outlined in a [Swift forums post in November ’21](https://forums.swift.org/t/swiftnio-swift-version-support/53232), SwiftNIO will only support the latest non-patch Swift release and the 2 immediately prior non-patch versions.
In this commit we drop support for Swift 5.2 and 5.3. We update CI for Swift 5.4 to run on bionic instead of focal to ensure that we still test bionic.
* Added Versions paragraph to Security document
* Apply suggestions from code review
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
SPM has built in functionality to check the API of modules against a target git treeish. We can use this to simplify our `check_no_api_breakages.sh` script. Closes https://github.com/apple/swift-nio/issues/1239
### Modifications:
This PR, exchanges the direct calls to Swift's API checker with the new SPM `diagnose-api-breaking-changes` tool. This allows us to get rid of the manual module parsing, build invocations and result comparisons.
### Result:
We are now using SPMs `diagnose-api-breaking-changes` to check for breaking changes.
motivation: fix broken doc generation script
changes:
* pass the --spm flag now required by source-kitten
* update location of source-kitten source to make it more robust
* update list of module to generate docs for to match the structural changes in nio
Motivation:
Frequently when folks have issues with SwiftNIO or related technologies,
we need to ask the same questions over and over again.
It'd be much nicer to just have a script that collects them.
Modifications:
Add `scripts/nio-diagnose`.
Result:
Users/devs can easily create nio-diagnose files which are markdown and
can easily be pasted into GH.
Motivation:
In #1888 a C compiler setting was added to Package.swift for CNIODarwin.
This isn't reflected in the associated CNIODarwin podspec so the pod
fails to build.
Modifications:
Modify the build_podspecs.sh script to add the missing compiler setting
for CNIODarwin.
Result:
CNIODarwin pod builds.
* Fix doc generation and jazzy version
From: https://github.com/apple/swift-nio-extras/pull/141
motivation: fix ci
changes:
* only install ruby and jazzy on focal since jazzy id not supported onlder versions of ubuntu
* fix doc generations script adjusting it to latest source-kittent syntax
* Always install ruby: we need it for generating a test manifest
Motivation:
The soundness check is failing because of the generated year range is
not as it should be. Moreover when re-generating the linux tests for the
length prefix tests (which are failing soundness) the generated header
has extra whitespace before the year; this also fails the soundness
check.
Modifications:
- Strip whitespace from the copyright year range generated for linux
tests
- Re-generate the test manifest for ByteBufferLengthPrefixTests
Result:
- Soundness is happier
Motivation:
The test generation script automatically fills in the dates for the
autogenerated test files. Unfortunately, it doesn't tolerate a brand-new
file very well and puts in year zero. That's not right.
Modifications:
Default to the current year if no years are present.
Result:
Better default dates.
* issue-1036: Fix for dates on Linux test generation script.
Motivation:
Fix for issue-1036 to add a date or date range for the header.
These dates are derived from the git log on the file.
Modifications:
Parsing method to extract the date or date range from the git log
of the file. This was added to generate_linux_tests.
Result:
Dynamic date or date range added to the file header.
* issue-1036: Added revised headers from generation script
* Regenerating dates on XCTests.
Motivation:
Addressing feedback from PR.
Modifications:
All of the XCTests now have -2021 as a header date attached to them.
Result:
Dates modified on the XCTests.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
The lastest NIO versions require Swift 5.2+ to compile. Given that
hopefully nobody ever created a NIO1 application for Swift 5.2+, I'd say
it's about time to remove the NIO1 API shims (and related docs).
Modification:
- remove the NIO1 API shims
- remove the migration docs
Result:
Less code.
Make Swift 5.2 the minimum requirement, dropping support for Swift 5.0 and 5.1.
Motivation:
Whenever we have problems, Swift 5.0 and 5.1 seem to be the culprits. Dropping support for these very old versions will require less maintenance and free up our time to work on new features.
Modifications:
Set the tools version in Package.swift to 5.2
Remove CI configurations for 5.0 and 5.1
Update the various readmes to reflect that this change will be rolled out in NIO 2.30.0
Result:
Swift 5.2 is the minimum version of Swift required to use NIO.
Motivation:
We need license headers but the soundness script couldn't check if the
scripts don't have a `.sh` extension.
Modification:
- Add missing license headers.
- Better license checking (search for shell shebangs too)
Result:
Proper license attribution.
Motivation:
Be able to run and test swift-nio on Android.
Modifications:
- Remove the custom ifaddrs and use the one from the Android NDK instead.
- Enable a bunch of conditionally-compiled code for Android.
- Add a handful of constants and other Android declarations.
- Cast some types because of mismatches specific to Android.
Result:
Most tests pass on Android AArch64 and ARMv7.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
We support watchOS 6+ with SwiftNIO Transport Services; as such we should
include watchOS as a deployment target for our CocoaPods.
Modifications:
- Add a watchOS deployment target to `build_podspecs.sh`
- Update docs
Result:
Users can deploy to watchOS 6+ with CocoaPods.
Motivation:
The information printed when invoking `build_podspecs.sh` without
arguments is outdated.
Modifications:
- Print usage information when invoking without arguments.
Result:
- 'build_podspecs.sh' is more helpful when you can't remember how it
should be invoked
Motivation:
It's 2020, we shouldn't use unnecessary offensive, insulting, or
outdated language.
Modifications:
Fix a few examples.
Result:
More inclusive.
Motivation:
list_topsorted_dependencies had multiple issues:
- it would list some executable targets (even in libs-only mode)
- it confused packages with modules
- it would also list dependencies from outside the current package
Modifications:
Address all of the above.
Result:
Script more useful, inside and outside of swift-nio.
Co-authored-by: David Evans <david.evans@blackpoint.co>
motivation: docs scripts is broken on selinux due to restriction on ssh-agent
changes: change the jazzy workspace to a directory accessible by the CI node so we can push from the agent instead of from the docker container
Motivation:
Somewhere in github or Cocoapods, there seems to be a problem that if
you push a pod A and quickly after a pod B that depends on A with
exactly the version you just pushed, the validation sometimes fails
claiming that the right version of A is not available. The workaround is
to wait a bit and to call `pod repo update`.
The modifications in this PR alleviate this problem somewhat, it now
calls `pod repo update` by itself and should it still go wrong, there's
a new `-f` (from) option that can be used to skip over all pods until
the one that failed last. Example
scripts/build_podspecs.sh -u -f SwiftNIO 2.13.1
Modifications:
- update the pod repo often
- provide a skipping option to skip over earlier repos in case something
still goes wrong
Result:
Easier way to update the pods.
Motivation:
NIO has allocation counter tests, NIO also has dev/malloc-aggregation.d
which can give you aggregated stacks that allocated alongside how many
times they allocated. What's missing is a script that can diff two
aggregated stack traces.
Modifications:
Add dev/stacktrace-dtrace which allows aggregated stack trace diffing.
Result:
Easier to track performance regressions.
motivation: tsan is broken on ubuntu 18.04.use 16.04 so 5.2 build can pass
changes:
* change 5.2 docker setup to use 16.04 instead of 18.04
* add the ability to pipe arguments to the integration tests via the docker setup
* skip tests_02 integration tests suite since it broken on 5.2 (swift-pm issue?)
fake the new limits from #1362
Co-authored-by: tomer doron <tomer@apple.com>
Motivation:
It's important to also test deprecated functionliaty. One way of
achieving this without warnings is to also deprecate the tests that test
this deprecated functionality. Unfortunately, on Linux we need to
generate lists of tests which would then reference deprecated tests
(which gives us a warning).
Modifications:
Deprecate test suites and the main test runner all the way to the top so
never get warnings.
Result:
Possible to test deprecated functionlity without warnings.
Motivation:
Jazzy generates Dash docsets for us, but currently they don't work. This
is because the URLs they generate aren't matching the URL scheme we
have. That makes them next to useless.
Modifications:
Add a trailing '/' to the root URL.
Result:
Calls to Ruby's URI.join won't blow away the final path component in the
root URL, which means that the docsets will have correct URLs going
forward.
Motivation:
We should print the same metric that we compare with.
Modifications:
Switch the printed metric to the one selected in `::METRIC`.
Result:
More correct printouts.
Motivation:
To be able to use SHA sums as git versions, we need to pull all the refs
to we get even the commit SHAs from PRs. That's important so CI can use
this script.
Modifications:
Fetch `+refs/pull/*:refs/remotes/origin/pr/*` refs.
Result:
Should be able to be usable in CI across forks.
Motivation:
swift-api-digester requires to be passed `-sdk /` on Linux.
Modifications:
Pass `-sdk /` on Linux and the SDK directory on Darwin.
Result:
check_no_api_breakages works on Linux.
One line description of your change
Motivation:
When using the script ad hoc to verify something you may have forgotten to pull all tags,
the script then fails mysteriously in silence and no output. Detective work can then lead the script mysteriously stopping after a checkout:
```
+ cd /tmp/.check-api_1Q1rvO/repo
+ git checkout 1.1.0
$ ...
```
which reminds you to perform a fetch; it would be nicer if it told us about the issue explicitly though.
Modifications:
Don't silence `git checkout` all error output, though use quiet mode so no progress is printed.
Result:
Script fails nicer and reminds you to perform a git fetch if you forgot to do so.
e.g.
```
[5/5] Merging module Metrics
error: pathspec '1.1.0' did not match any file(s) known to git
```
* Check jq is available before checking API breakages
Motivation:
If jq wasn't installed the API breakage script would erronesouly say
that there were no breaking API changes. That's pretty bad.
Modifications:
Check jq is available before running the check. Updated the usage
message to specify that a Swift 5.1+ toolchain is required.
Result:
More correctness.
Motivation:
Mean is not a great aggregation function because it's not resilient
against outliers at all, so let's switch to min.
Modifications:
Switch 'mean' to 'min' for aggregation.
Result:
Aggregation function that's more resilient against outliers.
Motivation:
It's incredibly hard not to accidentally break the public API, so let's
automate that :).
Modifications:
Add the first version of a public API check script.
Result:
Fewer public API breakages.
Cory Benfield