Remove EmbeddedChannel restriction to IOData
Motivation:
You should be able to read any type from the outbound buffer.
Modifications:
Change Embedded channel outboundBuffer and pendingOutboundBuffer to store NIOAny instead of IOData and refactor test code accordingly.
Result:
- Users can test what went through the pipeline without serializing everything and using "if case let" syntax everywhere.
- fixes#551
* Add cancel notification to RepeatedTask
Motivation:
When cancelling a RepeatedTask, it is often desirable to
wait until any still in-progress execution of the task is
finished - for example, to know when a periodic state save
is done before shutting down. The current API provides no
easy way to perform such a wait. With this addition,
repeating tasks can report completion of a cancellation
request regardless of in what state the task was in when the
request was issued.
Modifications:
A default-nil parameter is added to EventLoop.scheduleRepeatedTask(),
to which a promise can be provided; the promise will be
fulfilled at the time of the task's cancellation, plus any
time required for the task to complete running if it was
cancelled during execution. The promise is signaled even if
the cancellation is the result of the task throwing an error.
A default-nil paramter is also added to RepeatedTask.cancel(),
to which a second promise can be passed. This promise is
fulfilled under the same conditions as the first, even if the
first was not given; neither of the promises is dependent
upon the other and both are fulfilled at the same time if
provided. Further calls to cancel() with different promises
will result in fulfillment of each of those promises as well.
Result:
The observed behavior of RepeatedTask.cancel() now optionally
includes deterministic knowledge of when the task is no longer
executing or scheduled to be executed. The API changes are
designed to be additive in form, but the function signatures
do change. Source compatibility is maintained; binary
compatibility is not.
Motivation:
initialRingCapacity is not a great name, initialCapacity is much better
Modifications:
rename CircularBuffer(initialRingCapacity:) to initialCapacity
Result:
more consistent code
Motivation:
Remove the only warning in the project.
Modifications:
Use new withUnsafeBytes API.
Result:
The project will have no warnings.
(cherry picked from commit b41e50403b)
Add deadlines as an alternative to timeouts
Motivation:
Address #603
Modifications:
- Add NIODeadline type
- Add func scheduleTask<T>(deadline: NIODeadline, _ task: @escaping () throws -> T) -> Scheduled<T>
- Reduce the use of DispatchTime in favor of Time
- Replace timeout calls with deadline calls where it makes sense
Result:
Tasks can be scheduled after an amount of time has passed, or at a certain time.
Motivation:
ELFs and the stdlib's Result should be as compatible as possible so a
flatMapResult function which lifts and combines a result into an
EventLoopFuture sounds like a good idea.
Modifications:
add `flatMapResult` which does what it says.
Result:
more interop between `Result` and `EventLoopFuture`.
Motivation:
The name `mapIfError` implied that you can transform the error value but
really that's never what it did. Instead, it allowed you to recover
from a failure.
Modifications:
- rename `mapIfError` to `recover`
Result:
cleaner method names
Motivation:
EventLoopPromise.succeed/fail has extraneous labels that don't add
anything but noise.
Modifications:
remove extraneous labels
Result:
less noise in the code
Motivation:
ELF's API should be as close as possible to the new Result's API.
Therefore, we should rename `then` to `flatMap`
Modifications:
- renamed `then` to `flatMap`
- renamed `thenIfError` to `flatMapError`
- renamed ELF's generic parameter from `T` to `Value`
Result:
- more like Result
- fixes#688
Motivation:
When we write unit tests, we generally like them to actually test the
thing they claim to. This test never ran, and the assertions in place
to verify that it ran also never ran, so we didn't notice the test
not running.
Modifications:
- Moved the assertion that validates the test ran to a place where
it will definitely execute.
- Ensured the EmbeddedChannel gets marked as active, so that we
actually deliver the bytes from the HTTPDecoder.
Result:
This test will run, and actually pass.
Motivation:
BaseSocketChannel used to assert that `channelRead0` is only called when
the channel is active. That's bogus and it's trivial to hit this issue
when calling `fireChannelRead` in the last `ChannelHandler` when the
channel has gone inactive.
Modifications:
remove assertion, add comment
Result:
fewer crashes
Motivation:
We now depend on Swift 5.0, so we can remove all the backporting of
functionality.
Modifications:
deleted all backported Swift functionality
Result:
less code
Motivation:
No code is the best code, let's have the compiler generate more
Equatable instances for us.
Modifications:
remove some hand-written Equatable conformances
Result:
less code, possibly fewer bugs
Motivation:
Now that the stdlib has introduced the Result type, we can use it in the
implementation (and the whenComplete) function of EventLoopFuture
Modifications:
- replace EventLoopValue with Result
- make whenComplete provide the Result
Result:
use the new shiny stuff
Motivation:
DispatchData's APIs are less than optimal and the current one we used
did allocate at least on Linux to copy bytes out.
Modifications:
introduce ByteBuffer.write/read/set/getDispatchData
Result:
when Dispatch gets fixed, lower allocations
BUGFIX: BlockingIOThreadPool not releasing the latest WorkItem
Fix is to clear the latest work item just before waiting for
the next one to be submitted.
Added two tests to validate the fix: submitting only one work item
to an IO thread pool, and submitting two consecutive work items.
Thanks to Gwynne Raskind for help troubleshooting & fix
(cherry picked from commit 8df414b716)
Motivation:
The rest of the code in NIO uses multiline literals.
Modifications:
Converted string concatenations to multiline literals.
Result:
NFC, prettier code.
Motivation:
ContiguousCollection is no longer necessary because Sequence gained
withContiguousStorageIfAvailable.
Modifications:
Removed ContiguousCollection and switch to withContiguousStorageIfAvailable.
Result:
less code
Motivation:
CircularBuffer.replaceSubrange was crazy eager to double the capacity
which is bad.
Modifications:
Made CircularBuffer.replaceSubrange only double if necessary.
Result:
less memory usage
Motivation:
The new shiny Swift 5 gives us fast access to the UTF8 bytes (if
available, which mostly means not bridged from Cocoa), we should use
that.
Modifications:
use UTF8View.withContiguousStorageIfAvailable
Result:
faster String access
Motivation:
ByteBuffer has pretty good string writing APIs, but they're imperfect.
In particular, the methods in ByteBuffer-aux both return optional Int,
which is strange as they cannot fail. We also only have support for writing
strings in UTF-8.
Modifications:
- ByteBuffer.set(string:at:) and ByteBuffer.write(string:) now both
return `Int`, and their return value has been made discardable.
- NIOFoundationCompat has been expanded with two new methods for writing
strings to ByteBuffers in various encodings using Foundation's
encoding support: ByteBuffer.set(string:encoding:at:) and
ByteBuffer.write(string:encoding:).
- NIOFoundationCompat has also been expanded with a companion to the
already existing ByteBuffer.getString(at:length:encoding), namely
ByteBuffer.readString(length:encoding:).
- New error enum for reporting failures to encode.
Result:
Easier to work with strings in a wider variety of cases with
ByteBuffer
Resolves#253.
Motivation:
The latest Swift 5.0s bring us new goodies (hence tools version) and
also more warnings, hence the fixes for them.
Modifications:
- set tools-version to 5.0
- fix warnings
- workaround https://bugs.swift.org/browse/SR-9514
Result:
happier developers
* Change unnecessary functions to variables
Motivation:
The changed functions do not have any side effects and therefore better convey intent when being a simple computed variable.
Modifications:
Changes:
- `func markedElementIndex() -> Int?` to `var markedElementIndex: Int?`
- `func markedElement() -> E?` to `var markedElement: E?`
- `func hasMark() -> Bool` to `var hasMark: Bool`
Result:
- improved code quality
Motivation:
If we're going to have errors in enumerations, we should aim to have as
few as possible. We wanted to throw away these two, so let's do it.
Modifications:
- Removed `ChannelLifecycleError`, moved case to `ChannelError`.
- Removed `MulticastError`, moved cases to `ChannelError`.
- Removed manual `Equatable` conformance, used synthesised one.
- Pulled `connectFailed` out of `ChannelError`, as it made it impossible to
write a correct implementation of `Equatable`.
- Conformed `NIOConnectionError` to `Error` instead.
Result:
Better error handling.
Resolves#620
Motivation:
In Swift, abbreviations use the same case for all letters, therefore it
should be `SNI` and not `Sni`.
Modifications:
changes `Sni` to `SNI`
Result:
more consistent with naming guidelines
Motivation:
- `Int` is the currency type for integral things and we already sort of
changed the the type for ports but not everywhere.
- generic parameters that aren't just 'any type' shouldn't be just `T`
or `S` or something
Modifications:
- make more use of `Int`
- rename generic parameters to something more descriptive like `Bytes`
Result:
easier to read/write NIO code
Motivation:
Swift naming guidelines mandate that factory methods start with `make`,
like `makeSomething`. We had a few that were `newSomething`.
Modifications:
make all factories start with make
Result:
more compliant to Swift naming rules
Motivation:
Explain here the context, and why you're making that change.
What is the problem you're trying to solve.
Modifications:
Describe the modifications you've done.
Result:
After your change, what will change.
Motivation:
NIO2 development starts now.
Modifications:
Made NIO Swift 5-only for everything else see docs/public-api-changes-NIO1-to-NIO2.md
Result:
NIO2 development can start.
Motivation:
making a promise of a type is more grammatical than making a promise for
a type.
Modifications:
s/newPromise(for/newPromise(of/g
Result:
more grammar
Motivation:
When creating new promises I always find it very frustrating to type the
`: EventLoopPromise<Type>` type annotation but it's necessary for the
compiler to know type the promise will be fulfilled with.
Modifications:
allow an optional `for: SomeType.self` parameter for `newPromise` as
let p = eventLoop.newPromise(for: Int.self)
is much easier to type than
let p: EventLoopPromise<Int> = eventLoop.newPromise()
Result:
easier to write code
Motivation:
I recently removed a bogus assertion but forgot to remove some now
useless variables.
Modifications:
Remove those useless variables.
Result:
Only deprecation warnings left.
Motivation:
One ByteBuffer test was (for whatever reason) asserting that realloc
never succeeds in realloc'ing from 16 to 32 bytes. That just failed on
macOS, presumably an allocator change.
Modifications:
remove bogus assertion about reallocs ability to grow allocations.
Result:
tests more stable
Motivation:
Some network configurations might disallow connecting to unknown IP
addresses. The effect of that is that we might see `ECONNREFUSED` when
trying to connect to 198.51.100.254 which nominally is reserved for
documentation only (ie. we shouldn't get any RSTs).
Modifications:
ignore `ECONNREFUSED` in the timeout tests too
Result:
tests stable in less permissive network environments
Motivation:
ChannelPipeline.remove0 contains a comment that indicates that users
were expected to be able to use the ChannelHandlerContext when both
handlerRemoved and the promise for channel removal call out.
This works when invoking remove() from outside the event loop, but if
a handler invokes remove on itself then it will only be able to attach
a callback to the future *after* the callout occurs. This means that a
ChannelHandler removing itself from the pipeline cannot rely on there
being a moment when it can still invoke the ChannelHandlerContext, but
when it is no longer a formal part of the pipeline.
This kind of behaviour is useful in some unusual cases, and it would
be nice to support it.
Modifications:
- Added remove() methods that accept promises as input.
- Rewrote the current remove() methods in terms of the new ones.
- Added tests.
Result:
ChannelHandlers can perform cleanup with a valid ChannelHandlerContext
but outside the Channel.
Motivation:
Swift 5 complains on redundant modifiers. For example declaring a
`public func` inside of a `public extension` is now a warning. I don't
agree with this but I dislike warnings even more, so...
Modifications:
remove all redundant modifiers that the compiler now warns about such as
swift-nio/Sources/NIOPriorityQueue/PriorityQueue.swift:90:5: warning: 'public' modifier is redundant for property declared in a public extension
Result:
no warnings in Swift 5
* CONNECT method has no request body
### Motivation:
[rfc7231](https://tools.ietf.org/html/rfc7231#section-4.3.6) said:
> A payload within a `CONNECT` request message has no defined semantics;
sending a payload body on a `CONNECT` request might cause some existing
implementations to reject the request.
### Modifications:
Change `HTTPMethod.CONNECT.hasRequestBody` to returns `.no`.
### Result:
`HTTPRequestEncoder` does not generate chunked body for `CONNECT` method.
* Change `HTTPMethod.CONNECT.hasRequestBody` to returns `.unlikely`
Motivation:
BlockingIOThreadPool used to call out with a lock held and on top of
that on the wrong thread...
Modifications:
Make BlockingIOThreadPool call out on the supplied `queue`
Result:
fewer deadlock and surprises
Motivation:
There was an outstanding change in http_parser.c that we did not yet
have in our copy.
Modifications:
run the update http parser script and add changes as well as a test case
designed to hit the change.
Result:
ship the latest and greatest
Motivation:
On some machines we've seen tests that rapidly join and leave multicast groups
hitting spurious test failures. These test failures manifest as hitting ENOMEM on
setsockopt(IP_ADD_MEMBERSHIP). The reason appears to be that the IGMP join/leave
messages that the kernel needs to emit for these joins can be queued up waiting for
a timer to fire, and if we move sufficiently quickly we can push too many
messages into that queue.
This can be avoided by avoiding the need to emit the IGMP join/leave messages
at all. We can do that by forcing the multicast joins onto the loopback interface,
as there's no value in the kernel emitting join/leaves on the loopback. Given that
for these tests the join is necessary only to get the kernel to have IGMP state,
there's no reason not to do this.
Modifications:
Forced all multicast joins onto the loopback address.
Result:
No weird ENOMEM crashes in test runs.
### Motivation:
make swift-nio compatible with Android, and pass all test in Android.
### Modifications:
* add missed `ifaddrs` implementation
* add missed thread related functions - `CNIOLinux_pthread_getname_np`, `CNIOLinux_pthread_setaffinity_np` and `CNIOLinux_pthread_getaffinity_np`
* fix types incositency between Linux and Android, e.g. `Epoll.swift` class
* make bytes counter explicitely 64bits to avoid overflow on 32bit, in `PendingWritesManager.swift` and `PendingDatagramWritesState.swift`
* fix issue with Unix domain socket, first byte need to be zero in Android
* several incosistency fixes between Linux and Android api.
### Result:
now swift-nio works on Android. All tests passed!
Motivation:
In Swift 5, types that conform to ExpressibleByIntegerLiteral will now
have their init(integerLiteral:) initializer invoked whenever the
compiler sees code of the form `Type(integerLiteral)`, rather than
before where the integer literal would be created as `Int` and then
an alternative initializer would be called. This broke our tests, so
we should fix it now that we're aware of it.
Modifications:
Force the type of the argument for the one broken test line, add new
test lines that execute this code path.
Result:
Better testing, fewer compile errors.
Motivation:
A large number of very useful protocols are implemented using multicast
with UDP. As a result, it would be helpful to add support for joining and
leaving IP multicast groups using SwiftNIO.
Modifications:
- Defines a MulticastChannel protocol for channels that support joining and
leaving multicast groups.
- Adds an implementation of MulticastChannel to DatagramChannel.
- Adds a interfaceIndex property to NIONetworkInterface.
- Adds if_nametoindex to the Posix enum.
- Adds a isMulticast computed property to SocketAddress
- Adds a demo multicast chat application.
- Add a number of multicast-related socket options to SocketOptionProvider.
Result:
NIO users will be able to write channels that handle multicast UDP.
Motivation:
A small number of socket options have values that do not fit into a C
int type. Our current ChannelOption based approach for setting these
simply does not work, and cannot be extended to support the truly arbitrary
types that the setsockopt/getsockopt functions allow here.
This makes it impossible to use some socket options, which is hardly a
good place to be.
There were a number of ways we could have addressed this: we could have
special-cased all socket options with non-integer types in ChannelOption,
but I believe that would be too manual, and risk limiting users that need
to set other socket options. We could also have added a ChannelOption
that simply allows users to pass a buffer to write into or read from,
but that is a very un-Swift-like API that feels pretty gross to hold.
Ultimately, the nicest seemed to be a new protocol users could check
for, and that would provide APIs that let users hold the correct concrete
type. As with setsockopt/getsockopt, while this API is typed it is
not type-safe: ultimately, the struct we have here is treated just as a
buffer by setsockopt/getsockopt. We do not attempt to prevent users from
shooting themselves in the foot here.
This PR does not include an example use case in any server, as I will
provide such an example in a subsequent multicast PR.
Modifications:
- Added a SocketOptionChannel protocol.
- Conformed BaseSocketChannel to SocketOptionChannel.
- Wrote some tests for this.
Result:
Users can set and get sockopts with data that is larger than a C int.
Tanner