Skip to content

Commit

Permalink
Linked list + updated benchmarks (#2)
Browse files Browse the repository at this point in the history
* Linked list + updated benchmarks

* Fix build + additional test case

* Adding CI

* rename job

* mac test target

* Correct action version

* pin mac version

* Benchmark multiple types

* Update benches

* results updates

* formatted raw results

* Fix buggy tests

* Fix tests on linux
  • Loading branch information
gh123man committed Apr 7, 2024
1 parent e483395 commit 2ec45f6
Show file tree
Hide file tree
Showing 14 changed files with 315 additions and 152 deletions.
25 changes: 25 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Unit Tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
build:
name: Swift ${{ matrix.swift }} on ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-13]
swift: ["5.9"]
runs-on: ${{ matrix.os }}
steps:
- uses: swift-actions/setup-swift@v2
with:
swift-version: ${{ matrix.swift }}
- uses: actions/checkout@v4
- name: Build
run: swift build
- name: Run tests
run: swift test
1 change: 1 addition & 0 deletions Benchmarks/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ xcuserdata/
DerivedData/
.swiftpm/config/registries.json
.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata
.swiftpm/xcode/xcshareddata
.netrc

Examples
54 changes: 46 additions & 8 deletions Benchmarks/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,59 @@ All tests performed on an M1 max

## Swift vs Go

(Integer input only)

| Test Case | Go (seconds) | Swift (seconds) | Swift `n` times slower than go |
| --------------------------- | ----------- | ----------- | ----- |
| testSingleReaderManyWriter | `0.318661688` | `1.093256199` | `3.43x` |
| testHighConcurrency | `0.328830854` | `1.163244104` | `3.54x` |
| testHighConcurrencyBuffered | `0.362022931` | `1.482958496` | `4.10x` |
| testSyncRw | `0.132789557` | `2.987084699` | `22.49x` |
| testSelect | `0.306248166` | `1.612218893` | `5.26x` |
| testSingleReaderManyWriter | `0.318661688` | `0.702384305` | `2.15x` |
| testHighConcurrency | `0.328830854` | `0.7504368067` | `2.19x` |
| testHighConcurrencyBuffered | `0.362022931` | `1.144838405` | `3.05x` |
| testSyncRw | `0.132789557` | `1.547898102` | `11.36x` |
| testSelect | `0.306248166` | `1.401787996` | `5.05x` |

### Raw results

The below results are from running the whole benchmark suit which covers multiple data types.

| Test case | Data type | Result (seconds) |
|---------------------------------------|-------------|------------------|
| testSingleReaderManyWriter | `Int` | `0.684` |
| testHighConcurrency | `Int` | `0.719` |
| testHighConcurrencyBuffered | `Int` | `1.104` |
| testSyncRw | `Int` | `1.508` |
| testSelect | `Int` | `1.545` |
| testSingleReaderManyWriter | `String` | `0.811` |
| testHighConcurrency | `String` | `0.838` |
| testHighConcurrencyBuffered | `String` | `1.137` |
| testSyncRw | `String` | `1.706` |
| testSelect | `String` | `1.562` |
| testSingleReaderManyWriter | `ValueData` | `0.750` |
| testHighConcurrency | `ValueData` | `0.772` |
| testHighConcurrencyBuffered | `ValueData` | `1.159` |
| testSyncRw | `ValueData` | `1.518` |
| testSelect | `ValueData` | `1.560` |
| testSingleReaderManyWriter | `RefData` | `0.871` |
| testHighConcurrency | `RefData` | `0.926` |
| testHighConcurrencyBuffered | `RefData` | `1.187` |
| testSyncRw | `RefData` | `1.681` |
| testSelect | `RefData` | `1.543` |
| testAsyncAlgSingleReaderManyWriter | `Int` | `5.436` |
| testAsyncAlgSingleHighConcurrency | `Int` | `7.373` |
| testAsyncAlgSingleReaderManyWriter | `String` | `5.395` |
| testAsyncAlgSingleHighConcurrency | `String` | `7.340` |
| testAsyncAlgSingleReaderManyWriter | `ValueData` | `5.393` |
| testAsyncAlgSingleHighConcurrency | `ValueData` | `7.334` |
| testAsyncAlgSingleReaderManyWriter | `RefData` | `5.392` |
| testAsyncAlgSingleHighConcurrency | `RefData` | `7.328` |

## Async Channels vs Async Algorithms AsyncChannel

Apple has their own channel implementation in the [swift-async-algorithms package](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Channel.md). We cannot compare every benchmark since it does not support buffering or select.

| Test Case | This Library | Async Algorithms `AsyncChannel` | This library `n` times faster |
| --------------------------- | ----------- | ----------- | ----- |
| testSingleReaderManyWriter | `1.093256199` | `6.111190903` | `5.59x` |
| testHighConcurrency | `1.163244104` | `8.512602198` | `7.32x` |
| testSingleReaderManyWriter | `0.7498844027519226` | `5.4360926985740665` | `7.95x` |
| testHighConcurrency | `0.7715810060501098` | `7.373045003414154` | `10.25x` |

### Why is swift slower than go?

Expand All @@ -37,4 +74,5 @@ Aside from the above limitations, special care has been taken to use efficient l

## Future work

This has not yet been benchmarked on linux. Linux uses `pthread_mutex_t` so expect results to differ somewhat.
This has not yet been benchmarked on linux. Linux uses `pthread_mutex_t` so expect results to differ somewhat.

165 changes: 105 additions & 60 deletions Benchmarks/Sources/Benchmarks/main.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,122 +6,166 @@ import AsyncAlgorithms

let iterations = 10

await run()
await runAsyncAlg()

func run() async {
await testSingleReaderManyWriter()
await testHighConcurrency()
await testHighConcurrencyBuffered()
await testSyncRw()
await testSelect()

protocol Initializable {
init()
}

extension Int : Initializable {}
extension String : Initializable {
init() {
self.init("My test string for benchmarking")
}
}

struct ValueData: Initializable {
let foo: String
let bar: Int
init() {
foo = "My test string for benchmarking"
bar = 1234
}
}

class RefData: Initializable {
let foo: String
let bar: Int
required init() {
foo = "My test string for benchmarking"
bar = 1234
}
}

func runAsyncAlg() async {
await testAsyncAlgSingleReaderManyWriter()
await testAsyncAlgSingleHighConcurrency()
// MARK: Run Tests

print("Starting benchmarks with \(iterations) rounds")

await run(Int.self)
await run(String.self)
await run(ValueData.self)
await run(RefData.self)

await runAsyncAlg(Int.self)
await runAsyncAlg(String.self)
await runAsyncAlg(ValueData.self)
await runAsyncAlg(RefData.self)


func run<T: Initializable>(_ type: T.Type) async {
print(await testSingleReaderManyWriter(type))
print(await testHighConcurrency(type))
print(await testHighConcurrencyBuffered(type))
print(await testSyncRw(type))
print(await testSelect(type))
}

func timeIt(iterations: Int, block: () async -> ()) async {
func runAsyncAlg<T: Initializable>(_ type: T.Type) async {
print(await testAsyncAlgSingleReaderManyWriter(type))
print(await testAsyncAlgSingleHighConcurrency(type))
}

func timeIt(iterations: Int, block: () async -> ()) async -> Double {
var sum: CFAbsoluteTime = 0
for _ in 0..<iterations {
let startTime = CFAbsoluteTimeGetCurrent()
await block()
let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
sum += timeElapsed
}
print("Time elapsed: \(sum / Double(iterations))")
return sum / Double(iterations)

}

func testSingleReaderManyWriter() async {
print(#function)
await timeIt(iterations: iterations) {
let a = Channel<Int>()
func testSingleReaderManyWriter<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = Channel<T>()
var sum = 0

for _ in (0..<100) {
Task {
for _ in (0..<10000) {
await a <- 1
await a <- T()
}
}
}

while sum < 1_000_000 {
sum += (await <-a)!
await <-a
sum += 1
}
}
return ("\(#function) \(T.self)", result)
}

func testHighConcurrency() async {
print(#function)
await timeIt(iterations: iterations) {
let a = Channel<Int>()
func testHighConcurrency<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = Channel<T>()
var sum = 0

for _ in (0..<1000) {
Task {
for _ in (0..<1000) {
await a <- 1
await a <- T()
}
}
}

while sum < 1_000_000 {
sum += (await <-a)!
await <-a
sum += 1
}
}
return ("\(#function) \(T.self)", result)
}

func testHighConcurrencyBuffered() async {
print(#function)
await timeIt(iterations: iterations) {
let a = Channel<Int>(capacity: 20)
func testHighConcurrencyBuffered<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = Channel<T>(capacity: 20)
var sum = 0

for _ in (0..<1000) {
Task {
for _ in (0..<1000) {
await a <- 1
await a <- T()
}
}
}

while sum < 1_000_000 {
sum += (await <-a)!
await <-a
sum += 1
}
}
return ("\(#function) \(T.self)", result)
}

func testSyncRw() async {
print(#function)
await timeIt(iterations: iterations) {
let a = Channel<Int>(capacity: 1)
func testSyncRw<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = Channel<T>(capacity: 1)

for i in (0..<5_000_000) {
await a <- i
for _ in (0..<5_000_000) {
await a <- T()
await <-a
}
}
return ("\(#function) \(T.self)", result)
}



func testSelect() async {
print(#function)
await timeIt(iterations: iterations) {
let a = Channel<Int>()
let b = Channel<Int>()
let c = Channel<Int>()
let d = Channel<Int>()
let e = Channel<Int>()
let f = Channel<Int>()
func testSelect<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = Channel<T>()
let b = Channel<T>()
let c = Channel<T>()
let d = Channel<T>()
let e = Channel<T>()
let f = Channel<T>()

for chan in [a, b, c, d, e, f] {
Task {
for _ in (0..<100_000) {
await chan <- 1
await chan <- T()
}
}
}
Expand All @@ -131,34 +175,34 @@ func testSelect() async {
while sum < 6 * 100_000 {
await select {
rx(a) {
sum += $0!
sum += 1
}
rx(b) {
sum += $0!
sum += 1
}
rx(c) {
sum += $0!
sum += 1
}
rx(d) {
sum += $0!
sum += 1
}
rx(e) {
sum += $0!
sum += 1
}
rx(f) {
sum += $0!
sum += 1
}
}
}
}
return ("\(#function) \(T.self)", result)
}


// Compare to async algorithms channels

func testAsyncAlgSingleReaderManyWriter() async {
print(#function)
await timeIt(iterations: iterations) {
func testAsyncAlgSingleReaderManyWriter<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = AsyncChannel<Int>()
var sum = 0

Expand All @@ -177,11 +221,11 @@ func testAsyncAlgSingleReaderManyWriter() async {
}
}
}
return ("\(#function) \(T.self)", result)
}

func testAsyncAlgSingleHighConcurrency() async {
print(#function)
await timeIt(iterations: iterations) {
func testAsyncAlgSingleHighConcurrency<T: Initializable>(_ type: T.Type) async -> (String, Double) {
let result = await timeIt(iterations: iterations) {
let a = AsyncChannel<Int>()
var sum = 0

Expand All @@ -200,4 +244,5 @@ func testAsyncAlgSingleHighConcurrency() async {
}
}
}
return ("\(#function) \(T.self)", result)
}
Loading

0 comments on commit 2ec45f6

Please sign in to comment.