Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update benchmark deps and minor fix #346

Merged
merged 4 commits into from
Dec 30, 2023
Merged

Conversation

SgtPooki
Copy link
Member

@SgtPooki SgtPooki commented Dec 14, 2023

  • test: fix benchmark output
  • chore(bench/gc): update deps

NOTE: js-ipfs is disabled for the below runs

╰─ ✔ ❯ INCREMENT=1000 ITERATIONS=5 npm run start

> benchmarks-gc@1.0.0 start
> npm run build && node dist/src/index.js


> benchmarks-gc@1.0.0 build
> aegir build --bundle false

[22:05:14] tsc [started]
[22:05:15] tsc [completed]
helia, 1000, 377.00, 6100.80, 446.80, 398.80
kubo, 1000, 22477.20, 35530.60, 33168.00, 390.40
[8m18.000s]

Note that applying these changes on top of #231 results in the following:

╰─ ✔ ❯ INCREMENT=1000 ITERATIONS=5 npm run start

> benchmarks-gc@1.0.0 start
> npm run build && node dist/src/index.js


> benchmarks-gc@1.0.0 build
> aegir build --bundle false

[22:19:52] tsc [started]
[22:19:53] tsc [completed]
helia, 1000, 36.25, 16250.80, 54.40, 628.60
kubo, 1000, 21848.00, 35402.00, 31879.00, 249.80
[9m-5.418s]

@SgtPooki SgtPooki requested a review from a team as a code owner December 14, 2023 06:14
@SgtPooki
Copy link
Member Author

Full Bench run with npm run start and no env vars

Without iterable pinning:

┌─────────┬────────────────┬────────┬───────────┬──────┐
│ (index) │ Implementation │ ops/s  │   ms/op   │ runs │
├─────────┼────────────────┼────────┼───────────┼──────┤
│    0    │    'helia'     │ '0.20' │ '5098.44' │  5   │
│    1    │     'kubo'     │ '0.16' │ '6066.77' │  5   │
└─────────┴────────────────┴────────┴───────────┴──────┘

With iterable pinning:

┌─────────┬────────────────┬────────┬────────────┬──────┐
│ (index) │ Implementation │ ops/s  │   ms/op    │ runs │
├─────────┼────────────────┼────────┼────────────┼──────┤
│    0    │    'helia'     │ '0.09' │ '11533.61' │  5   │
│    1    │     'kubo'     │ '0.19' │ '5166.40'  │  5   │
└─────────┴────────────────┴────────┴────────────┴──────┘

@SgtPooki
Copy link
Member Author

Note:

Applying the below patch to both blockstore-fs and datastore-fs results in a small speedup, which puts our ops/s higher than Kubo again:

┌─────────┬────────────────┬────────┬───────────┬──────┐
│ (index) │ Implementation │ ops/s  │   ms/op   │ runs │
├─────────┼────────────────┼────────┼───────────┼──────┤
│    0    │    'helia'     │ '0.25' │ '4073.58' │  5   │
└─────────┴────────────────┴────────┴───────────┴──────┘
diff --git a/node_modules/blockstore-fs/dist/src/index.js b/node_modules/blockstore-fs/dist/src/index.js
index 9fdb295..72a9ca2 100644
--- a/node_modules/blockstore-fs/dist/src/index.js
+++ b/node_modules/blockstore-fs/dist/src/index.js
@@ -13,21 +13,23 @@
  */
 import fs from 'node:fs/promises';
 import path from 'node:path';
-import { promisify } from 'node:util';
+// import { promisify } from 'node:util';
 import { Errors } from 'blockstore-core';
 // @ts-expect-error no types
-import fwa from 'fast-write-atomic';
+// import fwa from 'fast-write-atomic';
 import glob from 'it-glob';
 import map from 'it-map';
 import parallelBatch from 'it-parallel-batch';
 import { NextToLast } from './sharding.js';
-const writeAtomic = promisify(fwa);
+import {Writer as StenoWriter} from 'steno';
+// const writeAtomic = promisify(fwa);
 /**
  * Write a file atomically
  */
 async function writeFile(file, contents) {
     try {
-        await writeAtomic(file, contents);
+      const writer = new StenoWriter(file, {flags: 'w'});
+      await writer.write(contents);
     }
     catch (err) {
         if (err.code === 'EPERM' && err.syscall === 'rename') {

see ipfs/js-stores#284 for more information

@achingbrain achingbrain merged commit 17e85f9 into main Dec 30, 2023
18 checks passed
@achingbrain achingbrain changed the title chore(bench/gc): update deps and minor fix chore: update benchmark deps and minor fix Dec 30, 2023
@achingbrain achingbrain deleted the bench/gc-update branch December 30, 2023 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

2 participants