feat: compute piece CID in client #925
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alanshaw
commented
Sep 14, 2023
| assert(service.upload.add.called) | ||
| assert.equal(service.upload.add.callCount, 1) | ||
| assert.equal(pieceCIDs.length, 1) | ||
| assert.equal(pieceCIDs[0].toString(), 'bafkzcibbammseumg3mjlev5odi5bpcsrp4gg62d7xnx44zkxzvgedq7nxldbc') |
There was a problem hiding this comment.
Does this look right to you guys? I was expecting baga... but then I think that is v1.
There was a problem hiding this comment.
Per spec it should come out as bafkz https://github.com/filecoin-project/FIPs/blob/master/FRCs/frc-0069.md
|
I think we should wasm version as it's noticeably faster, but I'm still struggling to find a way to make it bundler friendly |
vasco-santos
approved these changes
Sep 14, 2023
There was a problem hiding this comment.
LGTM
We can swap to https://github.com/web3-storage/fr32-sha2-256-trunc254-padded-binary-tree-multihash/ + hookup this to invocation as follow up PRs
alanshaw
pushed a commit
that referenced
this pull request
Sep 15, 2023
🤖 I have created a release *beep* *boop* --- ## [9.3.0](upload-client-v9.2.0...upload-client-v9.3.0) (2023-09-15) ### Features * compute piece CID in client ([#925](#925)) ([2297f01](2297f01)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Alan Shaw <alan.shaw@protocol.ai>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR alters the client to compute piece CIDs for each shard, and exposes it via the
CARMetadataprovided to theonShardStoredcallback.I removed the
ShardStoringStream. In hindsight, the only real utility it was providing was parallelizing the calls toStore.add(...). I swapped in a parallel transform stream that allows us to do the same thing, but also concurrently calculate the piece CID of the shard.Note: the client does not yet do anything with the piece CID, but in theory it will be trivial to collect them and then send a bunch of invocations to trigger filecoin.