Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: adopt piece cid v2 #21

Merged
merged 3 commits into from
Aug 9, 2023
Merged

feat!: adopt piece cid v2 #21

merged 3 commits into from
Aug 9, 2023

Conversation

Gozala
Copy link
Contributor

@Gozala Gozala commented Aug 8, 2023

Aligns implementation around Piece CID V2

Comment on lines 45 to 55
// assert.deepEqual(piece.paddedSize, data.out.paddedSize)

const json = piece.toJSON()
// assert.deepEqual(piece.toInfo(), {
// link: parseLink(data.out.cid),
// height: Math.log2(data.out.size / Node.Size),
// })

assert.deepEqual(json, {
link: {
'/': data.out.cid,
},
height: Math.log2(data.out.size / Node.Size),
})

const view = Piece.fromJSON(json)
assert.deepEqual(view.link, piece.link)
assert.deepEqual(view.size, piece.size)
assert.deepEqual(view.height, piece.height)
// const view = Piece.fromJSON(json)
// assert.deepEqual(view.link, piece.link)
// assert.deepEqual(view.size, piece.size)
// assert.deepEqual(view.height, piece.height)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like something in progress stayed here?

Copy link
Contributor

@vasco-santos vasco-santos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good to me. However, there are some comments in tests and tests skipped that I do not know reason for. Seems like commented ones could be passing relying on PieceInfo?

Comment on lines 281 to 287
// assert.deepEqual(
// JSON.stringify(build),
// JSON.stringify({
// link: build.link,
// height: Math.log2((1 << 20) / Node.Size),
// })
// )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like something in progress stayed here?

/Pieces are too large to fit in the index/
)
},

'pass bad value to estimate': async (assert) => {
'skip! pass bad value to estimate': async (assert) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the reason to skip this one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It became impossible to pass bad values (see line 172) because we switch to height and therefor size is always pow2. I probably will remove this test, I just had it disabled as I was trying to figure out what to do with it.


get link() {
if (this._link == null) {
this._link = Link.create(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gozala this creates a LegacyLink for the AggregateBuild.toInfo() which should not be expect right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand the question here. PieceInfo corresponds to same named go struct which uses older version of CID, which is why we create a legacy link here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is related to getting an Aggregate CID being Piece CIDv2. I think this is important for us for metrics at least, be able to know by Aggregate CID their encoded height/size

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, it looks like only CID we can get from here is through AggregateBuild.toInfo(). Maybe we just want a getter for Piece CIDv2 from aggregate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test/piece.spec.js Outdated Show resolved Hide resolved
test/piece.spec.js Outdated Show resolved Hide resolved
Comment on lines 281 to 287
// assert.deepEqual(
// JSON.stringify(build),
// JSON.stringify({
// link: build.link,
// height: Math.log2((1 << 20) / Node.Size),
// })
// )
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// assert.deepEqual(
// JSON.stringify(build),
// JSON.stringify({
// link: build.link,
// height: Math.log2((1 << 20) / Node.Size),
// })
// )

test/aggregate.spec.js Outdated Show resolved Hide resolved
@Gozala Gozala merged commit ae68ed1 into main Aug 9, 2023
3 checks passed
vasco-santos added a commit to storacha/w3up that referenced this pull request Aug 10, 2023
Upgrades `data-segment` lib with Piece CID v2
storacha/data-segment#21

Needs:
- [x] figure out legacy CID for
storacha/data-segment#21 (review)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants