Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(share/eds): remove bitswap cache wrap from eds store #3101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

walldiss
Copy link
Member

Two layers of caching were implemented for bitswap requests in full nodes using EDS store:

A bitswap cache that caches individual shares.
An internal EDS store cache that caches accessors.
In the event of a cache miss, both the internal cache of the EDS store and the bitswap cache will be populated with the same share, introducing redundancy in caching. Moreover, having a per-share cache doesn't seem to make much sense for full nodes, given the astronomically small chances of requesting the same share from a block that is not cached in the EDS cache.

The pull request removes the per-share cache as it brings close to no value while consuming RAM. Your understanding and consideration of this change are greatly appreciated.

@walldiss walldiss added kind:feat Attached to feature PRs kind:fix Attached to bug-fixing PRs labels Jan 12, 2024
@walldiss walldiss self-assigned this Jan 12, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 44.83%. Comparing base (2469e7a) to head (1778be5).
Report is 15 commits behind head on main.

Files Patch % Lines
nodebuilder/p2p/bitswap.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3101      +/-   ##
==========================================
- Coverage   44.83%   44.83%   -0.01%     
==========================================
  Files         265      263       -2     
  Lines       14620    14628       +8     
==========================================
+ Hits         6555     6558       +3     
- Misses       7313     7321       +8     
+ Partials      752      749       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Wondertan
Copy link
Member

Wondertan commented Jan 12, 2024

Iirc, it doesnt cache the full blocks there but their local availability for Has method via bloomfilters. It is an optimization for bitswap to avoid doing Hazes(reads). This might still be helpful, but needs more thoughts.

@walldiss
Copy link
Member Author

walldiss commented Jan 18, 2024

The blockstore cache implemented in our full node does not update when a block is acquired from shrex. If a block request fails because it's absent in the eds-store, the cache erroneously flags it as 'not available'. Consequently, when the block is later synchronized, the cache fails to reflect this update. This discrepancy leads to the risk of generating false 'not available' statuses:

  1. A block not stored receives an initial request (1), resulting in a 'not available' cache entry.
  2. The block is subsequently retrieved and stored.
  3. A second request (2) for the same block consults the cache, mistakenly producing a 'not available' response.

Considering this, disabling the blockstore cache might be advisable to prevent such inaccuracies.

@walldiss walldiss marked this pull request as draft January 18, 2024 11:43
@walldiss
Copy link
Member Author

This is not a problem for Light nodes, as blockstore cache will be updated on successful sampling.

@walldiss walldiss marked this pull request as ready for review March 27, 2024 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:feat Attached to feature PRs kind:fix Attached to bug-fixing PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants