Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding intervaltree to manage gaps in topics to prevent OOM #282

Merged
merged 4 commits into from
Aug 12, 2022

Conversation

patkivikram
Copy link
Collaborator

This should fix #86

@codecov-commenter
Copy link

Codecov Report

Merging #282 (54fa365) into master (84302f5) will decrease coverage by 0.01%.
The diff coverage is 90.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #282      +/-   ##
==========================================
- Coverage   94.71%   94.70%   -0.02%     
==========================================
  Files         100      100              
  Lines       10842    10851       +9     
  Branches     1481     1483       +2     
==========================================
+ Hits        10269    10276       +7     
  Misses        500      500              
- Partials       73       75       +2     
Impacted Files Coverage Δ
faust/transport/consumer.py 94.31% <90.00%> (-0.27%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84302f5...54fa365. Read the comment docs.

@wbarnha
Copy link
Member

wbarnha commented Aug 3, 2022

Any particular reason why this hasn't been merged? It seems fairly useful. I don't know much about the intervaltree library but anything that stops failures for large partitions is welcomed.

@wbarnha
Copy link
Member

wbarnha commented Aug 12, 2022

Tested everything, looks good. I'll merge.

@wbarnha wbarnha merged commit 1e7be3a into master Aug 12, 2022
@wbarnha wbarnha deleted the fix_offset_gap_oom branch July 3, 2023 03:18
vmirz added a commit to TotalWineLabs/faust that referenced this pull request Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Silent Failure / OOM on startup with large topic/partition lag
4 participants