Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle non-existent accounts in preload #15978

Merged
merged 5 commits into from
Aug 27, 2024

Conversation

georgeee
Copy link
Member

@georgeee georgeee commented Aug 26, 2024

Problem: account preloading is not able to preserve information about some accounts not existing in ledger, which invites future repetitive lookups to DB during staged ledger application.

Solution: keep a set of non-existent accounts for masks populated on preload.

Impact: after preload underlying ledger is guaranteed not to be queried.

Explain how you tested your changes:

Checklist:

  • Dependency versions are unchanged
    • Notify Velocity team if dependencies must change in CI
  • Modified the current draft of release notes with details on what is completed or incomplete within this project
  • Document code purpose, how to use it
    • Mention expected invariants, implicit constraints
  • Tests were added for the new behavior
    • Document test purpose, significance of failures
    • Test names should reflect their purpose
  • All tests pass (CI will check this if you didn't)
  • Serialized types are in stable-versioned modules
  • Does this close issues? None

@georgeee georgeee requested a review from a team as a code owner August 26, 2024 12:20
Problem: account preloading is not able to preserve information about
some accounts not existing inledger, which invites future repetetive
lookups to DB during staged ledger application.

Solution: keep a set of non-existent accounts for masks populated on
preload.
@georgeee georgeee force-pushed the georgeee/handle-non-existent-accounts-in-preload branch from d0f34ae to 99ed169 Compare August 26, 2024 12:29
if List.is_empty not_found then
List.map ~f:(fun (a, x) -> (a, Option.value_exn x)) self_found_or_none
else
let from_parent = lookup_parent ancestor not_found in
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code refactor is purely for clarity that call to parent happens only if some records were not found "locally" and underlying ledger needs to be queried.

Ensures no hash db lookups for non-existent accounts even in edge cases.
* use `t.current_location` instead of `last_filled t`
* check against `non_existent_accounts` in `get_or_create_account`

Ensure that underlying ledger won't be unnecessarily queried
using the `last_filled`.
@georgeee
Copy link
Member Author

!ci-build-me

@mrmr1993
Copy link
Member

!ci-build-me

@georgeee
Copy link
Member Author

georgeee commented Aug 26, 2024

A unit test is failing because of reparenting logic. This is a unit test for dead code effectively. I.e. the code is used, but never in a way suggested by the test. Still some code change is needed to either fix the dead code or remove it.

@mrmr1993
Copy link
Member

A unit test is failing because of reparenting logic. This is a unit test for dead code effectively. I.e. the code is used, but never in a way suggested by the test. Still some code change is needed to either fix the dead code or remove it.

The test is junk, feel free to just delete it.

@georgeee
Copy link
Member Author

!ci-build-me

@georgeee
Copy link
Member Author

georgeee commented Aug 27, 2024

The test is junk, feel free to just delete it.

So the actual issue was not in reparenting logic, but in the fact that registering a mask happened before the last edit of the parent. This is something we don't do in production code, but were doing in the test and with the last commit of this PR it started failing the test because instead of calling the last_filled a t.current_location if the mask started to be used.

In near future I hope we rewrite the masking code so that it doesn't allow modification of a mask that has some children

For now I fixed the test to make mask-attaching happen after edits to the base mask performed.

@georgeee georgeee merged commit c5a594c into compatible Aug 27, 2024
44 checks passed
@georgeee georgeee deleted the georgeee/handle-non-existent-accounts-in-preload branch August 27, 2024 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants