Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilize cache for EntityRecordItemListener bulk insert/update operations on domain objects #1440

Closed
Nana-EC opened this issue Jan 12, 2021 · 1 comment · Fixed by #2006
Closed
Labels
enhancement Type: New feature java Pull requests that update Java code P2 parser Area: File parsing
Milestone

Comments

@Nana-EC
Copy link
Contributor

Nana-EC commented Jan 12, 2021

Problem
Currently we have a mixture of insert only and update logic employed in the EntityRecordItemListener class.
Most domain objects are created and added to a list for batch insert later in EntityListener implementation e.g. SqlEntityListener.
However, items that require an update make the flow complex as they are currently run immediately.

The intent should be to batch insert and batch update all items.

Solution
Utilize the available cache manager by

  • Update EntityRecordItemListener to insert all items into the appropriate cache
  • Update EntityRecordItemListener to update the cache value when an update transaction is encountered for an item that was created in the same record file
  • Update EntityRecordItemListener to insert items created in previous record file into a separate cache
  • Update SqlEntityListener to batch insert (currently with PgCopy) all cache items in insert cache
  • Update SqlEntityListener to batch update (currently with Repository) all cache items in update cache

Alternatives

  • Update SqlEntityListener on??? methods to do check for cache collisions and apply update/insert logic appropriately

Additional Context
The impact currently is that for now we have to keep checking if an item exists in the db prior to deciding how to handle an update transaction

@Nana-EC Nana-EC added enhancement Type: New feature P2 parser Area: File parsing java Pull requests that update Java code labels Jan 12, 2021
@steven-sheehy
Copy link
Member

steven-sheehy commented Jan 12, 2021

I only think this is needed for entities that are updatable: entities, tokens and token accounts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type: New feature java Pull requests that update Java code P2 parser Area: File parsing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants