Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi batch support to upsert pgcopy flow to release 0.36 #2155

Merged
merged 1 commit into from
Jun 21, 2021

Conversation

Nana-EC
Copy link
Contributor

@Nana-EC Nana-EC commented Jun 21, 2021

Detailed description:
When batch inserts occur within the same record file the Upsert flow attempt to recreate the temporary table that already exists

  • Update Create table query to with if not exists
  • Add a truncate table regardless to ensure insert and update don't utilize previously inserted information

Cherry pick of PR 2153

Which issue(s) this PR fixes:
Fixes #2154

Special notes for your reviewer:

Checklist

  • Documentation added
  • Tests updated

Signed-off-by: Nana-EC <nana.essilfie-conduah@hedera.com>
@Nana-EC Nana-EC marked this pull request as draft June 21, 2021 17:11
@Nana-EC Nana-EC requested a review from a team June 21, 2021 17:11
@Nana-EC Nana-EC self-assigned this Jun 21, 2021
@Nana-EC Nana-EC added bug Type: Something isn't working database Area: Database java Pull requests that update Java code P2 labels Jun 21, 2021
@Nana-EC Nana-EC added this to the Mirror 0.36.0 milestone Jun 21, 2021
@codecov
Copy link

codecov bot commented Jun 21, 2021

Codecov Report

Merging #2155 (f7a582b) into release/0.36 (273fe5d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@               Coverage Diff               @@
##             release/0.36    #2155   +/-   ##
===============================================
  Coverage           81.70%   81.70%           
  Complexity           2114     2114           
===============================================
  Files                 410      410           
  Lines               10940    10944    +4     
  Branches              922      922           
===============================================
+ Hits                 8938     8942    +4     
  Misses               1691     1691           
  Partials              311      311           
Impacted Files Coverage Δ
...om/hedera/mirror/importer/parser/UpsertPgCopy.java 93.54% <100.00%> (+0.44%) ⬆️
...epository/upsert/AbstractUpsertQueryGenerator.java 90.59% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 273fe5d...f7a582b. Read the comment docs.

@sonarcloud
Copy link

sonarcloud bot commented Jun 21, 2021

@Nana-EC Nana-EC marked this pull request as ready for review June 21, 2021 17:26
@Nana-EC Nana-EC merged commit b547197 into release/0.36 Jun 21, 2021
@Nana-EC Nana-EC deleted the upsert-batch-0-36 branch June 21, 2021 17:29
Nana-EC added a commit that referenced this pull request Jun 23, 2021
When batch inserts occur within the same record file the Upsert flow attempt to recreate the temporary table that already exists

- Update Create table query to with `if not exists`
- Add a `truncate table` regardless to ensure insert and update don't utilize previously inserted information

Cherry pick of [PR 2153](#2153)

Signed-off-by: Nana-EC <nana.essilfie-conduah@hedera.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Type: Something isn't working database Area: Database java Pull requests that update Java code P2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants