Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: do not flush wal when receive consensus msgs #273

Merged
merged 2 commits into from
Jun 10, 2021
Merged

Conversation

wetcod
Copy link
Contributor

@wetcod wetcod commented Jun 10, 2021

Description

There is a performance degradation in the proposal step, because WAL is flushed too often by BlockPartMessage.
But no disk write occurs until signVote, so there is no need to flush until then.
So I fix to not flush wal when receive consensus msgs.

Consensus Steps

AS-IS

**** total average **** 
  proposal: 344.37994
  prevote: 11.697058
  precommit: 25.168762
  commit executing: 1007.19324
  commit committing: 49.402355
  commit rechecking: 339.38562
  waiting for new round: 22.440947
  block interval: 1764.7059
  TPS: 2377.2666

TO-BE

**** total average **** 
  proposal: 105.87284
  prevote: 13.862401
  precommit: 19.146082
  commit executing: 737.93524
  commit committing: 36.59688
  commit rechecking: 200.36572
  waiting for new round: 71.28
  block interval: 1200.0
  TPS: 3239.6

@wetcod wetcod changed the title perf: flush wal asynchronously perf: do not flush wal when receive consensus msgs Jun 10, 2021
@wetcod wetcod self-assigned this Jun 10, 2021
@wetcod wetcod requested a review from iproudhon June 10, 2021 06:39
@wetcod wetcod merged commit a52812a into ebony Jun 10, 2021
@tnasu tnasu deleted the wetcod/async-wal branch July 7, 2021 05:17
egonspace pushed a commit to egonspace/ostracon that referenced this pull request Jul 8, 2021
* perf: do not flush wal when receive consensus msgs
egonspace pushed a commit to egonspace/ostracon that referenced this pull request Jul 8, 2021
* perf: do not flush wal when receive consensus msgs
egonspace pushed a commit that referenced this pull request Aug 26, 2021
* feat: more prometheus metrics for monitoring performance (#146) (#175)

* chore: config timeout and connection pool (#150) (#171)

* fix: use line/tm-db instead of tendermint/tm-db

* bump up tm-db, iavl; re-apply #201

* chore: use default db backend among the available ones (#212)

* chore: use default db backend among the available ones

* chore: bump up iavl, tm-db

* feat: concurrent checkTx #213; fix tm-db call

* fix: rename TM to OC

* fix: modify key name; tendermint -> ostracon

* chore: rename tendermint to ostracon

* chore: remove mempool.postCheck (#158) (#217)

* fix: error handling after check tx

* fix: typo

* chore: (mempool) remove postCheck and impl reserve

* chore: fix tests

* chore: revise log (remove checkTx.Code)

* chore: add `CONTRACT` for `mem.proxyAppConn.CheckTxAsync()`

* chore: revise numTxs, txsBytes for `ErrMempoolIsFull` in reserve()

* chore: revise to remove redundant `isFull()`

* fix: remove tx from cache when `app errors` or `failed to reserve`

* Revert "chore: revise to remove redundant `isFull()`"

This reverts commit 55990ec.

* fix: revise to call Begin/EndRecheck even though mem.Size() is 0 (#219)

* fix: revise to call Begin/EndRecheck even though `mem.Size()` is 0

* chore: revise local_client.go

* fix: lint error

* chore: recheckTxs() just return if mem.Size() == 0

* feat: concurrent recheckTx (#163) (#221)

* chore: increase the value of maxPerPage (#223)

* chore: fix the type of consensus_block_interval_seconds from histogram to gauge (#224)

* feat: impl checkTxAsyncReactor() (#168) (#225)

* feat: impl checkTxAsyncReactor() (#168)

* fix: tests

* fix: lint errors

* chore: revise abci.Client, Async() interfaces (#169) (#226)

* chore: revise abci.Client, Async() interfaces

* chore: regen mock w/ mockery 2.7.4

* fix: lint error

* fix: test_race

* mempool.Flush() flushes all txs from mempool so it should get `Lock()` instead of `RLock()`

* chore: remove iavl dependency (#228)

* chore: remove iavl dependency

* chore: fix lint error

* fix: add more fixing for abci.Client, Async()

* feat: revise metric for measuring performance

* build: remove needless build tag `!libsecp256k1` (#246)

The build tag makes disable go implementation of secp256k1.
Cause there is no C implementation, a build error will occur when using tag `libsecp256k1`.

* feat: add duration metrics of gauge type (#256)

* perf: optimize checking the txs size (#264)

* perf: optimize checking the txs size

* ci: add GOPRIVATE to workflows

* test: add a unit test

* fix: fix lint errors

* perf: do not flush wal when receive consensus msgs (#273)

* perf: do not flush wal when receive consensus msgs

* fix: ci failure

* fix: lint failure

* fix: ci-e2e build failure

* fix: bump up tm-db

* fix: missing abci api

* fix: bump up tm-db; use memdb

* test: add test case to raise test coverage

* fix: race error

* fix: race error

* fix: race error

* fix: increase e2e test timeout

* fix: add test case for coverage

* fix: e2e docker file

* fix: apply comments

* fix: a Ostracon to an Ostracon
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants