Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve wallet consistency #3305

Merged
merged 5 commits into from
May 1, 2021
Merged

Improve wallet consistency #3305

merged 5 commits into from
May 1, 2021

Conversation

mariano54
Copy link
Contributor

@mariano54 mariano54 commented May 1, 2021 •
edited
Loading

We were seeing a wallet creating two transactions from the same coin, and one was pending and stuck forever (leaving a negative balance and inability to make transactions). I think the issue was:

  • coroutine 1: We select unspent coin A that we are going to spend
  • coroutine 2: Marks coin A as spent (we get back a tx from the blockchain), but there is no unconfirmed tx for A yet so we don't remove anything
  • coroutine 1: Create an unconfirmed TX for spending coin A, since we think it's still unspent since we selected it
    The problem was that a different lock was being used. Coroutine 1 was using tx_lock, but coroutine 2 was using blockchain lock and db_wrapper lock.

It looks like a big change, but it's just moving a lock around. Uses the WalletStateManager lock whenever selecting or spending coins, to avoid double spending the same coins. Before this lock was only in select_coins, but that was too narrow. It has now been moved to wallet_rpc_api, and it wraps any async operation which needs to use and spend coins. Also it wraps the blockchain state transition, so when we get new coins in the blockchain, it does not interfere with making transactions.

A few other nice to have features:

  • Self spending does not break the balance display.
  • Large performance boost when using -f in CLI, does not need to fetch things from the keychain
  • Reworded CLI balance to be much more clear, and analogous to the GUI
  • Show Sync status for wallet in CLI, and change to XCH instead of mojo

@wjblanke wjblanke requested a review from Yostra May 1, 2021 15:10
@Yostra
Copy link
Contributor

Yostra commented May 1, 2021

Good job! @mariano54

@Yostra Yostra merged commit fe257cd into main May 1, 2021
wjblanke added a commit that referenced this pull request May 1, 2021
@wjblanke @mariano54 @hoffmang9
Merge main into testnet7 (#3354)
40faaa4 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
wjblanke added a commit that referenced this pull request May 3, 2021
@wjblanke @mariano54
@hoffmang9 @arvidn
merge main into testnet7 (#3534)
3f1b9ff 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

* Apologies to @Chida82 who added log rotate count! (#3369)

* Rust parse serialized (#3444)

* use rust implementation for finding length of a serialized clvm program

* bump clvm_rs version

* Don't retry respond_peers message (#3508)

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
wjblanke added a commit that referenced this pull request May 3, 2021
@wjblanke @mariano54
@hoffmang9 @arvidn
merge main to testnet7 (#3552)
a28c333 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

* Apologies to @Chida82 who added log rotate count! (#3369)

* Rust parse serialized (#3444)

* use rust implementation for finding length of a serialized clvm program

* bump clvm_rs version

* Don't retry respond_peers message (#3508)

* don't increment counters for outgoing messages blocked by the rate limit. (#3518)

This was causing a problem where outbound messages, blocked by the rate limiter,
would still increment the counters as-if they had been sent. This, in turn,
could cause other message types to get blocked becuase the rate limiter thought
we had sent a lot of the other (blocked) message type.

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
wjblanke added a commit that referenced this pull request May 3, 2021
@wjblanke @mariano54
@hoffmang9 @arvidn @aqk
merge main into testnet7 (#3573)
dac6048 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

* Apologies to @Chida82 who added log rotate count! (#3369)

* Rust parse serialized (#3444)

* use rust implementation for finding length of a serialized clvm program

* bump clvm_rs version

* Don't retry respond_peers message (#3508)

* don't increment counters for outgoing messages blocked by the rate limit. (#3518)

This was causing a problem where outbound messages, blocked by the rate limiter,
would still increment the counters as-if they had been sent. This, in turn,
could cause other message types to get blocked becuase the rate limiter thought
we had sent a lot of the other (blocked) message type.

* hide secret wallet key by default with 'chia keys show' (#3565)

Co-authored-by: Adam Kelly <aqk@aqk.im>

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Adam Kelly <aqk@aqk.im>
wjblanke added a commit that referenced this pull request May 4, 2021
@wjblanke @mariano54
@hoffmang9 @arvidn @aqk @almogdepaz
merge main to testnet7 (#3624)
efcf655 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

* Apologies to @Chida82 who added log rotate count! (#3369)

* Rust parse serialized (#3444)

* use rust implementation for finding length of a serialized clvm program

* bump clvm_rs version

* Don't retry respond_peers message (#3508)

* don't increment counters for outgoing messages blocked by the rate limit. (#3518)

This was causing a problem where outbound messages, blocked by the rate limiter,
would still increment the counters as-if they had been sent. This, in turn,
could cause other message types to get blocked becuase the rate limiter thought
we had sent a lot of the other (blocked) message type.

* hide secret wallet key by default with 'chia keys show' (#3565)

Co-authored-by: Adam Kelly <aqk@aqk.im>

* brake if end of ses list (#3583)

* brake if end of ses list

* log

* throw if wp peak is not heavier

* comment

* handle sync from scratch

* comment

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Adam Kelly <aqk@aqk.im>
Co-authored-by: Almog De Paz <almogdepaz@gmail.com>
wjblanke added a commit that referenced this pull request May 4, 2021
@wjblanke @mariano54
@hoffmang9 @arvidn @aqk @almogdepaz
merge main into testnet7 (#3710)
e195f08 
* 2 harvesting features (#3331)

* 3 harvesting features:
- Debug level shows the time for every quality lookup
- Warning level if takes longer than 5 seconds
- Allow configuration of plot loading interval (default 2 minutes)

* Comment out super logging

* Improve wallet consistency (#3305)

* Improve wallet consistency

* Improve CLI significantly, and fix self-tx balances

* Fix await

* Fix deadlock and test

* Remove spam.sh

* Changelog for 1.1.3 (#3345)

* Changelog for 1.1.3

* minor updates

* updates part 3

* Those engineers who don't update changelogs... :)

* Apologies to @Chida82 who added log rotate count! (#3369)

* Rust parse serialized (#3444)

* use rust implementation for finding length of a serialized clvm program

* bump clvm_rs version

* Don't retry respond_peers message (#3508)

* don't increment counters for outgoing messages blocked by the rate limit. (#3518)

This was causing a problem where outbound messages, blocked by the rate limiter,
would still increment the counters as-if they had been sent. This, in turn,
could cause other message types to get blocked becuase the rate limiter thought
we had sent a lot of the other (blocked) message type.

* hide secret wallet key by default with 'chia keys show' (#3565)

Co-authored-by: Adam Kelly <aqk@aqk.im>

* brake if end of ses list (#3583)

* brake if end of ses list

* log

* throw if wp peak is not heavier

* comment

* handle sync from scratch

* comment

* fix indentation issue in message send retry logic. factor out retry coroutine (#3629)

* Fix test (#3350)

* Mempool sorting and accept reverted pending transactions (#3683)

* Sort by fee/cost, and fix pending tx issue in reorgs

* Fix test name

* Bring the seen list size back to normal.

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Adam Kelly <aqk@aqk.im>
Co-authored-by: Almog De Paz <almogdepaz@gmail.com>
@justinengland justinengland deleted the ms.wallet_consistency branch May 14, 2021 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yostra Yostra Yostra approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mariano54 @Yostra