[WIP][Persistence][Core][Savepoints/Rollbacks] Implement KISS SavePoints - Serialize WorldState from Persistence - KV Stores POC - (Issue #327)

[deblasis]

Description

⚒️ work in progress - proof of concept ⚒️

This PR is meant to provide an example of how the implementation that relies on a serialized state that uses the stateTrees would look like.

The implementation is not complete.

The goal of this POC was to achieve feature parity with #645 but unfortunately I am like ~70% there.

The test:

(source: persistence/test/savepoints_test.go)

should succeed but currently it fails because GetAllAccounts and GetAccountAmount are not implemented.

What's also missing is the fact that the backup(s) that's created by the SavepointFactory has to be passed to the savepoint in a way that makes it queriable.

An alternative way could be to extend the SMT functionality to provide access to all the keys, loop through them and recreate KVStores with that data...
Perhaps a backup-restore is more efficient and less complex.

I'll leave to the reader/implementoooor to choose the best path forward 🙂

Issue

If we choose to complete the implementation, it would fix #327

Type of change

Please mark the relevant option(s):

• New feature, functionality or library
• Bug fix
• Code health or cleanup
• Major breaking change
• Documentation
• Other

List of changes

Testing

• make develop_test; if any code changes were made
• Docker Compose LocalNet; if any major functionality was changed or introduced
• k8s LocalNet; if any infrastructure or configuration changes were made

Required Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added, or updated, godoc format comments on touched members (see: tip.golang.org/doc/comment)
• I have tested my changes using the available tooling
• I have updated the corresponding CHANGELOG

If Applicable Checklist

• I have updated the corresponding README(s); local and/or global
• I have added tests that prove my fix is effective or that my feature works
• I have added, or updated, mermaid.js diagrams in the corresponding README(s)
• I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

[cr-gpt]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

[github-actions]

The changelog validation failed with the following output:

Missing changelog in module: persistence/

Missing changelog in module: shared/

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

[reviewpad]

AI-Generated Summary: This pull request includes changes across multiple files related to the persistence package, introducing new functionality for handling historical states, JSON formatted queries, KVStore backups, and improvements in error handling. Notable updates include the addition of a savepoint module with the implementation of the modules.PersistenceReadContext interface, the SavepointFactory interface, backupable KVStores, new methods for getting actors in JSON format, and modifications to the KVStore with names and Backup methods. Additionally, a new method GetAllJSONQuery has been introduced to various protocol schema interfaces, and error handling has been improved by adding the errCannotScanNULL constant. This pull request also updates the go.mod file with a new version of the dependency github.com/mattn/go-isatty, and introduces a test file savepoints_test.go for testing the GetAccountAmount method in the savepoints package.

[cr-gpt]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables for this repo. you could follow readme for more information

[github-actions]

The changelog validation failed with the following output:

Missing changelog in module: persistence/

Missing changelog in module: shared/

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

[reviewpad]

AI-Generated Summary: This pull request includes several changes across multiple files, primarily focusing on implementing savepoints and introducing related functionality such as backups of node stores and value stores. Major updates include:

1. Addition of a new test file savepoints_test.go and a new file savepoint.go as the skeleton implementation for the savepoint feature.
2. Modifications to the stateTrees struct, adding new methods and synchronizing concurrent access using a mutex.
3. Dependency version update for github.com/mattn/go-isatty from v0.0.16 to v0.0.17.
4. Minor updates in formatting and function calls throughout the codebase.
5. Removal of the NewSavePoint function from the PostgresContext struct.
6. Addition of savepoints_factory.go, implementing the modules.SavepointFactory interface for creating savepoints, including backup and rehydration functionality.
7. Introduction of BackupableKVStore interface, along with modifications to badgerKVStore to implement this interface.
8. Changes to the PersistenceWriteContext, PersistenceReadContext, and the addition of the new SavepointFactory interface.

These updates lay the groundwork for a more robust and efficient savepoint feature in the persistence module. Please review these changes and let me know if any additional modifications are required.

[Olshansk]

Closing out the draft PR as it'll be used as a reference and reopened if/when we fully implement it.