- Project models in
models.py
must be subclassed fromdipdup.models.Model
instead oftortoise.Model
. - The deprecated
on_rollback
event hook has been removed in favor ofon_index_rollback
. - HTTP datasources no longer use caching.
cache show
andcache clear
commands have been removed.http.cache
config flag has been removed. --logging-config
option has been removed. Use thelogging
config section or set up logging manually.- Feature flag options of
run
command have been removed. Use theadvanced
config section instead.
- Replace
tortoise.Model
import withdipdup.models.Model
in yourmodels.py
module. - Remove the
on_rollback
event hook if it still persists in your project. Ensure thaton_index_rollback.py
containsctx.rollback
call, or remove it and calldipdup init
. - If you have used
buffer_size
config option, remove it to use database-level rollbacks. - Run
schema approve
command with every schema you want to use with DipDup 6.0.
The era of handling chain reorgs manually is finally over! Now when DipDup receives a reorg message from TzKT it just rewinds a database to the previous state reverting changes in backtracked blocks level by level. To make this possible, DipDup catches all database modifications and saves diffs in a separate table, dipdup_model_update
(you don't need to access it directly).
# INSERT saved with no data, just drop this row on reorg.
trader = Trader(name='Alice', balance=100, active=True)
await trader.save()
# UPDATE saved with data old values to set them on reorg.
# Diff: {'balance': 100}
trader.balance = 200
await trader.save()
# DELETE saved with full copy of data. On reorg this row will be recreated with the same PK.
# Diff: {'name': 'Alice', 'balance': 200, 'active': True}
await trader.delete()
Bulk class methods like bulk_insert
and bulk_update
are supported too. However, for raw queries, DipDup uses prefetching (additional SELECT) to save original values. So, ReallyHugeTable.filter().delete()
will create efficiently a full copy of the table in dipdup_model_update
. Most likely you will never need to perform such queries in handlers, but keep that detail in mind.
Since the Ithacanet protocol, only two last blocks may be backtracked. We do not need to store older diffs, they are removed automatically. If you need to keep more levels or disable this feature, adjust rollback_depth
config option.
advanced:
rollback_depth: 2 # 0 to disable
on_index_rollback
event hook now looks like this:
from dipdup.context import HookContext
from dipdup.index import Index
async def on_index_rollback(
ctx: HookContext,
index: Index, # type: ignore[type-arg]
from_level: int,
to_level: int,
) -> None:
await ctx.execute_sql('on_index_rollback')
await ctx.rollback(
index=index.name,
from_level=from_level,
to_level=to_level,
)
TzKT buffer_size
option remains available, but it's not required to handle chain reorgs anymore.
Now when DipDup catches unhandled exceptions, a crash dump will be saved to the temporary directory.
dipdup.exceptions.CallbackError: An error occured during callback execution
________________________________________________________________________________
`demo_token.hooks.on_restart` callback execution failed:
Exception:
Eliminate the reason of failure and restart DipDup.
________________________________________________________________________________
Crashdump saved to `/tmp/dipdup/crashdumps/veb7kz07.json`
This JSON file is the same data Sentry collects on crashes if integration is enabled. It includes a stack trace, local variables of each frame, and other information useful when investigating a crash. Attach this file when sending bug reports to GitHub Issues.
When preparing a crash dump Sentry can detect sensitive information like database passwords in the crash dump and remove it from the report. So it's generally safe to share the crash dump with the developers. Now you can also send these crash reports automatically to the Baking Bad team.
Your privacy matters; crash reporting is disabled by default. Simulate a crash with a random exception and inspect a crash dump before enabling this option to ensure that report doesn't contain secrets. Then add the following lines to your config:
advanced:
crash_reporting: True
- cli: Added
config export --full
flag to resolve templates before printing config. - config: Added
advanced.crash_reporting
flag to enable reporting crashes to Baking Bad. - config: Added
advanced.rollback_depth
field, a number of levels to keep in a database for rollback. - context: Added
rollback
method to perform database rollback. - database: Added an internal
ModelUpdate
model to store the latest database changes. - dipdup: Save Sentry crashdump in
/tmp/dipdup/crashdumps/XXXXXXX.json
on a crash.
- config: Do not perform env variable substitution in commented-out lines.
- prometheus: Fixed updating
dipdup_index_handlers_matched_total
metric.
- codegen:
on_index_rollback
hook callsctx.rollback
by default. - database: Project models must be subclassed from
dipdup.models.Model
- cli:
--logging-config
option is removed. - cli: All
run
command flags are removed. Use theadvanced
section of the config. - cli:
cache show
andcache clear
commands are removed. - config:
http.cache
flag is removed. - hooks: Removed deprecated
on_rollback
hook. - index: Do not try to avoid single-level rollbacks by comparing operation hashes.