Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
performance: about 5x speedup for inserting into the cache
Seems that `.values()` clause is pretty slow comparing to using `.execute` method instead. Didn't dig too much why exactly, but the docs kind of give a clue here: https://docs.sqlalchemy.org/en/14/core/dml.html#sqlalchemy.sql.expression.Insert.values > It is essential to note that passing multiple values is NOT the same as using traditional executemany() form. The above syntax is a special syntax not typically used. > To emit an INSERT statement against multiple rows, the normal method is to pass a multiple values list to the Connection.execute() method, which is supported by all database backends and is generally more efficient for a very large number of parameters. Benchmark: running `taskset -c 7 pytest -rap --durations=0 -s src/cachew/tests/test_cachew.py -k 'test_many[500000-False]'` on @karlicoss desktop computer. Using taskset to pin the process to a single core just in case, although that shouldn't matter (note that there is still some jitter, but not sure what it's caused by, might be page faults or something). baseline -- running without cache at all (@Cachew decorator commented out): 0.86s 0.82s 0.83s 0.82s 0.84s before the change: 23.29s 24.44s 25.25s 24.41s 24.46s after the change: 5.90s 5.83s 5.75s 5.72s 5.87s
- Loading branch information