Toy Payment Engine

This crate implements toy payment engine which is exposed as library and binary executable.

Payment engine works by processing input transactions. Each transaction goes through validation process and eventually either rejected or applied to a corresponding client account.

Example

Payment engine can be run via binary executable as shown in example below:

cargo run --release -- transactions.csv

with sample transactions.csv having following content:

type, client, tx, amount
deposit, 1, 1, 1.0 
badrecord
deposit, 2, 2, 2.0 
deposit, 1, 3, 2.0
deposit, 1, 3, -2.0
deposit, 1, 3,
withdrawal, 1, 4, 1.5 
withdrawal, 2, 5, 3.0

This should produce output similar to the following:

client,available,held,total,locked
2,2.0,0,2.0,false
1,1.5,0,1.5,false

Tests can be executed via cargo test command.

Payment Engine Details

This section describes some of the internal details and constraints of the payment engine.

Transactions can be of the following types:

Deposit
Withdrawal
Dispute
Resolve
Chargeback

• Deposit and Withdrawal transactions must have positive amount associated with them.

• Dispute can only be applied to previously processed Deposit transactions. Dispute transaction results in withholding of a certain amount from client account's available balance. Withheld amount can later be deducted from client account if Chargeback transaction follows, or it is put back to client's available balance back if Resolve transaction is issued.

• This crate depends decimal crate because floating point types are not suitable for financial calculation as they unable to precisely represent some of the decimal values.

Design overview

PaymentEngine is I/O agnostic and hence can be used in various environments.

Transaction store

Number of transaction that have to be stored for potential disputes can be quite large and thus might not fit into main memory and cause process crash. To address this problem, payment engine relies on redb embedded database which writes transactions onto disk.

Reading and writing every transaction to database on disk is quite slow. Transaction store has LRU-cache for quickly reading recently used transactions. Also write log buffers updates to transaction store so that transaction changes are flushed to disk in batches. This significantly increases write performance when saving transactions.

Usage within async runtime

Current API does not use async functions because underlying backing store (redb) only exposes synchronous API. However, this crate still can be used in server setting along with asynchronous runtime crates such as tokio. This can be achieved by running payment engine in a dedicated thread which would read input transactions from a mpsc channel. While tokio's worker threads would read transactions from network and put them into input channel.

Further optimizations

• Multiple instances of payment engine can be executed in seprate thread. Each payment engine instance would be partitioned by client account identifier, and will handle transactions for a subset of client accounts.

• Flushing of transactions from write log to disk can be done in a background thread.

Benchmarks

Benchmarks can be run via cargo bench command. Results of bechmarks are significantly affected by parameters of BufferedDepositStore such as sizes of write log, LRU cache and file storage cache. These parameters can be fine-tuned to adhere to workload and available resources.