-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fee bumping Package #114
base: main
Are you sure you want to change the base?
Fee bumping Package #114
Conversation
Create a new package for fee bumping functionality in Caravan. This package provides modular and reusable utilities for implementing RBF (Replace-By-Fee) and CPFP (Child-Pays-For-Parent) fee bumping strategies. Key additions: - Basic package structure and configuration files - Core modules for RBF and CPFP implementations - Fee estimation utilities - Type definitions for fee bumping operations - Constants and utility functions This package aims to enhance Caravan's transaction management capabilities by providing a flexible and extensible fee bumping solution that can be easily integrated into the Caravan coordinator wallet and other Bitcoin projects.
This commit introduces a new @caravan/fee-bumping package that provides robust functionality for handling Replace-By-Fee (RBF) and Child-Pays-For-Parent (CPFP) fee bumping strategies in Bitcoin transactions. The package is designed to work seamlessly with other Caravan packages and offers a flexible, modular approach to fee bumping. Key features: - Implement RBFHandler for Replace-By-Fee transactions - Implement CPFPHandler for Child-Pays-For-Parent transactions - Create FeeEstimator for dynamic fee rate calculations - Provide utility functions for transaction analysis and manipulation - Define comprehensive types for improved type safety and developer experience - Implement constants for fee-related thresholds and limits The package structure includes: - src/index.ts: Main entry point and FeeBumping class - src/types.ts: TypeScript interfaces and types - src/estimator.ts: FeeEstimator class for fee calculations - src/rbf.ts: RBFHandler class for RBF operations - src/cpfp.ts: CPFPHandler class for CPFP operations - src/utils.ts: Utility functions for transaction handling - src/constants.ts: Constant values used throughout the package This package provides a solid foundation for implementing fee bumping strategies in Caravan-based applications. It offers: 1. Flexible fee estimation based on network conditions 2. Configurable options for both RBF and CPFP operations 3. Built-in safeguards against excessive fees 4. Utility functions for transaction analysis and modification The implementation is designed with extensibility in mind, allowing for easy updates and additions to fee bumping strategies in the future. Next steps: - Implement comprehensive unit tests for all components - Integrate the package with the main Caravan UI - Create documentation and usage examples - Perform thorough testing with various transaction scenarios This addition significantly enhances Caravan's capabilities in handling dynamic fee environments, improving overall user experience and transaction reliability.
- Add new types for UTXO, TransactionOutput, TransactionAnalysis, FeeRate, FeeEstimate, TransactionDetails, BlockchainClientInterface, RBFOptions, and CPFPOptions in types.ts - Create getFeeEstimates function to fetch fee estimates from blockchain client - Implement calculateFee function to compute transaction fee based on vsize and fee rate - Use BigNumber for precise calculations - Import necessary constants and interfaces This commit lays the groundwork for RBF and CPFP implementations by defining essential types and providing core fee estimation functionality.
- Create new file analyzer.ts with analyzeTransaction function - Implement logic to determine if a transaction can use RBF or CPFP - Calculate current fee rate of the transaction - Provide a recommended fee bumping method based on transaction properties - Utilize utility functions isRBFSignaled and calculateEffectiveFeeRate - Return a TransactionAnalysis object with fee bumping options This commit adds crucial functionality for analyzing transactions to determine the best fee bumping strategy, supporting the implementation of RBF and CPFP features in the Caravan wallet.
This commit significantly refactors the fee bumping package to improve its functionality, integration with bitcoinjs-lib, and overall code structure. Key changes include: RBF (Replace-By-Fee): - Updated to work with bitcoinjs-lib Transaction objects - Implemented dynamic input and output handling - Added support for transaction cancellation and fee subtraction from outputs CPFP (Child-Pays-For-Parent): - Refactored to handle multiple parent outputs - Implemented dynamic input and output calculation - Added support for additional outputs in child transactions - Improved fee calculation accuracy Utils: - Added deriveAddress function for working with bitcoinjs-lib Output objects - Updated isRBFSignaled and calculateEffectiveFeeRate functions - Removed estimateVirtualSize in favor of using Transaction.virtualSize() Transaction Analysis: - Implemented analyzeTransaction function for recommending fee bumping strategies - Added logic to compare current fee rate with network fee rate Removed: - Eliminated estimator.ts file, integrating necessary functionality into other modules Tests: - Added comprehensive test suite for analyzeTransaction function - Updated existing tests to reflect new implementations of RBF and CPFP functions Types: - Updated and expanded type definitions to better integrate with bitcoinjs-lib - Added new interfaces for RBFOptions, CPFPOptions, and TransactionAnalysis
🦋 Changeset detectedLatest commit: 90b3227 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really good stuff here so far! Only reviewed cpfp so far, but wanted to submit some comments to start.
High level feedback is that I would be curious to see if you think you could reduce the dependence on a consumer of this package from having to use other external dependencies as much as possible. Right now someone that wants to interact and leverage this package outside of this monorepo MUST have some awareness of the use of bitcoinjs-lib and bignumber library and use the same libraries themselves. Ideally any use of bignumbers is contained to within the functions themselves. This means that any consumer of the function can use whatever BN library they prefer and just convert to/from strings when interacting with the function.
Same with Transaction
from bitcoinjs-lib. It would be nice if someone that wanted to estimate fees or construct a CPFP didn't have to first build a bitcoinjs-lib Transaction object (and what happens if they're using a different/incompatible version??)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some more comments on the other files. Excited about the progress!
…gnalling functionality to PSBTV2 Key changes: 1. rbf.ts: - Introduced RbfTransaction class to encapsulate RBF logic - Implemented methods for preparing accelerated and canceled transactions - Added helper methods for fee calculations and PSBT manipulations - Created public functions prepareRbfTransaction and prepareCancelTransaction as entry points for RBF operations 2. psbtv2.ts: - Added isRbfSignaled method to check if a transaction is RBF-enabled
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've only reviewed the PsbtV2
class changes. I like the addition of another value setter and getter. There are so many of these unbuilt which could make this package really powerful. However, I take issue with the bip125-specific changes. See line comments.
Additionally, adding a test for setInputSequence
would be good.
* Care should be taken when setting sequence numbers to ensure the desired | ||
* transaction properties are correctly signaled. | ||
*/ | ||
private setInputSequence(inputIndex: number, sequence: number) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to bip370, only the Updater operator role can change the input sequence. Check for isReadyForUpdater
to be true before changing sequence number else throw an error.
Does it also need to check for the existance of locktime? Im not sure. Maybe. An input is supposed to have a locktime with a sequence, but psbts have nLockTime
which is just an aggregate value of all locktimes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks , I'll make the correction , so is it good to keep it as a private method or make it public ?
Also nLocktime
is a transaction level feature while nSequence
is an input level feature and normally the node checks for nLocktime
first while validation , so I guess psbt's nLockTime
should work for this purpose .
Your thoughts ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, this would need to be a public method in order to provide the Updater role with the ability to add a sequence value to an input.
As for the locktime, well, I see now that I added some TODOs in the addInput
method for handling locktimes. So, you can currently add a sequence without a locktime on the input (I don't know if this is valid). At the very least, the global fallback locktime will be used. Since the locktime is outside the scope of your change, maybe just ensure this checks isReadyForUpdater
for now and leave the locktime checking for another time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is also all still a little vague for me as well, but based on this I think that leaving locktime for another time is. My read is that checking for locktime as well is if you're trying to enforce timelocks, but if you're just using it for signaling then setting the sequence is fine.
This should be easily verifiable though. update a tx with this (can be in regtest) and see if the node accepts it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couple more comments. I like the move to the class for the RBF!
This commit introduces new classes for Child-Pays-for-Parent (CPFP) and refactors majority of Replace-By-Fee (RBF) transaction creation, along with comprehensive tests. Key changes: - Add CPFPTransaction class for creating child transactions - Refactored RbfTransaction class for preparing replacement transactions - Create tests for CPFP, RBF - Update existing transaction utilities to support new classes The new classes support both PSBTv2 and PSBTv0 formats, handle various edge cases, and provide methods for fee calculation and transaction preparation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
leaving some more comments, going to wait for a full review as it looks like there are some changes still in progress.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The classes are much much better and the tests are improving as well. Still some things that need to be cleaned up though.
…e calculation - Add state management to track acceleration and cancellation status - Implement caching for fee increase and vsize calculations - Introduce incremental relay fee for accurate fee bump calculations - Refactor fee calculation logic to comply with BIP 125 rules - Optimize input selection by sorting UTXOs before addition - Improve RBF signaling by updating only one input sequence - Added more comments for better code understanding
…(Replace-By-Fee) support - setInputSequence(inputIndex: number, sequence: number): A method for setting the sequence number of a specific input, critical for signaling RBF and handling nLockTime. - get isRBFSignaled(): boolean: A method to check if the transaction signals RBF, as per BIP125 guidelines. - Added tests
…f the original fees
break; // Stop adding inputs once fee requirements are met | ||
if ( | ||
newTxTemplate.feeRateSatisfied && | ||
new BigNumber(newTxTemplate.currentFee).gte( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you still use “areFeesPaid” here, which should cover target fees, and then maybe there can be another method that does “are fees greater than” that you can pass the rbf value to for the comparison.
This is obviously a very important part of the logic so just thinking of ways to make it as readable as possible
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also I had a doubt ... do you think that instead of using
txAnalyzer.minimumRBFFee
we should check for max of the newTxTemplate.targetFeesToPay
and absolute of original tx
... as minimumRBFFee
is in turn calculated using the either targetFeeRate (set by user ) or old fee Rate + incrementalRelay FeeRate .... using the original tx size...
While newTxTemplate.targetFeesToPay
calculates based on rbfFeeRate (which is calculated using minimumRBFFee
) and current tx size (after bumping) ...
so
const fee = BigNumber.max(
newTxTemplate.targetFeesToPay,
txAnalyzer.fee,
);
would be correct code then ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is how we intialise the newTxTemplate
// Step 3: Create a new transaction template
const newTxTemplate = new BtcTransactionTemplate({
inputs: [],
outputs: [],
targetFeeRate: Number(txAnalyzer.rbfFeeRate), // Use RBF-specific fee rate from analyzer
dustThreshold,
network,
scriptType,
requiredSigners,
totalSigners,
});
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
like for this fixture ....
For this multisig transaction, we have two inputs and three outputs, each of P2WSH
type. The original transaction pays 20,880 sats, and the transaction vsize is 348 vB. Therefore, the current fee rate is 60 sat/vB.
Scenario: RBF to Cancel Transaction
Suppose we use RBF to cancel this transaction and set a target fee rate of 80 sat/vB. In a cancellation, the size of the transaction typically decreases because we have only one output (cancellation address), and in most cases, a single input.
- The new transaction size becomes 159 vB.
- If we pay the original fee of 20,880 sats, the new fee rate becomes 131 sat/vB.
However, even though a fee of 12,720 sats would achieve the desired fee rate of 80 sat/vB, we still end up paying 20,880 sats to satisfy the RBF conditions.
now as per current code the newTxTemplate.targetFeesToPay
gives 12,720 sats as 159vB * 80 sat/vB ... and txAnalyzer.minimumRBFFee
gives ... 348vB (original tx size) * 80 sat/vB ==> 27_840sats and feeRate becomes 175 sat/vB ... so we overpay a lot
here
// https://mempool.space/tx/be81b81620702718957d445611066bd596fe1840e219b84f6ea60e0114a7a305
{
case: "Create a valid cancel RBF transaction",
originalTx:
"010000000001022b2be3ac975c385defbda23a1eec30946b708c209981a5c07830fd726b3e25530200000000fdffffffe635a5d14ea5eb63d455948c0894d61ac35f7ba00f260f128b725f25855bd6550400000000fdffffff03400caa3b00000000220020701a8d401c84fb13e6baf169d59684e17abd9fa216c8cc5b9fc63d622ff8c58d886c5e1b000000002200200a618b712d918bb1ba59b737c2a37b40d557374754ef2575ce41d08d5f782df980960d04000000002200202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e048040047304402202c0be2d56da6f551b92e657a1f29bc410f244d68063bbc14297bae318d06a32d0220276ddf6da48d0dfdc717974aee32b94a45867393a663df8bc1e8382e1f7d9989014730440220649a73751ad934c9e517022bc8b510d53474772286e5678227ae71837ec0307d022031ec829069c7ab474759741c6acd1efb7e0918d5d63e4fbd1071ce3ea1bfc6ec01695221022dfa322241a4946b9ead36ab9c8c55bd4c4340a1290b5bf71d23a695aeb1240a21034d82610a17c332852205e063c64fee21a77fabc7ac0e6d7ada2a820922c9a5dc2103c96d495bfdd5ba4145e3e046fee45e84a8a48ad05bd8dbb395c011a32cf9f88053ae040047304402207361009ec2357c8d9f178a79772715d29308f1c265bf3c31f083d9253dcedd3c02203fbe90a73f2ca763233cacd9f6be1ebc24abdeff928d5ad454c6fd50b9bc869d014730440220416d0e1acfd95024dc31bc534fee0662ffd906dba98040ac4ed039bf4451176d02203b9416a93b4a116f4edcbfd256e5f232bc9b9f12f6270e84e23597ce68f24ca201695221022dfa322241a4946b9ead36ab9c8c55bd4c4340a1290b5bf71d23a695aeb1240a21034d82610a17c332852205e063c64fee21a77fabc7ac0e6d7ada2a820922c9a5dc2103c96d495bfdd5ba4145e3e046fee45e84a8a48ad05bd8dbb395c011a32cf9f88053ae00000000",
network: Network.MAINNET,
inputs: [
{
// https://mempool.space/tx/53253e6b72fd3078c0a58199208c706b9430ec1e3aa2bdef5d385c97ace32b2b#flow=&vout=2
txid: "53253e6b72fd3078c0a58199208c706b9430ec1e3aa2bdef5d385c97ace32b2b",
vout: 2,
value: "932049200",
sequence: 4294967293,
},
{
// https://mempool.space/tx/55d65b85255f728b120f260fa07b5fc31ad694088c9455d463eba54ed1a535e6#flow=&vout=4
txid: "55d65b85255f728b120f260fa07b5fc31ad694088c9455d463eba54ed1a535e6",
vout: 4,
value: "596144040",
sequence: 4294967293,
},
],
availableUtxos: [
// (Original Input) https://mempool.space/tx/53253e6b72fd3078c0a58199208c706b9430ec1e3aa2bdef5d385c97ace32b2b#flow=&vout=2
{
txid: "53253e6b72fd3078c0a58199208c706b9430ec1e3aa2bdef5d385c97ace32b2b",
vout: 2,
value: "932049200",
witnessUtxo: {
script: Buffer.from(
"00202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e048",
"hex",
),
value: 932049200,
},
prevTxHex:
"010000000145a8a366f1e073e4e1a03172b95e7db2f006d710d3585e5d14a137c218bf90ed03000000fc00473044022056da571c657d063d18217cb3d2cdb827d11fdd0fdef9ffc0ef69345628c0da260220092f7fe2b2d2cba7d1f4d53f0fb7db54dd15c6b8aac2f8e66102778aac0a62250147304402200bf2e4c82a2337168b7370deae483c88521367da8415c65ffa24b21ba4b13a7f02207a946b5c0f9f9213be130022d83f2ee91363c08de8cd2460765e020f59a3399a014c695221026fcec918a19aad4c92527f4bad924c7cc8dfdba935418f3ce217c1c839a58b952103c96d495bfdd5ba4145e3e046fee45e84a8a48ad05bd8dbb395c011a32cf9f8802103d1fc482d299248b97e38d9042a068eb102818a3556d5247d080849a0880a9e2653aefdffffff03400caa3b000000002200203eb5062a0b0850b23a599425289a091c374ca934101d03144f060c5b46a979be181d1a1a000000002200200a618b712d918bb1ba59b737c2a37b40d557374754ef2575ce41d08d5f782df930f18d37000000002200202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e04800000000",
},
{
// (Original Input 2) https://mempool.space/tx/55d65b85255f728b120f260fa07b5fc31ad694088c9455d463eba54ed1a535e6#flow=&vout=4
txid: "55d65b85255f728b120f260fa07b5fc31ad694088c9455d463eba54ed1a535e6",
vout: 4,
value: "596144040",
witnessUtxo: {
script: Buffer.from(
"00202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e048",
"hex",
),
value: 596144040,
},
prevTxHex:
"010000000145a8a366f1e073e4e1a03172b95e7db2f006d710d3585e5d14a137c218bf90ed05000000fc004730440220414f5352d874fad4171326bdc563f2724eadbbe8e3199fbc83f9d6d04dc02176022072ec117a06d340209b44cef533d1a2f21c0ace64f653f454e745add290938e53014730440220367ff913573c50706003423b47648553730d0377c7707e505d1a4f9a08313ab6022073fad7c23abeb4ca1d48075bd276c7a5aa7b0d942ad741c9a8a4233a264f2df1014c695221026fcec918a19aad4c92527f4bad924c7cc8dfdba935418f3ce217c1c839a58b952103c96d495bfdd5ba4145e3e046fee45e84a8a48ad05bd8dbb395c011a32cf9f8802103d1fc482d299248b97e38d9042a068eb102818a3556d5247d080849a0880a9e2653aefdffffff05400caa3b000000002200200a618b712d918bb1ba59b737c2a37b40d557374754ef2575ce41d08d5f782df90084d71700000000220020701a8d401c84fb13e6baf169d59684e17abd9fa216c8cc5b9fc63d622ff8c58d0084d71700000000220020701a8d401c84fb13e6baf169d59684e17abd9fa216c8cc5b9fc63d622ff8c58d3819b1110000000022002014b288dca5d59caa8868d1668c97c971e58ab3ccf10534ac567ea51aa8aba299a86f8823000000002200202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e04800000000",
},
{
// extra UTXO , NOT IN ORIGINAL TX
// https://mempool.space/tx/89caa4dff1251f3a0e62b662ee1b22e8cffeb32bf2cf839ff959d62fe56a3562
txid: "55d65b85255f728b120f260fa07b5fc31ad694088c9455d463eba54ed1a535e6",
vout: 3,
value: "296819000",
witnessUtxo: {
script: Buffer.from(
"002014b288dca5d59caa8868d1668c97c971e58ab3ccf10534ac567ea51aa8aba299",
"hex",
),
value: 296819000,
},
prevTxHex:
"010000000145a8a366f1e073e4e1a03172b95e7db2f006d710d3585e5d14a137c218bf90ed05000000fc004730440220414f5352d874fad4171326bdc563f2724eadbbe8e3199fbc83f9d6d04dc02176022072ec117a06d340209b44cef533d1a2f21c0ace64f653f454e745add290938e53014730440220367ff913573c50706003423b47648553730d0377c7707e505d1a4f9a08313ab6022073fad7c23abeb4ca1d48075bd276c7a5aa7b0d942ad741c9a8a4233a264f2df1014c695221026fcec918a19aad4c92527f4bad924c7cc8dfdba935418f3ce217c1c839a58b952103c96d495bfdd5ba4145e3e046fee45e84a8a48ad05bd8dbb395c011a32cf9f8802103d1fc482d299248b97e38d9042a068eb102818a3556d5247d080849a0880a9e2653aefdffffff05400caa3b000000002200200a618b712d918bb1ba59b737c2a37b40d557374754ef2575ce41d08d5f782df90084d71700000000220020701a8d401c84fb13e6baf169d59684e17abd9fa216c8cc5b9fc63d622ff8c58d0084d71700000000220020701a8d401c84fb13e6baf169d59684e17abd9fa216c8cc5b9fc63d622ff8c58d3819b1110000000022002014b288dca5d59caa8868d1668c97c971e58ab3ccf10534ac567ea51aa8aba299a86f8823000000002200202122f4719add322f4d727f48379f8a8ba36a40ec4473fd99a2fdcfd89a16e04800000000",
},
],
dustThreshold: "546",
targetFeeRate: 80, // original was 60 sat/vB
scriptType: SCRIPT_TYPES.P2WSH,
requiredSigners: 2,
totalSigners: 3,
cancelAddress:
"bc1qyy30guv6m5ez7ntj0ayr08u23w3k5s8vg3elmxdzlh8a3xskupyqn2lp5w", // https://mempool.space/address/bc1qyy30guv6m5ez7ntj0ayr08u23w3k5s8vg3elmxdzlh8a3xskupyqn2lp5w
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh interesting. That’s useful context. Then shouldn’t areFeesPaid
cover it since it’s set to target the rbf rate?
I’m on mobile so I’m sure I’m missing something from all the different contexts but if the templates target is set from the analyzers rbf recommendation basically, then why doesn’t that catch all of the conditionals you’re looking at?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ahh ... yes actually ....
txAnalyzer.rbfFeeRate
can be equal to targetFeeRate
iff it is greater than BIP-125
based feeRate ... i.e
(original Fee Rate + incremental Relay)
which in this case is as RBF suggests a feeRate of 61 sats/vB
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh interesting. That’s useful context. Then shouldn’t
areFeesPaid
cover it since it’s set to target the rbf rate?I’m on mobile so I’m sure I’m missing something from all the different contexts but if the templates target is set from the analyzers rbf recommendation basically, then why doesn’t that catch all of the conditionals you’re looking at?
I am still trying to figure out how can we define this more clearly and remove the complexity ... surrounding it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iff it is greater than BIP-125 based feeRate ... i.e (original Fee Rate + incremental Relay)
But I thought the only rate mentioned in the bip was the minimum relay fee setting
, so original fee rate doesn't really factor in here (but will for accelerations implicitly since you'll want a higher fee rate to fee bump).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, as I've been re-reviewing and thinking about this more, I think part of the complexity comes from this comment you made:
minimumRBFFee is in turn calculated using the either targetFeeRate (set by user ) or old fee Rate + incrementalRelay FeeRate .... using the original tx size...
I commented on this in the analyzer as well. This is redirection that can be confusing. It's not clear that first of all that this check is being made which makes minimumRBFFee
harder to understand. Have the property describing absolute fee, stick to the rules related to absolute fees.
Second, I think if you make the change at the top that I suggested, where the targetFeeRate from the user is validated to be at least as much as the original, then you can simplify to this:
if (
newTxTemplate.feeRateSatisfied &&
new BigNumber(newTxTemplate.currentFee).gte(txAnalyzer.minimumRbfFee)
) {
// Stop adding inputs when both conditions are met:
// 1. The fee rate satisfies the target rate (which must be gte original tx fee rate
// 2. The current fee is gte the minimum required absolute RBF fee amount
break;
}
this should handle the situation you described above where you can hit the target fee rate but still not be paying enough absolute fees (because the replacement tx is smaller) and so you have to keep adding inputs to increase the current fee to be higher than minimumRbfFee
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, one problem here is that you haven't added anything to the change yet (which is why you could run into the issue below about dusty change). So this check should actually confirm that there is enough left over to have more than dust as change. if you add a property to the analyzer needsChange
that just checks if there is leftover funds greater than the dust threshold, then you can do something like this:
if (
newTxTemplate.feeRateSatisfied &&
new BigNumber(newTxTemplate.currentFee).gte(txAnalyzer.minimumRbfFee) &&
newTxTemplate.needsChange
) {
// Stop adding inputs when the following conditions are met:
// 1. The fee rate satisfies the target rate (which must be gte original tx fee rate
// 2. The current fee is gte the minimum required absolute RBF fee amount
// 3. There is enough left over for change
break;
}
With this the checks on the change aren't necessary and the extra validation afterwards also probably isn't necessary.
Note, this is probably only necessary for cancelation since you need at least one output. For accelerations, you probably already have a change output, or at least have at least one other output and can remove the change if necessary.
packages/caravan-fees/src/rbf.ts
Outdated
@@ -143,14 +143,14 @@ export const createCancelRbfTransaction = ( | |||
if ( | |||
newTxTemplate.feeRateSatisfied && | |||
new BigNumber(newTxTemplate.currentFee).gte( | |||
BigNumber.max(newTxTemplate.targetFeesToPay, txAnalyzer.minimumRBFFee), | |||
BigNumber.max(newTxTemplate.targetFeesToPay, txAnalyzer.fee), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This points to some of what I find confusing about the naming. What is the fee
property on the analyzer again? Reading below, maybe it should be originalFee?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So basically this conditional is saying, “if the templates target fee rate is satisfied and absolute fees are greater than the larger of either target fees of template (which in this case is rbf target) OR the original fees of the transaction, then the template is good to generate a psbt from”. What if you just set the target fee to be the max of either the rbf value OR the original? Then you just have to check if fees are satisfied right? Second question: what is the original fees from? Is that from the transaction that’s being bumped? Don’t the RBF rules mean that the rbf target fees ALWAYS have to be larger than the original tx fees?
maybe the comment below shouldn’t say “based on desired rate by user”, since in this case it’s based on rbf calculations right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if you just set the target fee to be the max of either the rbf value OR the original? Then you just have to check if fees are satisfied right?
Actually the newTxTemplate.targetFeesToPay
is dynamic as it is calculated on based on current Template size * targetFeeRate
...
currentFee
is based on the actual inputs and outputs in the transaction, while targetFeesToPay
is calculated based on the target fee rate and estimated transaction size.
so we cannot set it to be max of either the rbf value OR the original
Second question: what is the original fees from? Is that from the transaction that’s being bumped? Don’t the RBF rules mean that the rbf target fees ALWAYS have to be larger than the original tx fees?
Yes it's from the original tx ...
….. REMOVED UNUSED VARIABLES
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Submitting some initial comments. Looking really good though!
}); | ||
|
||
const analysis = analyzer.analyze(); | ||
console.log(analysis); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Include an example output
packages/caravan-fees/README.md
Outdated
The template approach offers several advantages: | ||
|
||
- **Flexibility**: Easily add, remove, or modify inputs and outputs. | ||
- **Incremental Building**: Build transactions step-by-step, adjusting as needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also an important job it fulfills
- **Incremental Building**: Build transactions step-by-step, adjusting as needed. | |
- **Incremental Building**: Build transactions step-by-step, adjusting and validating as needed. |
packages/caravan-fees/README.md
Outdated
|
||
The package ensures that new RBF transactions meet the following criteria: | ||
|
||
- Higher fee rate than the original transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this still true? I would say that it basically lets a consumer of the function(s) define constraints for a fee bump and then ensure that the new transaction passes the BIP125 rules and performs certain sanity checks.
false // fullRBF | ||
); | ||
|
||
console.log("Cancel RBF PSBT:", cancelRbfTx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a PSBT at this point? or some other data structure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PSBT ...
packages/caravan-fees/README.md
Outdated
originalTxHex, | ||
availableUTXOs, | ||
'bc1q...', // cancel address | ||
Network.MAINNET, | ||
'546', // dust threshold | ||
'P2WSH', | ||
2, // required signers | ||
3, // total signers | ||
'1000', // original absolute fee | ||
false // fullRBF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
too complex of a function signature, we should turn this into an options object
packages/caravan-fees/README.md
Outdated
originalTxHex, | ||
availableUTXOs, | ||
1, // spendable output index | ||
'bc1q...', // change address | ||
Network.MAINNET, | ||
'546', // dust threshold | ||
'P2WSH', | ||
'1000', // original absolute fee | ||
2, // required signers | ||
3, // total signers | ||
true // strict mode |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same, very high arity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice readme!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file is looking really really good. Very clean and easy to read. I could see this being a useful standalone set of utilities to expose. Maybe there needs to be a @caravan/transactions
package, or this should be moved to @caravan/bitcoin to start (of course along with the transaction template too).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Absolutely ... From what I can tell is that ... a large chunk of @caravan/bitcoin
code is depreciated or moved to separate packages like the psbt functionality , and the coordinator I guess still uses the old code from @caravan/bitcoin for certain functionalities ... like I remember asking you where was psbtv2 used in the coordinator ... and you said that it still used the older version (I am not pretty sure about this `:| ) ...
Also I feel adding the btcTemplate Component
in the coordinator ... for all transaction creation processes can be a huge boost ... as it has inbuilt methods to output a psbt or be created from psbt ... and other more methods .. for which we currently depend on reducers in the UI
So if possible and you believe it makes rational reason to do so ... can we undertake this project to make all these necessary changes ... after we are finished completing the fees package implementation in the coordinator ???
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I definitely think this can be something we prioritize in the future. I see a lot of potential use cases for this tooling. Great point about use in the coordinator too!
Tbh, I'm not totally sure yet whether or not EVERYTHING in caravan/bitcoin should be deprecated and moved out, or if there is a place for fallback "container" package. But something like this could be in something like a caravan/transactions
package I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking really good! I've reviewed through the components, templates, and cpfp code. Still have more to do but wanted to submit these for now. Most of my comments are basically aesthetic. I'm not sure I'd consider all except maybe one blockers, but mainly just suggestions for aesthetic changes.
return new BtcTransactionTemplate({ | ||
...options, | ||
inputs, | ||
outputs, | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just noting that this is making the opinionated decision that inputs and outputs from the psbt should override those that might be passed in through options. I think this is probably right just wanted to confirm that this is the preference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup it's exactly as you stated
} | ||
|
||
/** | ||
* Gets the malleable outputs of the transaction. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Useful to add a definition of what a malleable output actually is here. E.g.
* Gets the malleable outputs of the transaction. | |
* Gets the malleable outputs of the transaction. Malleable outputs are all those that can have their amount changed, e.g. change change outputs. |
* Calculates the total input amount. | ||
* @returns {Satoshis} The total input amount in satoshis (as a string) | ||
*/ | ||
getTotalInputAmount(): Satoshis { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
optional nit:
getTotalInputAmount(): Satoshis { | |
get totalInputAmount(): Satoshis { |
* Calculates the total output amount. | ||
* @returns {Satoshis} The total output amount in satoshis (as a string) | ||
*/ | ||
getTotalOutputAmount(): Satoshis { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same (optional) nit
getTotalOutputAmount(): Satoshis { | |
get totalOutputAmount(): Satoshis { |
* 2. If removing change, it ensures the fee doesn't unexpectedly increase. | ||
*/ | ||
adjustChangeOutput(): void { | ||
if (this.malleableOutputs.length === 0) return; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this throw an error? If this method is called presumably there's an expectation that change needs to be adjusted. if it just returns (which is the same that happens when an adjustment happens successfully, you could either end up in an infinite loop if you're expecting results to change after calling this or potentially overpaying fees if you just assume that a return without erroring means it worked when in truth there's no adjusted change to account for excess fees.
Perhaps we need to either add a change output here or throw an error. I don't think the template has enough information to do the former so for now throwing an error is probably correct and the consumer can just check before calling it if there's a malleable output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unrelated, you do this check in a couple of places. I wonder if it makes sense to have a hasMalleableOutputs
property that returns this.malleableOutputs.length > 0
// Step 4: Add the spendable output from the parent transaction as an input | ||
const parentOutput = txAnalyzer.outputs[spendableOutputIndex]; | ||
if (!parentOutput) { | ||
throw new Error( | ||
`Spendable output at index ${spendableOutputIndex} not found in the parent transaction.`, | ||
); | ||
} | ||
|
||
const parentOutputUTXO = availableInputs.find( | ||
(utxo) => utxo.txid === analysis.txid && utxo.vout === spendableOutputIndex, | ||
); | ||
|
||
if (!parentOutputUTXO) { | ||
throw new Error( | ||
`UTXO for the spendable output (${analysis.txid}:${spendableOutputIndex}) not found in availableInputs.`, | ||
); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this feels like duplicative work no? If we have the parentOutput then we have the parentOutputUtxo. I'm not sure we need to require that it's in the available inputs since that's implied.
I would simplify this to check
- is there a spendableOutputIndex passed to the function?
- is there an output in the parent transaction that matches?
- If so, then set the parentOutputUtxo to that value (we should have the amount and txid already
* Note: While this approach is simpler than Bitcoin Core's full implementation, | ||
* it achieves the core goal of CPFP: ensuring the combined package (parent + child) | ||
* is attractive for miners to include in a block. | ||
*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fantastic readme 👌
isCPFPFeeSatisfied( | ||
parentFee, | ||
parentVsize, | ||
new BigNumber(childTxTemplate.currentFee), | ||
childTxTemplate.estimatedVsize, | ||
parseFloat(txAnalyzer.cpfpFeeRate), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be cleaner to let the isCPFPFeeSatisfied do its own conversion, then you can just do:
isCPFPFeeSatisfied( | |
parentFee, | |
parentVsize, | |
new BigNumber(childTxTemplate.currentFee), | |
childTxTemplate.estimatedVsize, | |
parseFloat(txAnalyzer.cpfpFeeRate), | |
) | |
isCPFPFeeSatisfied( | |
parentFee, | |
parentVsize, | |
childTxTemplate.currentFee, | |
childTxTemplate.estimatedVsize, | |
txAnalyzer.cpfpFeeRate, | |
) |
I'd also suggest using an options object for the argument to reduce the complexity of the func sig.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, you could simplify this even more... what if the function just took an analyzer (which has all the parent info and the cpfpFeeRate) and the childTxTemplate?
isCPFPFeeSatisfied( | |
parentFee, | |
parentVsize, | |
new BigNumber(childTxTemplate.currentFee), | |
childTxTemplate.estimatedVsize, | |
parseFloat(txAnalyzer.cpfpFeeRate), | |
) | |
isCPFPFeeSatisfied(txAnalyzer, childTxTemplate) |
|
||
if (changeAmount.lte(dustThreshold)) { | ||
if (strict) { | ||
throw new Error( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It concerns me this check is happening here. Shouldn't we be accounting for this when we're adding inputs? Is this just in the case there were not enough inputs?
My guess is probably, but then this code is confusing. The error should probably read that more inputs are needed ("increase inputs" was a little confusing) and then if this is the only scenario where we'd get here, what would adjustChangeOutput
be able do for a CPFP with just one output?
); | ||
} | ||
|
||
// Step 9: Ensure the combined (parent + child) fee rate meets the target |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And this could maybe be _validateCPFPPackage(txAnalyzer, childTxTemplate)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, getting really close! I think there is some cleanup that still needs to be done to the rbf functions primarily, but definitely almost there.
* RBF (Replace-By-Fee) Transaction Creation | ||
* | ||
* SECURITY NOTE: By default, these functions reuse all inputs from the original | ||
* transaction in the replacement transaction while acceleration. This is to prevent the "replacement |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* transaction in the replacement transaction while acceleration. This is to prevent the "replacement | |
* transaction in the replacement transaction for acceleration. This is to prevent the "replacement |
const originalTxFee = new BigNumber(txAnalyzer.fee); | ||
|
||
// Step 2: Verify RBF possibility and suitability | ||
if (!analysis.canRBF) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here. Maybe nicer to do txAnalyzer.canRBF()
?
const newTxTemplate = new BtcTransactionTemplate({ | ||
inputs: [], | ||
outputs: [], | ||
targetFeeRate: Number(txAnalyzer.rbfFeeRate), // Use RBF-specific fee rate from analyzer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this be:
targetFeeRate: Number(txAnalyzer.rbfFeeRate), // Use RBF-specific fee rate from analyzer | |
targetFeeRate: BN.max(txAnalyzer.rbfFeeRate, targetFeeRate).toNumber(), // Use RBF-specific fee rate from analyzer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I mentioned this on the analyzer, but I think you shouldn't be using this property at all. The code for rbfFeeRate
multiplies the minimum rbf fee by the size of the original transaction. This can be correct, but I believe this level of redirection that's happening around the rbf code is confusing for an already very confusing operation.
I think it's fine to set the targetFeeRate for the new tx we're building to be user defined. Whether the user is canceling or accelerating, they know what they're going for. This function should validate that the target rate is higher than the rate of the current transaction at the top, then here just pass in that target rate. Later on, when you're validating, you'll make sure that the absolute fee is higher than the txAnalyzer.minimumRBFFee
* @returns {string} The estimated RBF fee rate in satoshis per vbyte. | ||
* @see https://bitcoinops.org/en/topics/replace-by-fee/ | ||
*/ | ||
get rbfFeeRate(): string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is useful. The rbf fee rate would be based on the size of the new transaction, but in the analyzer this is the old transaction size.
|
||
// Calculate the fee based on the user-specified target fee rate | ||
// This addresses modern miner preferences for higher fee rates | ||
const targetFeeBasedOnUserRate = new BigNumber( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can simplify some of this by just removing this consideration from this part of the code. Keep the analyzer simple. It should only be thinking about values for the transaction being replaced. This property is also only about the absolute fee anyway, so keep it simple and just leave the rate out of it, especially since the rate should only be applied based on the size of the new transaction, which of course we don't know yet.
break; // Stop adding inputs once fee requirements are met | ||
if ( | ||
newTxTemplate.feeRateSatisfied && | ||
new BigNumber(newTxTemplate.currentFee).gte( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, one problem here is that you haven't added anything to the change yet (which is why you could run into the issue below about dusty change). So this check should actually confirm that there is enough left over to have more than dust as change. if you add a property to the analyzer needsChange
that just checks if there is leftover funds greater than the dust threshold, then you can do something like this:
if (
newTxTemplate.feeRateSatisfied &&
new BigNumber(newTxTemplate.currentFee).gte(txAnalyzer.minimumRbfFee) &&
newTxTemplate.needsChange
) {
// Stop adding inputs when the following conditions are met:
// 1. The fee rate satisfies the target rate (which must be gte original tx fee rate
// 2. The current fee is gte the minimum required absolute RBF fee amount
// 3. There is enough left over for change
break;
}
With this the checks on the change aren't necessary and the extra validation afterwards also probably isn't necessary.
Note, this is probably only necessary for cancelation since you need at least one output. For accelerations, you probably already have a change output, or at least have at least one other output and can remove the change if necessary.
* @see https://developer.bitcoin.org/devguide/transactions.html#locktime-and-sequence-number Bitcoin Core's guide on locktime and sequence numbers | ||
* @see https://bitcoinops.org/en/newsletters/2023/10/25/#fn:rbf-warning Bitcoin Optech on replacement cycle attacks | ||
*/ | ||
export const createAcceleratedRbfTransaction = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's see how much code can be shared between these two functions. I think if you try and break out the steps into their own functions, it will make the code much more readable (and easier to test) and you could probably reuse them, maybe with different arguments passed in if some different behavior is necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, could still use some work here but I think it's getting close. I think I understand a little better now the requirements on the rbf side and have a suggestion for how to clarify. Basically if the analyzer no longer thinks about rbf rates, and the rbf functions simply pass in the user defined target fee rate and then validates that value is higher than the original tx rate, that should clarify a lot.
I think you also need to make sure to add enough inputs for the cancelation tx so that the change is not dust. That should allow the change code and the validation at the end to be simplified.
Finally, if you try and modularize some more so that duplicated code can be shared, I think that will make things more readable.
config: { | ||
addressType?: ScriptType; | ||
numInputs?: number; | ||
numOutputs?: number; | ||
m?: number; | ||
n?: number; | ||
} = {}, | ||
): number { | ||
const { | ||
addressType = SCRIPT_TYPES.P2SH, | ||
numInputs = 1, | ||
numOutputs = 1, | ||
m = 1, | ||
n = 2, | ||
} = config; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could all be done in the function signature with destructuring.
Fee Bumping Package
Overview
Comprehensive refactor of the fee bumping package to enhance functionality, improve bitcoinjs-lib integration, and streamline code structure.
Key Improvements
Changes
Notes
Significant changes to fee bumping logic. Careful review needed, especially for transaction analysis and RBF/CPFP implementations.