Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XCM Fee Payment Runtime API #3607

Merged

Conversation

PraetorP
Copy link
Contributor

@PraetorP PraetorP commented Mar 7, 2024

The PR provides API for obtaining:

  • the weight required to execute an XCM message,
  • a list of acceptable AssetIds for message execution payment,
  • the cost of the weight in the specified acceptable AssetId.

It is meant to address an issue where one has to guess how much fee to pay for execution. Also, at the moment, a client has to guess which assets are acceptable for fee execution payment.
See the related issue #690.
With this API, a client is supposed to query the list of the supported asset IDs (in the XCM version format the client understands), weigh the XCM program the client wants to execute and convert the weight into one of the acceptable assets. Note that the client is supposed to know what program will be executed on what chains. However, having a small companion JS library for the pallet-xcm and xtokens should be enough to determine what XCM programs will be executed and where (since these pallets compose a known small set of programs).

pub trait XcmPaymentApi<Call>
	where
		Call: Codec,
	{
		/// Returns a list of acceptable payment assets.
		///
		/// # Arguments
		///
		/// * `xcm_version`: Version.
		fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
		/// Returns a weight needed to execute a XCM.
		///
		/// # Arguments
		///
		/// * `message`: `VersionedXcm`.
		fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
		/// Converts a weight into a fee for the specified `AssetId`.
		///
		/// # Arguments
		///
		/// * `weight`: convertible `Weight`.
		/// * `asset`: `VersionedAssetId`.
		fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
		/// Get delivery fees for sending a specific `message` to a `destination`.
		/// These always come in a specific asset, defined by the chain.
		///
		/// # Arguments
		/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
		///   size of the message.
		/// * `destination`: The destination to send the message to. Different destinations may use
		///   different senders that charge different fees.
		fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
	}

An example of a client side code.

@PraetorP PraetorP requested a review from a team as a code owner March 7, 2024 09:51
@mrshiposha
Copy link
Contributor

Hi @franciscoaguirre, it is the PR we spoke of yesterday :)

@franciscoaguirre
Copy link
Contributor

@mrshiposha Awesome! I'd like to add this comment about how we should also have a way of estimating delivery fees.
To better integrate them, I think a single API call should provide given the XCM program, both the execution and delivery fees.
How that looks like depends, right now delivery fees are only paid in a specific asset (at least per router, like a bridge), while execution fees work as you describe, first weight and then acceptable assets to buy that weight.
Probably something like:

/// Given a message, returns the execution `Weight` and the delivery fees needed to be paid.
/// The execution weight can be converted to fees by using `query_acceptable_payment_assets` and `query_weight_to_asset_fee`.
fn query_xcm_fees(message: VersionedXcm<Call>) -> Result<(Weight, Assets), Error>;

Alternatively, we could have it return just Assets but that removes the ability to pay in a different fee.
Unless we return a vector of options, but that might be too much work for a single API call.

We could also keep this call for execution weight and have a separate API call for delivery fees.
The thing is, everyone would need to use both in order to estimate fees as accurately as possible, so we might as well just make them one call.

What do you think?

Also, I'm adding @xlc and @acatangiu to this conversation.

@mrshiposha
Copy link
Contributor

A reply to #3607 (comment).

We haven't considered the delivery fee thing yet, but yeah, it should be covered somehow.
Give us some time to consider it so we can better understand the trade-offs related to the delivery fee.

We opened the PR in its current state to have a basis for further discussion, including the solution for the delivery fee

@acatangiu
Copy link
Contributor

acatangiu commented Mar 7, 2024

We opened the PR in its current state to have a basis for further discussion, including the solution for the delivery fee

This is great!

We haven't considered the delivery fee thing yet, but yeah, it should be covered somehow.
Give us some time to consider it so we can better understand the trade-offs related to the delivery fee.

I think the currently proposed XcmPaymentApi runtime api is good, like Cisco says, should also have a fn to expose the configured delivery fee mechanism (e.g. rococo config). IMO it can simply re-expose fn price_for_delivery() trait implementor.

(do ☝️ in this PR)


(👇 follow-up/different PR(s))

For the full fees story (like proposed in #690 (comment)), and providing users/builders/dapps with a nice UX, we will probably add specialized query_* runtime APIs for pallets like pallet-xcm and xtokens.
E.g. fn query_fees_for_transfer_assets(assets, dest, beneficiary) which will make use of underlying XcmPaymentApi to provide consolidated fees required, with single call.

@franciscoaguirre
Copy link
Contributor

IMO it can simply re-expose fn price_for_delivery() trait implementor.

I think this is a good first thing to add. But I think we could add an API that given a message, it'll give you the delivery fees needed to execute that message. So it dry-runs it to find all messages that need to be sent and to which destinations, and returns the resulting total fees.

@franciscoaguirre
Copy link
Contributor

Clarifying my previous comment. I think we should start simple. There are many complicated things we could do like the thing I described about dry-running, but those would be slow to review and merge. This API looks perfect for what we need right now, which is exposing runtime functions we don't expose currently. After we merge this we can build on top of it.

Basically what Adrian said 😄

@franciscoaguirre franciscoaguirre added the T6-XCM This PR/Issue is related to XCM. label Mar 15, 2024
@franciscoaguirre
Copy link
Contributor

I added the new function exposing delivery fees in UniqueNetwork#3

@franciscoaguirre franciscoaguirre added this pull request to the merge queue Mar 26, 2024
Merged via the queue into paritytech:master with commit 3c972fc Mar 26, 2024
129 of 130 checks passed
@Polkadot-Forum
Copy link

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/parity-tech-update-for-march/7226/1

@Morganamilo Morganamilo mentioned this pull request Apr 4, 2024
12 tasks
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Apr 9, 2024
The PR provides API for obtaining:
- the weight required to execute an XCM message,
- a list of acceptable `AssetId`s for message execution payment,
- the cost of the weight in the specified acceptable `AssetId`.

It is meant to address an issue where one has to guess how much fee to
pay for execution. Also, at the moment, a client has to guess which
assets are acceptable for fee execution payment.
See the related issue
paritytech#690.
With this API, a client is supposed to query the list of the supported
asset IDs (in the XCM version format the client understands), weigh the
XCM program the client wants to execute and convert the weight into one
of the acceptable assets. Note that the client is supposed to know what
program will be executed on what chains. However, having a small
companion JS library for the pallet-xcm and xtokens should be enough to
determine what XCM programs will be executed and where (since these
pallets compose a known small set of programs).
```Rust
pub trait XcmPaymentApi<Call>
	where
		Call: Codec,
	{
		/// Returns a list of acceptable payment assets.
		///
		/// # Arguments
		///
		/// * `xcm_version`: Version.
		fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
		/// Returns a weight needed to execute a XCM.
		///
		/// # Arguments
		///
		/// * `message`: `VersionedXcm`.
		fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
		/// Converts a weight into a fee for the specified `AssetId`.
		///
		/// # Arguments
		///
		/// * `weight`: convertible `Weight`.
		/// * `asset`: `VersionedAssetId`.
		fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
		/// Get delivery fees for sending a specific `message` to a `destination`.
		/// These always come in a specific asset, defined by the chain.
		///
		/// # Arguments
		/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
		///   size of the message.
		/// * `destination`: The destination to send the message to. Different destinations may use
		///   different senders that charge different fees.
		fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
	}
```
An
[example](https://gist.github.com/PraetorP/4bc323ff85401abe253897ba990ec29d)
of a client side code.

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Daniel Shiposha <mrshiposha@gmail.com>
github-merge-queue bot pushed a commit that referenced this pull request May 2, 2024
…i` nits (#4355)

This PR:
- changes `xcm::v4` to `xcm::prelude` imports for coretime stuff
- changes `query_acceptable_payment_assets` /
`query_weight_to_asset_fee` implementations to be more resilient to the
XCM version change
- adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the
AssetHubRococo/Westend exposing a native token as acceptable payment
asset

Continuation of: #3607

Closes: #4297

## Possible follow-ups

- [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as
acceptable payment assets ?
github-merge-queue bot pushed a commit that referenced this pull request May 8, 2024
# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](#3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
paritytech-ci pushed a commit that referenced this pull request May 8, 2024
# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](#3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
github-merge-queue bot pushed a commit that referenced this pull request May 31, 2024
…4634)

Depends on #4621.

Implemented the
[`XcmPaymentApi`](#3607)
and [`DryRunApi`](#3872)
on all system parachains.

More scenarios can be tested on both rococo and westend if all system
parachains implement this APIs.
The objective is for all XCM-enabled runtimes to implement them.
After demonstrating fee estimation in a UI on the testnets, come the
fellowship runtimes.

Step towards #690.
hitchhooker pushed a commit to ibp-network/polkadot-sdk that referenced this pull request Jun 5, 2024
…ytech#3872)

# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](paritytech#3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
hitchhooker pushed a commit to ibp-network/polkadot-sdk that referenced this pull request Jun 5, 2024
…aritytech#4634)

Depends on paritytech#4621.

Implemented the
[`XcmPaymentApi`](paritytech#3607)
and [`DryRunApi`](paritytech#3872)
on all system parachains.

More scenarios can be tested on both rococo and westend if all system
parachains implement this APIs.
The objective is for all XCM-enabled runtimes to implement them.
After demonstrating fee estimation in a UI on the testnets, come the
fellowship runtimes.

Step towards paritytech#690.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T6-XCM This PR/Issue is related to XCM.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants