Trimming metadata file breaks subxt program

[peterwht]

I was attempting to trim the Rococo metadata file by specifying only the pallets that are needed by my subxt script. This is done with a command similar to the following:

subxt metadata -f bytes --file metadata/rococo.scale --pallets Balances,System,Sudo,Utility,XcmPallet -o metadata/trimmed-rococo.scale

However, the subxt program fails to compile when using the trimmed-rococo.scale with the following error:

    |     let sudo_set_balance = runtime::tx().sudo().sudo(set_alice_balance);
    |                                                 ---- ^^^^^^^^^^^^^^^^^ expected `bool`, found `RuntimeCall`
    |                                                 |
    |                                                 arguments to this method are incorrect

To try to debug this, I re-ran the command to specify the pallets, but I specified every pallet from the original metadata (as seen by using subxt explore). Even with every pallet, and every runtime-api from the original metadata file, I still get the same error.

However, when I use the original metadata file the subxt program compiles (and works) successfully.

[niklasad1]

Hey @peterwht

Can you please specify which version of subxt and subxt-cli you are using to get this issue?

We had something similar and it was fixed for a while ago but maybe something else

[jsdw]

Mmm, I'd like to know whether this happens on current master too, and wonder if it has anything to do with the scale-typegen issue that led to a scale-info 2.11.1 release or is something else!

[jsdw]

Just a quick dig into this:

I reproduced this by running the following in the subxt root on current master (or thereabouts):

# download metadata
cargo run --bin subxt metadata --url wss://rococo-rpc.polkadot.io > TEMP_METADATA.scale
# trim it
cargo run --bin subxt metadata --file TEMP_METADATA.scale --pallets Balances,System,Sudo,Utility,XcmPallet -o TRIMMED_METADATA.scale

Then modified a random example (I modified the subxt/examples/tx_basic.rs one) to look like so:

#[subxt::subxt(runtime_metadata_path = "../TRIMMED_METADATA.scale")]
pub mod runtime {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let call = runtime::Call::System(runtime::system::Call::remark { remark: vec![] });
    let sudo_set_balance = runtime::tx().sudo().sudo(call);
    Ok(())
}

Then ran with cargo run --example tx_basic.

This spat out the error above, ie:

error[E0308]: mismatched types
 --> subxt/examples/tx_basic.rs:7:54
  |
7 |     let sudo_set_balance = runtime::tx().sudo().sudo(call);
  |                                                 ---- ^^^^ expected `bool`, found `RuntimeCall`
  |                                                 |
  |                                                 arguments to this method are incorrect

(I attached the metadatas I generated here incase anything changes: Archive.zip

So ok, let's see what the codegen looks like then:

cargo run --bin subxt codegen --file TRIMMED_METADATA.scale| rustfmt > TRIMMED_METADATA.rs

In our generated code, the RuntimeCall looks ok:

pub enum RuntimeCall {
    #[codec(index = 0)]
    System(runtime_types::frame_system::pallet::Call),
    #[codec(index = 4)]
    Balances(runtime_types::pallet_balances::pallet::Call),
    #[codec(index = 24)]
    Utility(runtime_types::pallet_utility::pallet::Call),
    #[codec(index = 99)]
    XcmPallet(runtime_types::pallet_xcm::pallet::Call),
    #[codec(index = 255)]
    Sudo(runtime_types::pallet_sudo::pallet::Call),
}

What does runtime_types::frame_system::pallet::Call look like:

#[doc = "Contains a variant per dispatchable extrinsic that this pallet has."]
pub enum Call {
    #[codec(index = 0)]
    #[doc = "Make some on-chain remark."]
    #[doc = ""]
    #[doc = "Can be executed by every `origin`."]
    remark {
        remark: ::subxt::ext::subxt_core::alloc::vec::Vec<::core::primitive::u8>,
    },
    #[codec(index = 1)]
    #[doc = "Set the number of pages in the WebAssembly environment's heap."]
    set_heap_pages { pages: ::core::primitive::u64 },
    #[codec(index = 2)]
    #[doc = "Set the new runtime code."]
    set_code {
        code: ::subxt::ext::subxt_core::alloc::vec::Vec<::core::primitive::u8>,
    },
    // .....
}

Also OK! So all of the basic codegen is fine.

But wait! What's the type signature of tx().sudo().sudo()?

pub fn sudo(
    &self,
    call: types::sudo::Call,
) -> ::subxt::ext::subxt_core::tx::payload::Payload<types::Sudo> {
    ::subxt::ext::subxt_core::tx::payload::Payload::new_static(
        "Sudo",
        "sudo",
        types::Sudo {
            call: ::subxt::ext::subxt_core::alloc::boxed::Box::new(call),
        },
        [
            213u8, 59u8, 45u8, 101u8, 75u8, 67u8, 81u8, 45u8, 1u8, 227u8, 202u8,
            17u8, 73u8, 206u8, 26u8, 84u8, 136u8, 58u8, 69u8, 3u8, 255u8, 97u8,
            172u8, 202u8, 95u8, 223u8, 62u8, 254u8, 25u8, 37u8, 28u8, 60u8,
        ],
    )
}

Looks ok so far. But what is the type of types::sudo::Call?

pub mod sudo {
    use super::runtime_types;
    pub type Call = ::core::primitive::bool;
}

Ah! It shouldn't be bool, here!

So I think something goes wrong when trimming w.r.t our generated type aliases.

[jsdw]

For my own notes, this is a way of taking some metadata file (here, TEMP_METADATA.scale), trimming it, and testing whether the issue exists in the trimmed result (omit first line to test for issue without trimming):

cargo run --bin subxt metadata --file TEMP_METADATA.scale --pallets Balances,System,Sudo,Utility,XcmPallet -o TRIMMED_METADATA.scale
cargo run --bin subxt metadata --file TRIMMED_METADATA.scale -f json -o TEMP_TEST_METADATA.json
CALLS_TY_ID=$(cat TEMP_TEST_METADATA.json | jq '.[1].V15.pallets[] | select(.name=="Sudo").calls.ty')
ARG_TY_ID=$(cat TEMP_TEST_METADATA.json | jq ".[1].V15.types.types[$CALLS_TY_ID].type.def.variant.variants[] | select(.name==\"sudo_as\").fields[] | select(.name==\"call\").type")
cat TEMP_TEST_METADATA.json | jq ".[1].V15.types.types[$ARG_TY_ID]"

If the result of this is a "bool" type, then things are wrong. if it's a big RuntimeCall variant then things are (hopefully) ok.

This shows that there is an issue with trimmed metadata but not the non-trimmed metadata. I suspect the issue is in the registry.retain() method in scale-info; will dig more!

[peterwht]

Thanks for looking into this @jsdw! Love the commentary of walking through it.

Just for reference, the workaround I used was generating interface.rs files for the non-trimmed version and for the trimmed version. Copy-pasting the pallets I needed from the non-trimmed to the trimmed (just to ensure everything matched up). The interface.rs method is probably better anyway :)

[jsdw]

Thanks! Just as a heads up, I narrowed down the issue and hope to have a fix out for this on the next few days! See paritytech/scale-info#206

[jsdw]

I've just published scale-info v2.11.2, which should fix this issue. Running cargo update should update your lockfile to bring in this dep. I'll close this for now under the assumption that things are fixed, but please re-open if not!