Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

codegen: Generate type aliases for better API ergonomics #1249

Merged
merged 33 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0be80cc
codegen: Generate type alias for storage return types
lexnv Nov 8, 2023
f522e5a
codegen: Generate type alias for call function arguments
lexnv Nov 8, 2023
d050859
testing: Update polkadot.rs code from commit 2e2a75ff81
lexnv Nov 8, 2023
bd8f602
codegen: Type aliases for runtime API parameters
lexnv Nov 8, 2023
bfa70d1
codegen: Type alias for runtime apis output
lexnv Nov 8, 2023
c5dff9b
storage: Change path of the aliased module
lexnv Nov 8, 2023
c6bb349
codegen: Adjust module indentation
lexnv Nov 9, 2023
10fdc75
codegen: Do not alias for api::runtime_types with unresolved generics
lexnv Nov 9, 2023
8008665
codegen: Fix and document runtime API alias generation
lexnv Nov 9, 2023
2bd89d4
Update artifacts
lexnv Nov 9, 2023
a777163
Update cargo.lock file
lexnv Nov 9, 2023
977b4fb
codegen: Generate composite structs with alias unnamed fields
lexnv Nov 9, 2023
e353f16
testing: Update polkadot.rs file
lexnv Nov 9, 2023
0a316b0
Merge remote-tracking branch 'origin/master' into lexnv/codegen-type-…
lexnv Nov 9, 2023
0798ad9
codegen: Alias storage unnamed parameters
lexnv Nov 9, 2023
b9398dd
Update polkadot.rs
lexnv Nov 9, 2023
8c51c2a
examples: Change polkadot to rococo runtime
lexnv Nov 9, 2023
3ccb99f
codegen: Fix compiling tests in the codegen crate
lexnv Nov 9, 2023
354704e
codegen: Extend storage test with alias module
lexnv Nov 9, 2023
8761283
cli/tests: Adjust exepcted commands to the latest metadata
lexnv Nov 9, 2023
43d9175
Merge remote-tracking branch 'origin/master' into lexnv/codegen-type-…
lexnv Nov 9, 2023
c408e97
Merge remote-tracking branch 'origin/master' into lexnv/codegen-type-…
lexnv Nov 17, 2023
6ec86d9
codegen: Remove missleading comment and docs
lexnv Nov 17, 2023
17bdaeb
codegen: Ensure unique names for generated runtime API types
lexnv Nov 17, 2023
b8425b9
codegen/tests: Test expected runtime type generation
lexnv Nov 17, 2023
1588869
codegen/tests: Check duplicate params in runtime APIs
lexnv Nov 17, 2023
ab105eb
codegen/tests: Test colliding names of type aliases and parameters
lexnv Nov 17, 2023
4bf9ac5
Fix clippy
lexnv Nov 22, 2023
5a9f834
Merge remote-tracking branch 'origin/master' into lexnv/codegen-type-…
lexnv Nov 22, 2023
6142e2f
Merge branch 'master' into lexnv/codegen-type-aliases
lexnv Nov 27, 2023
4352c01
codegen: Separate alias module from struct definition
lexnv Dec 7, 2023
d65be15
Update polkadot.rs
lexnv Dec 7, 2023
4c0c9f3
codegen: Remove outdated docs from composite def
lexnv Dec 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified artifacts/polkadot_metadata_full.scale
Binary file not shown.
Binary file modified artifacts/polkadot_metadata_small.scale
Binary file not shown.
Binary file modified artifacts/polkadot_metadata_tiny.scale
Binary file not shown.
17 changes: 9 additions & 8 deletions cli/src/commands/explore/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ pub mod tests {
async fn test_commands() {
// show pallets:
let output = simulate_run("").await;
assert_eq!(output.unwrap(), "Usage:\n subxt explore <PALLET>\n explore a specific pallet\n\nAvailable <PALLET> values are:\n Balances\n Multisig\n ParaInherent\n Staking\n System\n Timestamp\n");
assert_eq!(output.unwrap(), "Usage:\n subxt explore <PALLET>\n explore a specific pallet\n\nAvailable <PALLET> values are:\n Balances\n Multisig\n ParaInherent\n System\n Timestamp\n");
// if incorrect pallet, error:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to remind myself to make these updates in #1290 later, will probably have merge conflicts.

let output = simulate_run("abc123").await;
assert!(output.is_err());
Expand All @@ -198,19 +198,20 @@ pub mod tests {
let output = simulate_run("Balances abc123").await;
assert!(output.is_err());
// check that we can explore a certain call:
let output = simulate_run("Balances calls transfer").await;
assert!(output.unwrap().starts_with("Usage:\n subxt explore Balances calls transfer <SCALE_VALUE>\n construct the call by providing a valid argument\n\nThe call expect expects a <SCALE_VALUE> with this shape:\n {\n dest: enum MultiAddress"));
let output = simulate_run("Balances calls transfer_allow_death").await;
assert!(output.unwrap().starts_with("Usage:\n subxt explore Balances calls transfer_allow_death <SCALE_VALUE>\n construct the call by providing a valid argument\n\nThe call expect expects a <SCALE_VALUE> with this shape:\n {\n dest: enum MultiAddress"));
// check that unsigned extrinsic can be constructed:
let output =
simulate_run("Balances calls transfer {\"dest\":v\"Raw\"((255,255, 255)),\"value\":0}")
.await;
let output = simulate_run(
"Balances calls transfer_allow_death {\"dest\":v\"Raw\"((255,255, 255)),\"value\":0}",
)
.await;
assert_eq!(
output.unwrap(),
"Encoded call data:\n 0x24040507020cffffff00\n"
"Encoded call data:\n 0x24040400020cffffff00\n"
);
// check that we can explore a certain constant:
let output = simulate_run("Balances constants ExistentialDeposit").await;
assert_eq!(output.unwrap(), "Description:\n The minimum amount required to keep an account open. MUST BE GREATER THAN ZERO!\n\nThe constant has the following shape:\n u128\n\nThe value of the constant is:\n 10000000000\n");
assert_eq!(output.unwrap(), "Description:\n The minimum amount required to keep an account open. MUST BE GREATER THAN ZERO!\n\nThe constant has the following shape:\n u128\n\nThe value of the constant is:\n 33333333\n");
// check that we can explore a certain storage entry:
let output = simulate_run("System storage Account").await;
assert!(output.unwrap().starts_with("Usage:\n subxt explore System storage Account <KEY_VALUE>\n\nDescription:\n The full account information for a particular account ID."));
Expand Down
3 changes: 2 additions & 1 deletion codegen/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,5 @@ getrandom = { workspace = true, optional = true }
[dev-dependencies]
bitvec = { workspace = true }
scale-info = { workspace = true, features = ["bit-vec"] }
pretty_assertions = { workspace = true }
pretty_assertions = { workspace = true }
frame-metadata = { workspace = true }
33 changes: 23 additions & 10 deletions codegen/src/api/calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,34 +32,44 @@ pub fn generate_calls(

let mut struct_defs = super::generate_structs_from_variants(
type_gen,
types_mod_ident,
call_ty,
|name| name.to_upper_camel_case().into(),
"Call",
crate_path,
should_gen_docs,
)?;
let (call_structs, call_fns): (Vec<_>, Vec<_>) = struct_defs

let result = struct_defs
.iter_mut()
.map(|(variant_name, struct_def)| {
let (call_fn_args, call_args): (Vec<_>, Vec<_>) = match struct_def.fields {
.map(|(variant_name, struct_def, aliases)| {
let fn_name = format_ident!("{}", variant_name.to_snake_case());

let result: Vec<_> = match struct_def.fields {
CompositeDefFields::Named(ref named_fields) => named_fields
.iter()
.map(|(name, field)| {
let fn_arg_type = &field.type_path;
let call_arg = if field.is_boxed() {
quote! { #name: ::std::boxed::Box::new(#name) }
} else {
quote! { #name }
};
(quote!( #name: #fn_arg_type ), call_arg)

let alias_name =
format_ident!("{}", name.to_string().to_upper_camel_case());

(quote!( #name: types::#fn_name::#alias_name ), call_arg)
})
.unzip(),
.collect(),
CompositeDefFields::NoFields => Default::default(),
CompositeDefFields::Unnamed(_) => {
return Err(CodegenError::InvalidCallVariant(call_ty))
}
};

let call_fn_args = result.iter().map(|(call_fn_arg, _)| call_fn_arg);
let call_args = result.iter().map(|(_, call_arg)| call_arg);

let pallet_name = pallet.name();
let call_name = &variant_name;
let struct_name = &struct_def.name;
Expand All @@ -69,7 +79,7 @@ pub fn generate_calls(
call_name.to_string(),
));
};
let fn_name = format_ident!("{}", variant_name.to_snake_case());

// Propagate the documentation just to `TransactionApi` methods, while
// draining the documentation of inner call structures.
let docs = should_gen_docs.then_some(struct_def.docs.take()).flatten();
Expand All @@ -78,6 +88,8 @@ pub fn generate_calls(
let call_struct = quote! {
#struct_def

#aliases

impl #crate_path::blocks::StaticExtrinsic for #struct_name {
const PALLET: &'static str = #pallet_name;
const CALL: &'static str = #call_name;
Expand All @@ -101,9 +113,10 @@ pub fn generate_calls(

Ok((call_struct, client_fn))
})
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.unzip();
.collect::<Result<Vec<_>, _>>()?;

let call_structs = result.iter().map(|(call_struct, _)| call_struct);
let call_fns = result.iter().map(|(_, client_fn)| client_fn);

let call_type = type_gen.resolve_type_path(call_ty);
let call_ty = type_gen.resolve_type(call_ty);
Expand Down
27 changes: 16 additions & 11 deletions codegen/src/api/events.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,27 +52,32 @@ pub fn generate_events(

let struct_defs = super::generate_structs_from_variants(
type_gen,
types_mod_ident,
event_ty,
|name| name.into(),
"Event",
crate_path,
should_gen_docs,
)?;

let event_structs = struct_defs.iter().map(|(variant_name, struct_def)| {
let pallet_name = pallet.name();
let event_struct = &struct_def.name;
let event_name = variant_name;
let event_structs = struct_defs
.iter()
.map(|(variant_name, struct_def, aliases)| {
let pallet_name = pallet.name();
let event_struct = &struct_def.name;
let event_name = variant_name;

quote! {
#struct_def
quote! {
#struct_def

impl #crate_path::events::StaticEvent for #event_struct {
const PALLET: &'static str = #pallet_name;
const EVENT: &'static str = #event_name;
#aliases

impl #crate_path::events::StaticEvent for #event_struct {
const PALLET: &'static str = #pallet_name;
const EVENT: &'static str = #event_name;
}
}
}
});
});
let event_type = type_gen.resolve_type_path(event_ty);
let event_ty = type_gen.resolve_type(event_ty);
let docs = &event_ty.docs;
Expand Down
63 changes: 60 additions & 3 deletions codegen/src/api/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -358,12 +358,13 @@ impl RuntimeGenerator {
/// Return a vector of tuples of variant names and corresponding struct definitions.
pub fn generate_structs_from_variants<F>(
type_gen: &TypeGenerator,
types_mod_ident: &syn::Ident,
type_id: u32,
variant_to_struct_name: F,
error_message_type_name: &str,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<Vec<(String, CompositeDef)>, CodegenError>
) -> Result<Vec<(String, CompositeDef, TypeAliases)>, CodegenError>
where
F: Fn(&str) -> std::borrow::Cow<str>,
{
Expand All @@ -386,19 +387,75 @@ where
type_gen,
)?;

let alias_module_name = format_ident!("{}", var.name.to_snake_case());

let docs = should_gen_docs.then_some(&*var.docs).unwrap_or_default();
let struct_def = CompositeDef::struct_def(
&ty,
struct_name.as_ref(),
Default::default(),
fields,
fields.clone(),
Some(parse_quote!(pub)),
type_gen,
docs,
crate_path,
Some(alias_module_name.clone()),
)?;

Ok((var.name.to_string(), struct_def))
let type_aliases = TypeAliases::new(fields, types_mod_ident.clone(), alias_module_name);

Ok((var.name.to_string(), struct_def, type_aliases))
})
.collect()
}

/// Generate the type aliases from a set of enum / struct definitions.
///
/// The type aliases are used to make the generated code more readable.
#[derive(Debug)]
pub struct TypeAliases {
fields: CompositeDefFields,
types_mod_ident: syn::Ident,
mod_name: syn::Ident,
}

impl TypeAliases {
pub fn new(
fields: CompositeDefFields,
types_mod_ident: syn::Ident,
mod_name: syn::Ident,
) -> Self {
TypeAliases {
fields,
types_mod_ident,
mod_name,
}
}
}

impl quote::ToTokens for TypeAliases {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
let has_fields = matches!(&self.fields, CompositeDefFields::Named(fields) if !fields.is_empty())
|| matches!(&self.fields, CompositeDefFields::Unnamed(fields) if !fields.is_empty());
if !has_fields {
return;
}

let visibility: syn::Visibility = parse_quote!(pub);

let aliases = self
.fields
.to_type_aliases_tokens(Some(visibility).as_ref());

let mod_name = &self.mod_name;
let types_mod_ident = &self.types_mod_ident;

tokens.extend(quote! {
pub mod #mod_name {
use super::#types_mod_ident;

#aliases
}
})
}
}
Loading
Loading