modifying documents in compiler context

[lrlna]

Context

It's becoming more important for the compiler to be able to modify and work on modified versions of its type system. While compiler's internals should remain immutable it'd be useful to come up with an API to return a modified version of a given document in either a String or perhaps a HIR format that could be accessed programmatically.

Use cases

• creating API Schema (modifying original type system to remove field marked @inaccessible)
• modifying executable documents to remove certain fields
• break down executable documents into several documents

Possible example

Accessing values of API Schema:

    let schema = Path::new("crates/apollo-compiler/examples/documents/type_system.graphql");
    let src = fs::read_to_string(schema).expect("Could not read type system file.");
    compiler.add_type_system(&src, schema);
   
    let query = Path::new("crates/apollo-compiler/examples/documents/get_dog_name.graphql");
    let src = fs::read_to_string(query).expect("Could not read executable file.");
    compiler.add_executable(&src, query);
    
    // modified version of the type system that has `@inaccessible` fields removed
    let api_type_system = compiler.db.api_type_system(); 
    // should probably have the exact same surface level api as the original type system?
    let query_obj_type = api_schema.find_object_type_by_name("Query".into()); 

I am not entirely sure how the internals of the compiler need to change to accommodate this just yet (aside from saying let's not mutate the state!). I think it'd be good to gather a few more examples of how we'd like this to be used before we settle on the architecture and design. If anyone has any specific examples of how they'd like to use modifying documents with the compiler, please bring them forward!

[Geal]

The design of immutable collection libraries could provide some inspiration here. Instead of modifying, it's building a new view that reuses elements of the old one. As.an example, if you have a tree, you can "insert" or "delete" by building a new tree that reuses some sub trees

[lrlna]

As.an example, if you have a tree, you can "insert" or "delete" by building a new tree that reuses some sub trees

iirc that's how rowan deals with reparsing, which i think should tie together very nicely with this API as well

[lrlna]

something i also just thought about is that it's likely that examples such as APISchema creation will actually have to live outside the compiler. Which means we should be providing the semantics and architecture to deal with context modification, but also allow users to specify how they want it done.

^ perhaps was already implicit part of this issue, but better be said out loud

[SimonSapin]

Converting to apollo-encoder types, modifying as needed, then converting back sounds ok (even if the latter involves reparsing). However apollo-encoder would need to gain many more API for mutation. For example ObjectDefinition::field allows adding a field, but there is no way that I can find to remove or modify an existing one. Similarly for every type representing a piece of GraphQL document.

[lrlna]

something to think about here is that this could be quite similar to a macro system/macro rewrite rules.

i'd be keen to gather a few examples of how we could possibly modifying either the type system or executable definitions.

There is currently an example of adding an additional directive @b to all fields that already have directive @a. So this exectuable definition,

query ExampleQuery {
  topProducts {
    name @a
  }
  ... multipleSubscriptions
}

fragment multipleSubscriptions on Subscription {
  newMessage @a {
    body
    sender
  }
}

will become this:

query ExampleQuery {
  topProducts {
    name @a @b
  }
  ... multipleSubscriptions
}

fragment multipleSubscriptions on Subscription {
  newMessage @a @b {
    body
    sender
  }
}

@SimonSapin @Geal do you have other examples of how you'd like to see executable definitions be expanded? I remember there being something about splitting up queries before?

[Geal]

yes, another idea was that when you get

query ExampleQuery {
  topProducts {
    name @a
    reviews {
      body
    }
  }
}

You could get two different queries from there:

query ExampleQuery {
  topProducts {
    reviews {
      body
    }
  }
}

query ExampleQuery {
  topProducts {
    name 
  }
}

That's more or less what is done with @defer right now

[Geal]

in apollographql/router#2489 we introduced query rewrites that change a field name if there can be some incompatibilities due to interfaces (cf apollographql/federation#1257 ) so a subgraph query can be changed to alias a field

[Geal]

could this be considered a particular case of incremental reparsing? Do we get access to rowan's green nodes from the compiler?

[lrlna]

compiler.db.document() gives you the top level green node, so theoretically you do get access to the green node. it is a bit harder to do something like compiler.db.find_operation("getName").lower(), as in, we don't have a way to do that yet. i imagine coming up with a graceful way of handing document modification will require us to figure out a way to lower certain HIR nodes in order to modify them

[lrlna]

another example. given this type system:

directive @a(if: Boolean!) on FIELD | FRAGMENT_SPREAD | INLINE_FRAGMENT
directive @b on OBJECT | FIELD_DEFINITION

type Query {
    topProducts: Product
    customer: User
}

type Product {
    type: String
    price(setPrice: Int): Int
    reviews: [Review] @b
}

type Review {
    body: String
    author: User
}

type User @b {
    id: ID
    name: String
}

with original executable looking like this:

query MyQuery {
    topProducts {
        type
        reviews {
            body
        }
    }

    customer {
        name
    }
}

modified executable should look like this:

query after:
query MyQuery($isB: Boolean) {
  topProducts {
    type
    reviews @a(if: $isB) {
      body
    }
  }
  customer @a(if: $isB) {
    name
  }
}

[zackangelo]

The design of immutable collection libraries could provide some inspiration here. Instead of modifying, it's building a new view that reuses elements of the old one. As.an example, if you have a tree, you can "insert" or "delete" by building a new tree that reuses some sub trees

Any data structures that maintain their history upon mutation are known as "Persistent Data Structures" in FP circles.

[zackangelo]

Converting to apollo-encoder types, modifying as needed, then converting back sounds ok (even if the latter involves reparsing). However apollo-encoder would need to gain many more API for mutation. For example ObjectDefinition::field allows adding a field, but there is no way that I can find to remove or modify an existing one. Similarly for every type representing a piece of GraphQL document.

I'm currently round tripping through the compiler in this way to make modifications to HIR types. I've added my own mutation methods to the encoder types, I'd be happy to open a PR if it'd be useful!

[lrlna]

@zackangelo what sorts of modifications are your currently implementing? is it along the lines of removing/adding fields? or something else entirely?

[zackangelo]

@lrlna currently I'm removing fields and implemented interfaces. I'm also renaming types.

[SimonSapin]

Speaking of renaming, after a first step that’s equivalent removing an item and adding a new one with a different name, it’d be awesome if apollo-rs could provide support for renaming and automatically updating all existing references to that name

[lrlna]

Speaking of renaming, after a first step that’s equivalent removing an item and adding a new one with a different name, it’d be awesome if apollo-rs could provide support for renaming and automatically updating all existing references to that name

heheheh apollo-rs entering its LSP era

[SimonSapin]

I think we can consider this solved with the release of apollo-compiler 1.0 beta: https://docs.rs/apollo-compiler/1.0.0-beta.1/