Schematize lambda.CallbackFunction overlay and add docs for it #4352

flostadler · 2024-08-07T18:07:10Z

The CallbackFunction resource overlay was not present in the registry before. This change adds the schema for it and adds the docs for the registry.
The schema is mostly based on the lambda.Function resource with modifications to adhere to the typescript class here:

pulumi-aws/sdk/nodejs/lambda/lambdaMixins.ts

Line 245 in 7461d68

export interface CallbackFunctionArgs<E, R> extends BaseCallbackFunctionArgs {

How to review?

Check out this PR locally
Check out https://github.com/pulumi/registry and run the following in the root of that repository:
- Follow the setup steps here: https://github.com/pulumi/registry?tab=readme-ov-file#installing-dependencies
- go install github.com/pulumi/registry/tools/resourcedocsgen@master
- resourcedocsgen docs --schemaFile ../pulumi-aws/provider/cmd/pulumi-resource-aws/schema.json --docsOutDir content/registry/packages/aws/api-docs --packageTreeJSONOutDir static/registry/packages/navs/aws.json --version 6.0.0-alpha.0+dev (you might have to tweak the path to the schema.json to match your dir layout)
- make serve
Visit http://localhost:1313/registry/packages/aws/api-docs/lambda/callbackfunction

Fixes #4351

The CallbackFunction resource overlay was not present in the registry before. This change adds the schema for it and adds the docs for the registry.

flostadler · 2024-08-07T18:08:30Z

provider/types.go

+)
+
+// extraTypes augment the types defined by the upstream AWS provider
+var extraTypes = map[string]schema.ComplexTypeSpec{


I moved this to its own file to make it easier to work with and add additional extraTypes sources (like the overlays)

Thank you so much, I dread working in the super long resources.go.

flostadler · 2024-08-07T18:09:57Z

provider/overlays.go

+			TypeSpec: schema.TypeSpec{
+				Ref: "#/types/aws:lambda/Runtime:Runtime",
+			},
+			Description: "The Lambda runtime to use. If not provided, will default to `NodeJS20dX`.",


This is not true yet, but we're currently updating it here: #4348

This PR will only go in after that one

github-actions · 2024-08-07T18:23:10Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

lambda/callbackFunction.CallbackFunction

Maintainer note: consult the runbook for dealing with any breaking changes.

t0yv0 · 2024-08-07T20:54:21Z

provider/overlays.go

+// typeOverlays augment the types defined by the upstream AWS provider
+var typeOverlays = map[string]schema.ComplexTypeSpec{}
+
+//go:embed pkg/overlays/examples/callbackFunction.md


This is a moot point for just one file but if we're doing it a lot we can avoid shipping this into pulumi-resource-aws binary to avoid bulking it up. Trouble with resources.go code is that it's evaluated twice, once for tfgen and once for runtime startup in the provider, so it's a little tricky, but can be done with os.ReadFile that only runs for make tfgen case.

IMO we should opt for the simpler option here to avoid bugs and trouble down the road. Even if we had multiple files it wouldn't be a lot of storage size we're adding. The markdown file for the callbackFunction has ~9KB, so we'd need a lot of them before this has meaningful impact on the overall size of the provider (~794MB right now).

You are right, I concede the point.. The provider binary is too large but we need to look holistically as to what to do with that, this is not really the root problem here.

t0yv0 · 2024-08-07T20:56:58Z

provider/pkg/overlays/examples/callbackFunction.md

+You can provide the JavaScript function used for the Lambda Function's Handler either directly by setting the `callback` input property or instead specify the `callbackFactory`, which is a Javascript function that will be called to produce the callback function that is the entrypoint for the AWS Lambda.
+Using `callbackFactory` is useful when there is expensive initialization work that should only be executed once. The factory-function will be invoked once when the final AWS Lambda module is loaded. It can run whatever code it needs, and will end by returning the actual function that Lambda will call into each time the Lambda is invoked.
+
+The callback function should follow the following signature: `(event: E, context: Context, callback: (error?: Error | string | null, result?: R) => void) => Promise<R> | void`. It is recommended to use an async function, otherwise the Lambda execution will run until the `callback` parameter is called and the event loop is empty. See [Define Lambda function handler in Node.js](https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html) for additional details.


Consider avoiding inline code gory details in favor of a code example (or just entirely drop).

t0yv0 · 2024-08-07T20:57:16Z

provider/pkg/overlays/examples/callbackFunction.md

+
+See [Function Serialization](https://www.pulumi.com/docs/concepts/inputs-outputs/function-serialization/) for additional details on this process.
+
+If no IAM Role is specified, CallbackFunction will automatically use the following managed policies:


Is this important information to have upfront, or perhaps reference section at the bottom?

t0yv0 · 2024-08-07T20:57:30Z

provider/pkg/overlays/examples/callbackFunction.md

+
+Notice the path to the file is now `/opt/config/config.json` --- `/opt` being the path at which AWS Lambda extracts the contents of a layer. The configuration layer is now manageable and deployable independently of the Lambda itself, allowing changes to be applied immediately across all functions that use it.
+
+#### Using layers for Node.js dependencies


TIL, very nice.

t0yv0

Very nice.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [@pulumi/aws](https://pulumi.io) ([source](https://togithub.com/pulumi/pulumi-aws)) | dependencies | minor | [`6.48.0` -> `6.49.0`](https://renovatebot.com/diffs/npm/@pulumi%2faws/6.48.0/6.49.0) | --- ### Release Notes <details> <summary>pulumi/pulumi-aws (@pulumi/aws)</summary> ### [`v6.49.0`](https://togithub.com/pulumi/pulumi-aws/releases/tag/v6.49.0) [Compare Source](https://togithub.com/pulumi/pulumi-aws/compare/v6.48.0...v6.49.0) ##### Does the PR have any schema changes? Looking good! No breaking changes found. ##### New resources: - `ebs/snapshotBlockPublicAccess.SnapshotBlockPublicAccess` - `lambda/callbackFunction.CallbackFunction` - `rds/integration.Integration` ##### New functions: - `rds/getClusterParameterGroup.getClusterParameterGroup` - `secretsmanager/getSecretVersions.getSecretVersions` ##### What's Changed - Update GitHub Actions workflows. by [@pulumi-bot](https://togithub.com/pulumi-bot) in [pulumi/pulumi-aws#4353 - Upgrade CallbackFunction default runtime to Node 20.x by [@t0yv0](https://togithub.com/t0yv0) in [pulumi/pulumi-aws#4348 - Schematize lambda.CallbackFunction overlay and add docs for it by [@flostadler](https://togithub.com/flostadler) in [pulumi/pulumi-aws#4352 - Upgrade terraform-provider-aws to v5.62.0 by [@flostadler](https://togithub.com/flostadler) in [pulumi/pulumi-aws#4359 **Full Changelog**: pulumi/pulumi-aws@v6.48.0...v6.49.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: lumiere-bot[bot] <98047013+lumiere-bot[bot]@users.noreply.github.com>

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [@pulumi/aws](https://pulumi.io) ([source](https://togithub.com/pulumi/pulumi-aws)) | dependencies | minor | [`6.48.0` -> `6.49.0`](https://renovatebot.com/diffs/npm/@pulumi%2faws/6.48.0/6.49.0) | --- ### Release Notes <details> <summary>pulumi/pulumi-aws (@pulumi/aws)</summary> ### [`v6.49.0`](https://togithub.com/pulumi/pulumi-aws/releases/tag/v6.49.0) [Compare Source](https://togithub.com/pulumi/pulumi-aws/compare/v6.48.0...v6.49.0) ##### Does the PR have any schema changes? Looking good! No breaking changes found. ##### New resources: - `ebs/snapshotBlockPublicAccess.SnapshotBlockPublicAccess` - `lambda/callbackFunction.CallbackFunction` - `rds/integration.Integration` ##### New functions: - `rds/getClusterParameterGroup.getClusterParameterGroup` - `secretsmanager/getSecretVersions.getSecretVersions` ##### What's Changed - Update GitHub Actions workflows. by [@pulumi-bot](https://togithub.com/pulumi-bot) in [pulumi/pulumi-aws#4353 - Upgrade CallbackFunction default runtime to Node 20.x by [@t0yv0](https://togithub.com/t0yv0) in [pulumi/pulumi-aws#4348 - Schematize lambda.CallbackFunction overlay and add docs for it by [@flostadler](https://togithub.com/flostadler) in [pulumi/pulumi-aws#4352 - Upgrade terraform-provider-aws to v5.62.0 by [@flostadler](https://togithub.com/flostadler) in [pulumi/pulumi-aws#4359 **Full Changelog**: pulumi/pulumi-aws@v6.48.0...v6.49.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: lumiere-bot[bot] <98047013+lumiere-bot[bot]@users.noreply.github.com>

mikhailshilkov · 2024-08-19T12:45:18Z

This PR has been shipped in release v6.49.0.

flostadler added 2 commits August 7, 2024 19:49

Schematize lambda.CallbackFunction overlay and add docs for it

9722b80

The CallbackFunction resource overlay was not present in the registry before. This change adds the schema for it and adds the docs for the registry.

Additional method doc

86b1389

flostadler requested review from t0yv0, corymhall and a team August 7, 2024 18:07

flostadler self-assigned this Aug 7, 2024

flostadler commented Aug 7, 2024

View reviewed changes

t0yv0 reviewed Aug 7, 2024

View reviewed changes

t0yv0 approved these changes Aug 7, 2024

View reviewed changes

flostadler added 3 commits August 8, 2024 17:57

Addressed PR comments

a5e56a4

Merge branch 'master' into flostadler/callback-function-overlay-docs

fec9ab4

Regen schema

5dcf917

flostadler requested a review from t0yv0 August 8, 2024 16:45

t0yv0 approved these changes Aug 8, 2024

View reviewed changes

flostadler enabled auto-merge (squash) August 8, 2024 17:52

flostadler merged commit 8f1755f into master Aug 8, 2024
30 checks passed

flostadler deleted the flostadler/callback-function-overlay-docs branch August 8, 2024 17:54

mjeffryes added this to the 0.108 milestone Aug 16, 2024

mikhailshilkov mentioned this pull request Aug 19, 2024

Document CallbackFunction resource overlay #4351

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Schematize lambda.CallbackFunction overlay and add docs for it #4352

Schematize lambda.CallbackFunction overlay and add docs for it #4352

flostadler commented Aug 7, 2024

flostadler Aug 7, 2024

t0yv0 Aug 7, 2024

flostadler Aug 7, 2024 •

edited

Loading

github-actions bot commented Aug 7, 2024

t0yv0 Aug 7, 2024

flostadler Aug 8, 2024

t0yv0 Aug 8, 2024

t0yv0 Aug 7, 2024

t0yv0 Aug 7, 2024

t0yv0 Aug 7, 2024

t0yv0 left a comment

mikhailshilkov commented Aug 19, 2024


		See [Function Serialization](https://www.pulumi.com/docs/concepts/inputs-outputs/function-serialization/) for additional details on this process.

		If no IAM Role is specified, CallbackFunction will automatically use the following managed policies:


		Notice the path to the file is now `/opt/config/config.json` --- `/opt` being the path at which AWS Lambda extracts the contents of a layer. The configuration layer is now manageable and deployable independently of the Lambda itself, allowing changes to be applied immediately across all functions that use it.

		#### Using layers for Node.js dependencies

Schematize lambda.CallbackFunction overlay and add docs for it #4352

Schematize lambda.CallbackFunction overlay and add docs for it #4352

Conversation

flostadler commented Aug 7, 2024

How to review?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flostadler Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Aug 7, 2024

Does the PR have any schema changes?

New resources:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

t0yv0 left a comment

Choose a reason for hiding this comment

mikhailshilkov commented Aug 19, 2024

flostadler Aug 7, 2024 •

edited

Loading