Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CloudStorageEmulatorContainer and PubSubEmulatorContainer to gcloud module #805

Merged
merged 18 commits into from
Aug 22, 2024

Conversation

weyert
Copy link
Contributor

@weyert weyert commented Jul 28, 2024

Adds two new emulator containers to the gcloud-package:

  • CloudStorageEmulatorContainer: Cloud Storage
  • PubSubEmulatorContainer: Cloud PubSub

The Cloud PubSub emalator container uses the same approach as used by the existing containers of using the gcloud cli tos tart the PubSub emulator.

The Cloud Storage emulator container uses an existing fake GCS emulator named fsouza/fake-gcs-server.

Adds two new emulator containers to the `gcloud`-package:

 - `CloudStorageEmulatorContainer`: Cloud Storage
 - `PubSubEmulatorContainer`: Cloud PubSub

The Cloud PubSub emalator container uses the same approach as used by the
existing containers of using the `gcloud cli` tos tart the PubSub emulator.

The Cloud Storage emulator container uses an existing fake GCS emulator
named `fsouza/fake-gcs-server`.
Copy link

netlify bot commented Jul 28, 2024

Deploy Preview for testcontainers-node ready!

Name Link
🔨 Latest commit e102113
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-node/deploys/66bf53ecc79c6d00083c0e28
😎 Deploy Preview https://deploy-preview-805--testcontainers-node.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@cristianrgreco cristianrgreco added enhancement New feature or request minor Backward compatible functionality labels Jul 28, 2024
@cristianrgreco
Copy link
Collaborator

cristianrgreco commented Jul 29, 2024

Thanks @weyert, could you review the lint issues? Should be a case of running npm run format. It should have run as part of the git commit hooks

@cristianrgreco cristianrgreco changed the title feat: emulators for Google Cloud Storage and PubSub Add CloudStorageEmulatorContainer and PubSubEmulatorContainer to gcloud module Jul 29, 2024
@cristianrgreco
Copy link
Collaborator

cristianrgreco commented Jul 29, 2024

@weyert Looks like some tests are failing

FAIL packages/modules/gcloud/src/cloudstorage-emulator-container.test.ts (8.023 s)
  ● CloudStorageEmulatorContainer › should work using default version

    Unable to detect a Project Id in the current environment. 
    To learn more about authentication and Google APIs, visit: 
    https://cloud.google.com/docs/authentication/getting-started

      at GoogleAuth.findAndCacheProjectId (../../../node_modules/google-auth-library/build/src/auth/googleauth.js:147:19)
      at async setProjectId (../../../node_modules/@google-cloud/storage/build/cjs/src/nodejs-common/util.js:401:29)
      at async onAuthenticated (../../../node_modules/@google-cloud/storage/build/cjs/src/nodejs-common/util.js:424:33)
FAIL packages/modules/gcloud/src/pubsub-emulator-container.test.ts (240.724 s)
  ● PubSubEmulatorContainer › should work using default version

    Sorry, we cannot connect to Cloud Services without a project
     ID. You may specify one with an environment variable named
     "GOOGLE_CLOUD_PROJECT".

      at replaceProjectIdToken (../../../node_modules/@google-cloud/projectify/build/src/index.js:47:19)
      at replaceProjectIdToken (../../../node_modules/@google-cloud/projectify/build/src/index.js:40:30)
      at ../../../node_modules/@google-cloud/pubsub/src/pubsub.ts:1356:38
      at ../../../node_modules/@google-cloud/pubsub/src/pubsub.ts:1278:17

@cristianrgreco
Copy link
Collaborator

@weyert Will you look into why the tests are failing? Merging main won't fix them. Same for the with platform PR

@weyert
Copy link
Contributor Author

weyert commented Aug 10, 2024

@cristianrgreco I have made some changes this week to make it pass but the workflows aren't running here so I made a second PR in my own repo and there the tests succeeded: https://github.com/weyert/testcontainers-node/actions/runs/10334199546/job/28607861739

Only problem is that the Selenium tests are failing.

@weyert
Copy link
Contributor Author

weyert commented Aug 12, 2024

@cristianrgreco Anything else needed to get this merged?

@cristianrgreco
Copy link
Collaborator

Hi @weyert, I'll run the build now. Could you add a few lines in the docs as done here? https://github.com/testcontainers/testcontainers-node/blob/693ea8c26b62975de581f8a38f0ce39d214d31f1/docs/modules/gcloud.md#firestore-datastore-mode

@weyert
Copy link
Contributor Author

weyert commented Aug 13, 2024

@cristianrgreco I updated the documentation to add both mention the new pubsub and storage to the cgloud.md documentation file

@cristianrgreco cristianrgreco merged commit ce6a032 into testcontainers:main Aug 22, 2024
94 checks passed
@alecmev
Copy link
Contributor

alecmev commented Aug 25, 2024

Thanks for implementing this! Would have saved me some time back in May 😛 Have you considered adding automatic this.updateExternalUrl(this.getEmulatorEndpoint()) upon launching the container, if withExternalUrl isn't used? I feel like there's no downside (but could add an opt-out, just in case).

@alecmev
Copy link
Contributor

alecmev commented Aug 25, 2024

Also, I think the wait strategy needs to be updated to include Wait.forListeningPorts(). Otherwise calling updateExternalUrl right after starting a container often results in ECONNREFUSED.

Disregard, I forgot that this is a Colima quirk.

Let me know if you'd like to add these improvements yourself, or prefer that I do it.

@weyert
Copy link
Contributor Author

weyert commented Aug 25, 2024

Thanks for implementing this! Would have saved me some time back in May 😛 Have you considered adding automatic this.updateExternalUrl(this.getEmulatorEndpoint()) upon launching the container, if withExternalUrl isn't used? I feel like there's no downside (but could add an opt-out, just in case).

I could saved myself time by finding your example for fake-gcs-server 🤪 Guess, we could do that. Feel free to raise a PR for it :)) I can make the changes in a few weeks time.

I mainly wrote these two containers as I want to test the integrations of two packages at work to ensure the behaviour is the same when using PubSub or Kafka, and between S3 and Cloud Storage :)

@alecmev
Copy link
Contributor

alecmev commented Aug 27, 2024

Done 😉 #825

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request minor Backward compatible functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants