Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rules js #2091

Merged
merged 108 commits into from
May 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
eb37afd
Simplify and abstract build files.
Zemnmez Feb 3, 2023
6523d5a
Simplify and abstract build files.
Zemnmez Feb 3, 2023
21ef4df
Merge branch 'simplify-and-abstract-build-files' into rules_js
Zemnmez Feb 3, 2023
0ee1d65
Use rules_js and rules_ts.
Zemnmez Feb 3, 2023
04e6fc7
Merge branch 'simplify-and-abstract-build-files' into rules_js
Zemnmez Feb 4, 2023
2b22faa
more progress
Zemnmez Feb 4, 2023
b82b2d6
Use eslint binary compatible with rules_js
Zemnmez Feb 5, 2023
b283bb5
Accommodate rules_js by changing build tag
Zemnmez Feb 5, 2023
73abf6d
continue work
Zemnmez Feb 6, 2023
62c614d
migrate schemastore
Zemnmez Feb 6, 2023
b13abb0
Continue progress on migration to rules_js
Zemnmez Feb 6, 2023
4d8a52c
Automagically insert colons into @npm references.
Zemnmez Feb 6, 2023
b9bf7bc
Remove vestigial 'lock minimization' code.
Zemnmez Feb 6, 2023
76cab20
Docs says to do this:
Zemnmez Feb 6, 2023
1c2dd0c
more progress
Zemnmez Feb 7, 2023
323aff1
use explicit npmrc
Zemnmez Feb 8, 2023
3546604
rewrite npm imports to use rules_js style node_modules
Zemnmez Feb 9, 2023
0bc7497
//ts/iter builds :)
Zemnmez Feb 9, 2023
9bee265
Package_json fixes
Zemnmez Feb 9, 2023
f4b1d83
rewrite absolute imports to rules_js format
Zemnmez Feb 9, 2023
0333af0
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Feb 9, 2023
c9b730a
pass binary to build rule
Zemnmez Feb 12, 2023
2d999eb
open up visibility of api extractor binary
Zemnmez Feb 12, 2023
371e27a
add BAZEL_BINDIR to js build action env
Zemnmez Feb 12, 2023
0348380
api extractor works :)
Zemnmez Feb 12, 2023
ab2fe0b
some progress
Zemnmez Feb 12, 2023
73b8829
use correct binary for api-documenter
Zemnmez Feb 12, 2023
c190052
:(
Zemnmez Feb 12, 2023
abef061
Yet again abandomn usage of MODULE.bazel.
Zemnmez Feb 13, 2023
e703a65
Add aspect_rules_swc
Zemnmez Feb 13, 2023
8b556f9
Correct visibility of api-documenter binary
Zemnmez Feb 13, 2023
7aaf646
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Feb 14, 2023
f563a80
make npmrc explicit
Zemnmez Feb 20, 2023
5c0bace
Remove CSS module import tests.
Zemnmez Feb 20, 2023
a93b507
Use next_binary instead of next.
Zemnmez Feb 20, 2023
a0a8ce2
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Feb 20, 2023
dbd8378
Progress on making next.js work.
Zemnmez Feb 20, 2023
35b9a2b
Next.js works! (but the typings fail)
Zemnmez Feb 24, 2023
93139b8
Make next.js work :)
Zemnmez Feb 24, 2023
788e97e
Get auto fixers to start working and do some fixes.
Zemnmez Feb 24, 2023
448b9ff
Make automatic fixers work again and apply automatic fixes.
Zemnmez Feb 24, 2023
d995bbb
ts/next.js tests all green :)
Zemnmez Feb 24, 2023
7c38cd6
Make inkscape tests compatible with rules_js
Zemnmez Feb 24, 2023
4dbfa7b
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Feb 24, 2023
ea99038
Demonstrate issue in api-extractor by using more complex import.
Zemnmez Feb 24, 2023
8e1d244
Instructions for fixing api-extractor
Zemnmez Feb 25, 2023
dcd202a
Attempt at fixing api-extractor under rules_js.
Zemnmez Mar 4, 2023
00d3770
move api extractor config generation into typescript
Zemnmez Mar 8, 2023
93999d1
force running js binaries to respect the absolute node import paths used
Zemnmez Mar 8, 2023
fe42d3a
mostly get api extractor to work. it runs fine but is now missing som…
Zemnmez Mar 8, 2023
eb4346b
typecheck and formatting fixes for the api extractor
Zemnmez Mar 8, 2023
4b772ce
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Mar 16, 2023
eb3e119
make compatible with typescript@latest
Zemnmez Mar 17, 2023
10d2cec
make microsoft_json_schemas over http only
Zemnmez Mar 17, 2023
ac42087
allow api-extractor to run by tricking it into running in local mode …
Zemnmez Mar 17, 2023
74f51b0
partial fix to api-extractor
Zemnmez Mar 17, 2023
c1635df
fix some things and break some more things
Zemnmez Mar 17, 2023
7b40c1f
remove empty test file
Zemnmez Mar 17, 2023
709477c
specify BAZEL_BINDIR for cultist/gen
Zemnmez Mar 17, 2023
0454305
fix test //ts/fs:tests
Zemnmez Mar 17, 2023
dee5bf3
ensure pnpm lockfile up to date
Zemnmez Mar 17, 2023
fb6394f
attempt at fixing cultist gen ... it's having trouble locating some f…
Zemnmez Mar 17, 2023
3690d90
this should have already been included
Zemnmez Mar 17, 2023
734cf22
force csstype to appear as a dep. Always install pnpm peer deps.
Zemnmez Mar 17, 2023
a7b170f
allow svgshot types to build
Zemnmez Mar 17, 2023
77769a5
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez Mar 18, 2023
c8f1dac
demonstrate that there are still issues with api-extractor
Zemnmez Mar 24, 2023
5a72784
expose .d.ts files to api-extractor!!!!
Zemnmez Mar 25, 2023
73d9ca0
make npm_pkg work
Zemnmez Mar 25, 2023
5aa3530
fix core locations
Zemnmez Apr 4, 2023
f8d6ecc
Merge branch 'multiplatform-pulumi' into rules_js
Zemnmez Apr 4, 2023
ec3f8c3
update typescript hash
Zemnmez Apr 4, 2023
74a8099
add hash of microsoft_json_schemas
Zemnmez Apr 4, 2023
7e11cb7
use rules_js style import paths
Zemnmez Apr 4, 2023
f494141
allow do-sync tests to at least run
Zemnmez Apr 4, 2023
bed38b4
lint fixes
Zemnmez Apr 4, 2023
fb11828
this isn't needed anymore
Zemnmez Apr 4, 2023
daad892
fix api-documenter
Zemnmez Apr 4, 2023
82111dc
(theoretically) fix npm publish
Zemnmez Apr 4, 2023
731df41
introduce a new copy_to_local rule specifically to allow old cultist …
Zemnmez May 4, 2023
4ffc3e6
Merge branch 'allow-projects-to-compile' into rules_js
Zemnmez May 4, 2023
27c5859
remove reference to vestigial file jsconfig.json
Zemnmez May 4, 2023
c0148a8
fix svgo issue https://github.com/svg/svgo/issues/1766
Zemnmez May 5, 2023
b83b327
allow absolute imports from js_test
Zemnmez May 5, 2023
6ada0a3
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez May 5, 2023
7ee0125
fix missing node types in ts/pulumi/me/...
Zemnmez May 7, 2023
e59b534
The fact that this change, which is not a type error for pulumi fixes…
Zemnmez May 8, 2023
d609fdf
add node runtime to the tshadwell zone file
Zemnmez May 8, 2023
aae25f0
fix npm publish's cache complaints
Zemnmez May 8, 2023
4e6f29a
remove vestigial file
Zemnmez May 8, 2023
6711b83
add pulumi types to ts/pulumi/im/shadwell/cert
Zemnmez May 8, 2023
12ff851
get specific version of json-schemas
Zemnmez May 8, 2023
cfce011
Alter patch to work more portably (i have no fucking clue
Zemnmez May 8, 2023
398637a
fix api-documenter
Zemnmez May 8, 2023
464fcff
fix typescript hash (it keeps flip-flopping and I have no idea why)
Zemnmez May 8, 2023
d9a9509
Simplify pulumi logic, add .gitignore to ignore yarn.lock
Zemnmez May 22, 2023
3f441a6
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez May 22, 2023
b6fa69b
update rules_ts
Zemnmez May 22, 2023
fe7aa97
fix svgshot for rules_js and automatic fixes
Zemnmez May 22, 2023
e1e4f48
update README.md
Zemnmez May 22, 2023
8a9902c
add types for node so the inkscape runner passes typecheck
Zemnmez May 22, 2023
84e08df
Rejoice!!! SVGSHOT TESTS FINALLY WORK!!!!!!!!
Zemnmez May 23, 2023
d029956
automatic linting and fixes:
Zemnmez May 23, 2023
217f863
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez May 23, 2023
31a2910
fix future version pnpm lock
Zemnmez May 23, 2023
b115656
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez May 23, 2023
4709445
Merge remote-tracking branch 'origin/main' into rules_js
Zemnmez May 23, 2023
16bc082
missed this
Zemnmez May 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,4 @@ build --action_env=AWS_ACCESS_KEY_ID --action_env=AWS_SECRET_ACCESS_KEY --action
test --action_env=PULUMI_ACCESS_TOKEN

# be careful with this!
test --test_tag_filters=-do_not_run_on_main
test --test_tag_filters=-do_not_run_on_main
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,7 @@ yarn-error.log


# generated by yarn run bazel //rs:gen_rust_project
rust-project.json
rust-project.json

# ignore yarn since we use pnpm now
yarn.lock
2 changes: 2 additions & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
hoist=false
auto-install-peers=true
58 changes: 42 additions & 16 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,28 @@

load("//js:rules.bzl", "copy_to_bin", "js_library")
load("@com_github_bazelbuild_buildtools//buildifier:def.bzl", "buildifier")
load("@npm//@bazel/esbuild:esbuild_config.bzl", "esbuild_config")
load("//ts:rules.bzl", "ts_config")
load("@npm//renovate:index.bzl", "renovate_config_validator_test")
load("@npm//:renovate/package_json.bzl", "bin")
load("@npm//:defs.bzl", "npm_link_all_packages")

package(default_visibility = [":__subpackages__"])

buildifier(
name = "buildifier",
)

# Useful commands to run!
alias(
name = "fix",
actual = "//bzl:fix",
)

alias(
name = "pnpm",
# bazel run -- //:pnpm --dir $PWD
actual = "@pnpm//:pnpm",
)

buildifier(
name = "buildifier",
)

ts_config(
name = "tsconfig",
src = "tsconfig.json",
Expand All @@ -41,27 +48,46 @@ exports_files(
".editorconfig",
".eslintrc.json",
"css_loader.d.ts",
"yarn.lock",
".git",
"jsconfig.json",
],
visibility = ["//visibility:public"],
)

esbuild_config(
name = "esbuild_config",
config_file = "esbuild.config.mjs",
deps = [
"@npm//esbuild-css-modules-plugin",
],
)

copy_to_bin(
name = "swcrc",
srcs = [".swcrc.json"],
)

renovate_config_validator_test(
copy_to_bin(
name = "prettierrc",
srcs = [".prettierrc.json"],
)

copy_to_bin(
name = "eslintrc",
srcs = [".eslintrc.json"],
)

copy_to_bin(
name = "gitignore",
srcs = [".gitignore"],
)

copy_to_bin(
name = "editorconfig",
srcs = [".editorconfig"],
)

copy_to_bin(
name = "package_json",
srcs = ["package.json"],
)

bin.renovate_config_validator_test(
name = "validate_renovate_config_test",
args = ["$(location renovate.json)"],
data = ["renovate.json"],
)

npm_link_all_packages(name = "node_modules")
21 changes: 14 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,22 @@
# Monorepo

This is a bazel monorepo that I'm hoping eventually will contain most of my side-projects.
This is a bazel monorepo containing most of my side-projects and websites etc. Using a monorepo in this way makes it much easier to ensure linting and deployment are consistent across many projects, and prevents a lot of version rot.

Via the github actions workflows in [.github/workflows/ci.yml], changes in any dependencies are automatically tested and merged.
## Usage

Then, a secondary action runs which increments any version numbers necessary and commits to the [versioned branch]. This branch tracks and increments versions for export to third-party package managers like NPM.
The repo is going through a bit of an identity crisis at the moment. I have just completed migration to rules_js, which uses PNPM -- and the PNPM binary is controlled by bazel itself.

[.github/workflows/ci.yml]: .github/workflows/ci.yml
[versioned branch]: https://github.com/Zemnmez/monorepo/tree/versioned
I recommend having a local 'bazel' installation. However, if you just have NPM installed, you can:

The deploy scripts in [deploy/BUILD] run only on the versioned branch.
```
npx @bazel/bazelisk test //...
```

This will recursively run all tests in parallel.

[deploy/build]: deploy/BUILD
## Neat features

1. **Continuous patching**. Because Bazel allows this repo to be fully tested, Mend is able to automatically integrate patches for any and all dependencies of any project in this repo automatically.
2. **Linting**. There are central rules that enforce basic linting standards.
3. **Continuous deployment**. Every day, this repo runs a deploy script which detects any changes in any files and re-deploys or publishes any changed packages or services.
4. **Automatic versioning**. The packages published on NPM use the venerable api-extractor to detect major or minor version changes to the API, and automatically increment version numbers as relevant.
165 changes: 114 additions & 51 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,6 @@ load("//bzl:deps.bzl", "fetch_dependencies")

fetch_dependencies()

load("@build_bazel_rules_nodejs//:repositories.bzl", "build_bazel_rules_nodejs_dependencies")

build_bazel_rules_nodejs_dependencies()

load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")

rules_pkg_dependencies()
Expand All @@ -29,17 +25,6 @@ load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

# The yarn_install rule runs yarn anytime the package.json or yarn.lock file changes.
# It also extracts and installs any Bazel rules distributed in an npm package.
load("@build_bazel_rules_nodejs//:index.bzl", "node_repositories", "yarn_install")

yarn_install(
# Name this npm so that Bazel Label references look like @npm//package
name = "npm",
package_json = "//:package.json",
yarn_lock = "//:yarn.lock",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
Expand Down Expand Up @@ -75,21 +60,6 @@ rules_proto_dependencies()

rules_proto_toolchains()

load("@rules_typescript_proto//:index.bzl", "rules_typescript_proto_dependencies")

rules_typescript_proto_dependencies()

load("@npm//@bazel/labs:package.bzl", "npm_bazel_labs_dependencies")

npm_bazel_labs_dependencies()

load("@rules_nodejs//nodejs:repositories.bzl", "nodejs_register_toolchains")

nodejs_register_toolchains(
# You can choose whatever name you like for the node toolchains, and can register more than one version.
name = "nodejs",
)

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")

# If you use WORKSPACE.bazel, use the following line instead of the bare gazelle_dependencies():
Expand All @@ -100,27 +70,6 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")

protobuf_deps()

load("@build_bazel_rules_nodejs//toolchains/esbuild:esbuild_repositories.bzl", "esbuild_repositories")

esbuild_repositories(
npm_repository = "npm",
)

load("@aspect_rules_swc//swc:dependencies.bzl", "rules_swc_dependencies")

rules_swc_dependencies()

# Fetches a pre-built Rust-node binding from
# https://github.com/swc-project/swc/releases.
# If you'd rather compile it from source, you can use rules_rust, fetch the project,
# then register the toolchain yourself. (Note, this is not yet documented)
load("@aspect_rules_swc//swc:repositories.bzl", "LATEST_VERSION", "swc_register_toolchains")

swc_register_toolchains(
name = "swc",
swc_version = LATEST_VERSION,
)

load("@rules_rust//rust:repositories.bzl", "rules_rust_dependencies", "rust_register_toolchains")

rules_rust_dependencies()
Expand All @@ -146,3 +95,117 @@ crate_repositories()
load("@rules_rust//tools/rust_analyzer:deps.bzl", "rust_analyzer_dependencies")

rust_analyzer_dependencies()

load("@aspect_rules_js//js:repositories.bzl", "rules_js_dependencies")

rules_js_dependencies()

load("@rules_nodejs//nodejs:repositories.bzl", "DEFAULT_NODE_VERSION", "nodejs_register_toolchains")

nodejs_register_toolchains(
name = "nodejs",
node_version = DEFAULT_NODE_VERSION,
)

load("@aspect_rules_js//npm:npm_import.bzl", "npm_translate_lock")

npm_translate_lock(
name = "npm",
# no idea how to do this yet and while there are docs
# the examples have a file called 'patches' that
# I don't have and also don't know how to generate
# verify_patches = "//patches",
data = [
"//:patches/svgo@3.0.2.patch",
],
# https://github.com/aspect-build/rules_js/blob/11d3c4655b07d3094e0682a75402e972e8dae361/WORKSPACE#L139
lifecycle_hooks_execution_requirements = {
"puppeteer": [
"no-sandbox",
# Workaround Engflow not honoring requires-network on build actions
"no-remote-exec",
"requires-network",
],
},
npmrc = "//:.npmrc",
# tool just breaks if you remove this lol
patch_args = {"*": ["-p1"]},
pnpm_lock = "//:pnpm-lock.yaml",
verify_node_modules_ignored = "//:.bazelignore",
)

load("@npm//:repositories.bzl", "npm_repositories")

npm_repositories()

# https://github.com/Zemnmez/rules_js/blob/bdf3d5a61c6af9e5d7d584cfe5f78f42d1e3f227/docs/faq.md#can-i-use-bazel-managed-pnpm
load("@aspect_rules_js//npm:npm_import.bzl", "pnpm_repository")

pnpm_repository(name = "pnpm")

##################
# rules_ts setup #
##################
# Fetches the rules_ts dependencies.
# If you want to have a different version of some dependency,
# you should fetch it *before* calling this.
# Alternatively, you can skip calling this function, so long as you've
# already fetched all the dependencies.
load("@aspect_rules_ts//ts:repositories.bzl", "rules_ts_dependencies")

rules_ts_dependencies(

# Alternatively, you could pick a specific version, or use
# load("@aspect_rules_ts//ts:repositories.bzl", "LATEST_VERSION")
# ts_version = LATEST_VERSION

# HEAD is currently further along than rules_ts has patches for.
# this will break updates to typescript, so should probably be removed when
# rules_ts is updated.
# As per https://docs-legacy.aspect.build/aspect-build/rules_ts/v0.10.0/docs/repositories-docgen.html
# Generate via
# curl --silent https://registry.npmjs.org/typescript/5.0.3 | jq -r '.dist.integrity'
#ts_integrity = "sha512-cW9T5W9xY37cc+jfEnaUvX91foxtHkza3Nw3wkoF4sSlKn0MONdkdEndig/qPBWXNkmplh3NzayQzCiHM4/hqw==",
# This keeps the TypeScript version in-sync with the editor, which is typically best.
ts_version_from = "//:package.json",
)

# Fetch and register node, if you haven't already
load("@rules_nodejs//nodejs:repositories.bzl", "DEFAULT_NODE_VERSION", "nodejs_register_toolchains")

nodejs_register_toolchains(
name = "node",
node_version = DEFAULT_NODE_VERSION,
)

# Register aspect_bazel_lib toolchains;
# If you use npm_translate_lock or npm_import from aspect_rules_js you can omit this block.
load("@aspect_bazel_lib//lib:repositories.bzl", "register_copy_directory_toolchains", "register_copy_to_directory_toolchains")

register_copy_directory_toolchains()

register_copy_to_directory_toolchains()

###################
# rules_swc setup #
###################

# Fetches the rules_swc dependencies.
# If you want to have a different version of some dependency,
# you should fetch it *before* calling this.
# Alternatively, you can skip calling this function, so long as you've
# already fetched all the dependencies.
load("@aspect_rules_swc//swc:dependencies.bzl", "rules_swc_dependencies")

rules_swc_dependencies()

# Fetches a SWC cli from
# https://github.com/swc-project/swc/releases
# If you'd rather compile it from source, you can use rules_rust, fetch the project,
# then register the toolchain yourself. (Note, this is not yet documented)
load("@aspect_rules_swc//swc:repositories.bzl", "LATEST_VERSION", "swc_register_toolchains")

swc_register_toolchains(
name = "swc",
swc_version = LATEST_VERSION,
)
5 changes: 1 addition & 4 deletions bin/linux/pulumi/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ package(default_visibility = ["//:__subpackages__"])
load("@bazel_skylib//rules:native_binary.bzl", "native_test")
load("@bazel_skylib//lib:selects.bzl", "selects")


config_setting(
name = "darwin_arm64",
constraint_values = [
Expand Down Expand Up @@ -53,7 +52,6 @@ config_setting(
],
)


alias(
name = "pulumi",
actual = select({
Expand All @@ -67,12 +65,11 @@ alias(
}),
)


native_test(
# The name of the test rule.
name = "smoke",
# label; path of the pre-built executable
src = ":pulumi",
# output; an output name for the copy of the binary
out = "pulumi.local",
)
)
Loading