Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: only set cache when find module succeed #36642

Closed
wants to merge 1 commit into from

Conversation

ZYSzys
Copy link
Member

@ZYSzys ZYSzys commented Dec 27, 2020

Fixes: #36638

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

@ZYSzys ZYSzys added the module Issues and PRs related to the module subsystem. label Dec 27, 2020
aduh95
aduh95 previously approved these changes Dec 27, 2020
lib/internal/modules/cjs/loader.js Outdated Show resolved Hide resolved
@ZYSzys ZYSzys added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 27, 2020
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 27, 2020
@nodejs-github-bot
Copy link
Collaborator

@BridgeAR BridgeAR added the benchmark Issues and PRs related to the benchmark subsystem. label Dec 27, 2020
@BridgeAR
Copy link
Member

This will likely render the cache significantly less useful. Folders and files are AFAIK also handled separately and this only saves one of them in the cache?
We should validate the cache hit ratio during the loading of a regular project (with lots of modules).

It's implemented like this for a long time and it could be considered a breaking change. Thus, we should carefully evaluate to change this.

Trott
Trott previously approved these changes Dec 27, 2020
@Trott
Copy link
Member

Trott commented Dec 27, 2020

It's implemented like this for a long time and it could be considered a breaking change. Thus, we should carefully evaluate to change this.

If this change doesn't land, a test should be added for the current behavior to make it clear that this is the desired behavior.

@Trott Trott added the semver-major PRs that contain breaking changes and should be released in the next major version. label Dec 27, 2020
@vsemozhetbyt
Copy link
Contributor

And if this change doesn't land, maybe this needs some documenting? The initial reporter was surprised that the documented cache cannot be cleaned via API in this case.

@mscdex
Copy link
Contributor

mscdex commented Dec 27, 2020

The initial reporter was surprised that the documented cache cannot be cleaned via API in this case.

There is no real cleaning to do, the statCache is temporary and only exists for the duration of an original/root level require().

@mscdex
Copy link
Contributor

mscdex commented Dec 27, 2020

I'm also -1 on this for the performance reasons and because it seems like a minor edge case.

@Trott Trott dismissed their stale review December 27, 2020 19:20

Removing approval in light of subsequent comments.

@aduh95 aduh95 dismissed their stale review December 27, 2020 23:02

Removing approval in light of subsequent comments, but I think we should fix the current behaviour and/or document it.

@ZYSzys
Copy link
Member Author

ZYSzys commented Dec 28, 2020

I'm also -1 on this for the performance reasons and because it seems like a minor edge case.

But AFAIK, it seems that the benchmark result didn't show any regression on performance.
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/817/

@@ -149,7 +149,10 @@ function stat(filename) {
if (result !== undefined) return result;
}
const result = internalModuleStat(filename);
if (statCache !== null) statCache.set(filename, result);
if (statCache !== null && result >= 0) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folders and files are AFAIK also handled separately and this only saves one of them in the cache?

Folders and files are both handled now. /cc @BridgeAR

@Trott
Copy link
Member

Trott commented Dec 28, 2020

I'm +1 on this if there's no measurable performance difference, but I would like one or more modules folks to review. @nodejs/modules

@mscdex
Copy link
Contributor

mscdex commented Dec 28, 2020

But AFAIK, it seems that the benchmark result didn't show any regression on performance.

The current benchmarks don't measure the code being changed here.

@ZYSzys
Copy link
Member Author

ZYSzys commented Dec 28, 2020

The only change here is we didn't set the statCache when the result of internalModuleStat(filename) less than 0, which usually means there was an error(maybe MODULE_NOT_FOUND or something). And I think this was always an edge case, most of time we throw error directly in this case right ?

The most normal scene(we can require module successfully) still cached if the result of internalModuleStat(filename) >= 0, so I don't think there would be some performance regression.

How do you think ?
/cc @mscdex @BridgeAR @nodejs/modules

@bmeck
Copy link
Member

bmeck commented Dec 28, 2020

This seems fine to me.

@ZYSzys ZYSzys force-pushed the module-cache branch 2 times, most recently from 4970529 to 2ec982a Compare December 30, 2020 09:18
Copy link
Contributor

@guybedford guybedford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to do this for CJS do we also want to extend this functionality to the ES module resolver too?

On this topic - perhaps it's time to consider flushing failed module loads when there is a not found / fetch error to allow retries in the ESM loader too (as I believe the browser does although haven't tested this recently). //cc @bmeck

@bmeck
Copy link
Member

bmeck commented Dec 30, 2020

The determinism/cache only must populate on success for import() per https://tc39.es/ecma262/#sec-hostimportmoduledynamically , I think ejecting prior to any user code/side effects/partial eval seems fine. Reading https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script the actual impact of failed network seems a bit complex and only ejects the failed requests keeping successful responses. It never links if there is an ejection though. That behavior seems fine since even with a cycle you can never get to an orphaned source text.

@ZYSzys ZYSzys added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 5, 2021
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 5, 2021
@nodejs-github-bot
Copy link
Collaborator

@aduh95
Copy link
Contributor

aduh95 commented Jan 12, 2021

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/891/ (queued, will 404 until it starts)

EDIT: no significant perf regression or improvement.

                                               confidence improvement accuracy (*)    (**)   (***)
 module/module-require.js n=10000 type='dir'                  -1.58 %       ±6.23%  ±8.28% ±10.78%
 module/module-require.js n=10000 type='.js'                   5.57 %       ±9.73% ±12.95% ±16.88%
 module/module-require.js n=10000 type='.json'                 3.36 %       ±9.18% ±12.22% ±15.90%

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Jan 17, 2021

@ZYSzys
Copy link
Member Author

ZYSzys commented Jan 17, 2021

Could this get some more reviews from @nodejs/tsc since this is semver major(although I don't think it should be) ?

Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not TSC, but I'm +1 for this change.

@targos
Copy link
Member

targos commented Jan 18, 2021

@Trott do you feel strongly about this being semver-major?

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Jan 25, 2021

@Trott do you feel strongly about this being semver-major?

I don't feel strongly, but @BridgeAR might?

ZYSzys pushed a commit to zys-contrib/node that referenced this pull request Feb 12, 2021
PR-URL: nodejs#36642
Fixes: nodejs#36638
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@ZYSzys
Copy link
Member Author

ZYSzys commented Feb 12, 2021

Landed in aecd5eb.

@ZYSzys ZYSzys closed this Feb 12, 2021
@ZYSzys ZYSzys deleted the module-cache branch February 12, 2021 16:21
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable Changes:

Deprecations and Removals

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **async_hooks**: add thisArg to AsyncResource.bind (James M Snell)
  [#36782]
- **buffer**: expose btoa and atob as globals (James M Snell)
  [#37786]
- **build**: remove support for Python 2 (Christian Clauss)
  [#36691]
- **build**: default PYTHON to python3 in Makefile (Michaël Zasso)
  [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#37587]
- **build**: include minimal V8 headers in distribution (Michaël Zasso)
  [#37570]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#37330]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#36139]
- **build**: use C++11 ABI with libstdc++ (Anna Henningsen)
  [#36634]
- **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 1648e050cade (Michaël Zasso)
  [#37587]
- **deps**: silence irrelevant V8 warnings (Michaël Zasso)
  [#37587]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **deps**: bump minimum ICU version to 68 (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick 8957d4677aa7 (Michaël Zasso)
  [#37330]
- **deps**: V8: backport a11395433dbd (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick deb0813166f3 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a6a22874c81 (Michaël Zasso)
  [#36139]
- **deps**: silence irrelevant V8 warning (Michaël Zasso)
  [#37330]
- **deps**: workaround stod() limitations on SmartOS (Colin Ihrig)
  [#37330]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick deb0813166f3 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a6a22874c81 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 2059ee813359 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick bde7ee5473d6 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a712984025e (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 0b96e5b0bfb2 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick fbb28902e049 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 821fb3883a8e (Michaël Zasso)
  [#35700]
- **deps**: workaround stod() limitations on SmartOS (Colin Ihrig)
  [#36139]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: V8: silence irrelevant warnings (Michaël Zasso)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 821fb3883a8e (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 45e49775f5a3 (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 7b3a27b7ae65 (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick d76abfed3512 (Michaël Zasso)
  [#35415]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: update V8 postmortem metadata script (Colin Ihrig)
  [#35415]
- **deps**: update V8 postmortem metadata script (Colin Ihrig)
  [#33579]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: V8: silence irrelevant warnings (Michaël Zasso)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **dns**: use url module instead of punycode for IDNA (Antoine du
  Hamel) [#35091]
- **doc**: update minimum supported Xcode to 11 (Michaël Zasso)
  [#37872]
- **doc**: update minimum supported GCC to 8.3 (Michaël Zasso)
  [#37871]
- **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **doc**: fixup http.IncomingMessage deprecation code (Guy Bedford)
  [#36917]
- **doc**: add http.IncomingMessage#connection (Pranshu Srivastava)
  [#33768]
- **events**: change EventTarget handler exception behavior (Nitzan
  Uziely) [#37237]
- **fs**: remove permissive rmdir recursive (Antoine du Hamel)
  [#37216]
- **fs**: add validation for fd and path (Dylan Elliott)
  [#35187]
- **fs**: runtime deprecate rmdir recursive option (Antoine du Hamel)
  [#37302]
- **fs**: fix flag and mode validation (James M Snell)
  [#37480]
- **http**: use objects with null prototype in Agent (Michaël Zasso)
  [#36409]
- **lib**: runtime deprecate access to process.binding('http_parser')
  (James M Snell) [#37813]
- **lib**: runtime deprecate access to process.binding('url') (James M
  Snell) [#37799]
- **lib**: make process.binding('util') return only type checkers (Anna
  Henningsen) [#37819]
- **lib**: runtime deprecate access to process.binding('crypto') (James
  M Snell) [#37790]
- **lib**: runtime deprecate access to process.binding('signal_wrap')
  (James M Snell) [#37800]
- **lib**: runtime deprecate access to process.binding('v8') (James M
  Snell) [#37789]
- **lib**: aggregate errors to avoid error swallowing (Antoine du Hamel)
  [#37460]
- **lib**: load v8_prof_processor dependencies as ESM (Michaël Zasso)
  [#37587]
- **lib**: runtime deprecate access to process.binding('async_wrap')
  (James M Snell) [#37576]
- **lib**: remove usage of url.parse (raisinten)
  [#36853]
- **lib**: add error handling for input stream (rexagod)
  [#31603]
- **lib,src**: update cluster to use Parent (Michael Dawson)
  [#36478]
- **module**: runtime deprecate subpath folder mappings (Antoine du
  Hamel) [#37215]
- **module**: runtime deprecate "main" index and extension lookups
  (Antoine du Hamel) [#37206]
- **module**: runtime deprecate invalid package.json main entries
  (Antoine du Hamel) [#37204]
- **module**: remove module.createRequireFromPath (Antoine du Hamel)
  [#37201]
- **module**: only set cache when finding module succeeds (Yongsheng
  Zhang) [#36642]
- **perf_hooks**: make performance a global (James M Snell)
  [#37970]
- **perf_hooks**: complete overhaul of the implementation (James M
  Snell) [#37136]
- **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **process**: runtime deprecate changing process.config (James M Snell)
  [#36902]
- **readline**: cursorTo throw error on NaN (Zijian Liu)
  [#36379]
- **src**: use non-deprecated GetCreationContext from V8 (Michaël Zasso)
  [#37587]
- **src**: remove V8_FT_ADAPTOR for V8 update (Colin Ihrig)
  [#37587]
- **src**: use non-deprecated V8 module APIs (Michaël Zasso)
  [#37587]
- **src**: update NODE_MODULE_VERSION to 93 (Michaël Zasso)
  [#37587]
- **src**: use non-deprecated V8 module and script APIs (Michaël Zasso)
  [#37330]
- **src**: update NODE_MODULE_VERSION to 92 (Michaël Zasso)
  [#37330]
- **src**: update NODE_MODULE_VERSION to 91 (Michaël Zasso)
  [#36139]
- **src**: mark internally exported functions as explicitly internal
  (Tyler Ang-Wanek) [#37000]
- **src**: inline AsyncCleanupHookHandle in headers (Tyler Ang-Wanek)
  [#37000]
- **src**: fix v8 api deprecation (Jiawen Geng)
  [#35700]
- **src**: update NODE_MODULE_VERSION to 90 (Michaël Zasso)
  [#35700]
- **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **test**: mark test-return-on-exit as flaky (Michaël Zasso)
  [#36139]
- **test**: mark WASI's test-return-on-exit as flaky (Colin Ihrig)
  [#36139]
- **tools**: update V8 gypfiles for 9.0 (Michaël Zasso)
  [#37587]
- **tools**: update V8 gypfiles for 8.9 (Michaël Zasso)
  [#37330]
- **tools**: update V8 gypfiles for 8.8 (Michaël Zasso)
  [#36139]
- **tools**: update V8 gypfiles for 8.7 (Michaël Zasso)
  [#35700]
- **worker**: send correct error status for worker init (Yash Ladha)
  [#36242]

PR-URL: #37678
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable changes:

Deprecations and Removals:

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2 (Christian
  Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **deps**: update ICU to 69.1 (Michaël Zasso)
  [#38178]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **(SEMVER-MAJOR)** **async_hooks**: add thisArg to AsyncResource.bind
  (James M Snell) [#36782]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **(SEMVER-MAJOR)** **build**: update Makefile to support fat binary
  (Ash Cripps) [#37861]
- **(SEMVER-MAJOR)** **build**: include minimal V8 headers in
  distribution (Michaël Zasso) [#37570]
- **(SEMVER-MAJOR)** **build**: use C++11 ABI with libstdc++
  (Anna Henningsen) [#36634]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **(SEMVER-MAJOR)** **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **(SEMVER-MAJOR)** **dns**: use url module instead of punycode for
  IDNA (Antoine du Hamel) [#35091]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **(SEMVER-MAJOR)** **doc**: add http.IncomingMessage#connection
  (Pranshu Srivastava) [#33768]
- **(SEMVER-MAJOR)** **events**: change EventTarget handler exception
  behavior (Nitzan Uziely) [#37237]
- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: add validation for fd and path
  (Dylan Elliott) [#35187]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **fs**: fix flag and mode validation
  (James M Snell) [#37480]
- **(SEMVER-MAJOR)** **http**: use objects with null prototype in Agent
  (Michaël Zasso) [#36409]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell)
  [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell)
  [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: aggregate errors to avoid error swallowing
  (Antoine du Hamel) [#37460]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **lib**: remove usage of url.parse
  (raisinten) [#36853]
- **(SEMVER-MAJOR)** **lib**: add error handling for input stream
  (rexagod) [#31603]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: only set cache when finding module
  succeeds (Yongsheng Zhang) [#36642]
- **(SEMVER-MAJOR)** **perf_hooks**: make performance a global (James M
  Snell) [#37970]
- **(SEMVER-MAJOR)** **perf_hooks**: complete overhaul of the
  implementation (James M Snell) [#37136]
- **(SEMVER-MAJOR)** **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]
- **(SEMVER-MAJOR)** **readline**: cursorTo throw error on NaN
  (Zijian Liu) [#36379]
- **(SEMVER-MAJOR)** **src**: mark internally exported functions as
  explicitly internal (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: inline AsyncCleanupHookHandle in headers
  (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **(SEMVER-MAJOR)** **worker**: send correct error status for worker
  init (Yash Ladha) [#36242]

PR-URL: #37678
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable changes:

Deprecations and Removals:

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2 (Christian
  Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **deps**: update ICU to 69.1 (Michaël Zasso)
  [#38178]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **(SEMVER-MAJOR)** **async_hooks**: add thisArg to AsyncResource.bind
  (James M Snell) [#36782]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **(SEMVER-MAJOR)** **build**: update Makefile to support fat binary
  (Ash Cripps) [#37861]
- **(SEMVER-MAJOR)** **build**: include minimal V8 headers in
  distribution (Michaël Zasso) [#37570]
- **(SEMVER-MAJOR)** **build**: use C++11 ABI with libstdc++
  (Anna Henningsen) [#36634]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **(SEMVER-MAJOR)** **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **(SEMVER-MAJOR)** **dns**: use url module instead of punycode for
  IDNA (Antoine du Hamel) [#35091]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **(SEMVER-MAJOR)** **doc**: add http.IncomingMessage#connection
  (Pranshu Srivastava) [#33768]
- **(SEMVER-MAJOR)** **events**: change EventTarget handler exception
  behavior (Nitzan Uziely) [#37237]
- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: add validation for fd and path
  (Dylan Elliott) [#35187]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **fs**: fix flag and mode validation
  (James M Snell) [#37480]
- **(SEMVER-MAJOR)** **http**: use objects with null prototype in Agent
  (Michaël Zasso) [#36409]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell)
  [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell)
  [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: aggregate errors to avoid error swallowing
  (Antoine du Hamel) [#37460]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **lib**: remove usage of url.parse
  (raisinten) [#36853]
- **(SEMVER-MAJOR)** **lib**: add error handling for input stream
  (rexagod) [#31603]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: only set cache when finding module
  succeeds (Yongsheng Zhang) [#36642]
- **(SEMVER-MAJOR)** **perf_hooks**: make performance a global (James M
  Snell) [#37970]
- **(SEMVER-MAJOR)** **perf_hooks**: complete overhaul of the
  implementation (James M Snell) [#37136]
- **(SEMVER-MAJOR)** **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]
- **(SEMVER-MAJOR)** **readline**: cursorTo throw error on NaN
  (Zijian Liu) [#36379]
- **(SEMVER-MAJOR)** **src**: mark internally exported functions as
  explicitly internal (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: inline AsyncCleanupHookHandle in headers
  (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **(SEMVER-MAJOR)** **worker**: send correct error status for worker
  init (Yash Ladha) [#36242]

PR-URL: #37678
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Issues and PRs related to the benchmark subsystem. module Issues and PRs related to the module subsystem. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fs, cjs modules: failed require prevents valid require from loading a module just written by writeFileSync