[API Feature Request] Installing native node add-ons using node-gyp

[loris]

Description

Right now, it looks like we cannot use any native node add-ons which are built using node-gyp. For instance, I tried various npm package to access sqlite database but failed: sqlite, sqlite3, better-sqlite3, etc

Steps To Reproduce

1. npm install and try to import in your extension any package relying on node-gyp (for instance: https://github.com/JoshuaWise/better-sqlite3)

The current behavior

You will get a nodejs error (such as Error: Could not find module root given file: "node:internal/modules/cjs/loader". Do you have a package.json file?)

The expected behavior

No error

[bromanko]

sqlite3 also does not work because it tries to read package.json in the module and that file isn't found when running in Raycast.

Error: package.json does not exist at /Users/bromanko/.config/raycast/extensions/package.json

        throw new Error("package.json does not exist at " + package_json_path);
              ^

Object.exports.find:pre-binding.js:18:15

---
15: 
16: exports.find = function(package_json_path,opts) {
17:    if (!existsSync(package_json_path)) {
18:         throw new Error("package.json does not exist at " + package_json_path);
19:    }
20:    var package_json = require(package_json_path);
21:    versioning.validate_config(package_json,opts);
---

node_modules/sqlite3/lib/sqlite3-binding.js:sqlite3-binding.js:3:27
__require:searchChrome.js:9:44
node_modules/sqlite3/lib/sqlite3.js:sqlite3.js:2:15
__require:searchChrome.js:9:44
Object.<anonymous>:browserHistory.tsx:4:21
Module._compile:loader:1101:14
Object.Module._extensions..js:loader:1153:10
Module.load:loader:981:32
Function.Module._load:loader:822:12

[bromanko]

FWIW I was able to use the WASM-based sql.js in #172

[fe9lix]

Hey guys, I'm flagging this as "feature" instead of bug because this is more like a known JS bundler limitation (in our case esbuild) and something that could be argued is "missing" rather than being buggy.

For now, there a workarounds such as marking certain modules as external via package.json and using other alternatives such as the mentioned wasm-based package above.

[ayroblu]

👋 Hey, based on this issue: evanw/esbuild#1051 is there still anything that can't be done here? My understanding is that esbuild simply grabs all the .node files and bundles them along? Though sqlite3 fails for weird reasons, better-sqlite3 looks like it could work with this native node modules plugin approach?

I get a list of errors like:

Error: Could not locate the bindings file. Tried:
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/out/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/out/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/default/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/compiled/18.10.0/darwin/arm64/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/release/install-root/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/debug/install-root/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/default/install-root/better_sqlite3.node

[gutenye]

For now, there a workarounds such as marking certain modules as external via package.json and using other alternatives such as the mentioned wasm-based package above.

@fe9lix How to make it as external via package.json? Can you give an example?