Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[POC] Use WebAssembly build of package-spec for package validation #128076

Closed
wants to merge 1 commit into from
Closed

[POC] Use WebAssembly build of package-spec for package validation #128076

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions x-pack/plugins/fleet/server/services/epm/archive/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import { getBufferExtractor } from './extract';
export * from './cache';
export { getBufferExtractor, untarBuffer, unzipBuffer } from './extract';
export { generatePackageInfoFromArchiveBuffer } from './parse';
export { validateZipBufferWithPackageSpec } from './validation/validate';

export interface ArchiveEntry {
path: string;
Expand Down
35 changes: 35 additions & 0 deletions x-pack/plugins/fleet/server/services/epm/archive/validation/validate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import fs from 'fs';
import path from 'path';

// Provides global.Go runtime
import './wasm_exec';
// @ts-expect-error
const go = new Go();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: I'm not familiar with wasm_exec.js\tinygo, is it possible to use its importObject without polluting the global scope with Go?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without editing the wasm_exec.js from Go (this is not from tinygo), I don't see a way. I think it would be a small edit though and I agree we shouldn't merge this in its current state (polluting the global scope).


const wasmBuffer = fs.readFileSync(path.join(__dirname, 'validator.wasm'));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Just curious, I see this module is about 12mb, I assume it's not optimized yet (with more aggressive build optimization flags, wasm-opt and friends)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep we still need to explore making this smaller and distribute it via a proper npm module. This is currently being built with go-wasm, but I think it would make sense to look at tinygo.

/**
* Validates a zip archive using the package-spec
*/
export async function validateZipBufferWithPackageSpec(
name: string,
size: number,
buffer: Uint8Array
): Promise<void> {
const validator = await WebAssembly.instantiate(wasmBuffer, go.importObject);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if it's called multiple times then it'd probably make sense to compile module just once and reuse it for instantiation (with new WebAssembly.Module or WebAssembly.compile).

go.run(validator.instance);

try {
// @ts-expect-error
await global.elasticPackageSpec.validateFromZipReader(name, size, buffer);
} finally {
// @ts-expect-error
global.elasticPackageSpec.stop();
}
}
Binary file added x-pack/plugins/fleet/server/services/epm/archive/validation/validator.wasm
View file
Open in desktop
Binary file not shown.
Loading