Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

Add a Cypress Test 🌲 #8295

Merged
merged 45 commits into from
Apr 14, 2022
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
8d22be3
A first, maybe working cypress test
dbkr Apr 12, 2022
0854ee0
Merge remote-tracking branch 'origin/develop' into dbkr/cypress
dbkr Apr 12, 2022
6308954
Fix yaml
dbkr Apr 12, 2022
f4f7cac
This file is important
dbkr Apr 12, 2022
53a30ed
Merge remote-tracking branch 'origin/develop' into dbkr/cypress
dbkr Apr 12, 2022
8312ead
try & find where it's put the artifact
dbkr Apr 12, 2022
e2be179
Download artifact to a directory
dbkr Apr 12, 2022
2c6265d
pics or it didn't happen
dbkr Apr 12, 2022
0ad99d7
Add conditional, otherwise no artifacts on failure...
dbkr Apr 12, 2022
358b692
Try increasing timeout
dbkr Apr 12, 2022
7ab9222
Try in chrome
dbkr Apr 12, 2022
67a59bd
Get docker logs to see why it's failing
dbkr Apr 12, 2022
2d8200b
Try changing mode on homeserver.yaml
dbkr Apr 12, 2022
b2adcf5
debug
dbkr Apr 12, 2022
d9861a8
More debugging
dbkr Apr 12, 2022
d08597d
more file permissions debugging
dbkr Apr 12, 2022
322a750
ARGH
dbkr Apr 12, 2022
02965aa
more debug
dbkr Apr 12, 2022
6c0acdb
sigh
dbkr Apr 12, 2022
20a6963
Eugh, that's not how arguments work
dbkr Apr 12, 2022
9f6128c
Add the option to really allow open registration
dbkr Apr 12, 2022
41030ec
failure to yaml
dbkr Apr 12, 2022
b7c7f64
Upload docker logs as artifacts
dbkr Apr 12, 2022
e2b7235
Put the conditional back
dbkr Apr 12, 2022
0ef494a
Merge remote-tracking branch 'origin/develop' into dbkr/cypress
dbkr Apr 12, 2022
bf80b65
Upgrade types in end to end tests
dbkr Apr 12, 2022
979b5ff
Try reducing timeout a bit
dbkr Apr 12, 2022
ea6f8af
Hex is not octal
dbkr Apr 12, 2022
2aca928
Remove file mode
dbkr Apr 12, 2022
38bbb55
Give the log files extensions
dbkr Apr 12, 2022
a1e9350
Rename workflow file now it also does tests
dbkr Apr 13, 2022
1e8953a
Add cypress scripts
dbkr Apr 13, 2022
53b8a1c
copyright headers
dbkr Apr 13, 2022
1619330
Use ? operator
dbkr Apr 13, 2022
b56da36
Use develop synapse image
dbkr Apr 13, 2022
a78d212
Tidy up any remaining synapses after each spec run
dbkr Apr 13, 2022
4dd0545
Don't upload video on test pass
dbkr Apr 13, 2022
ca86bad
Enable linting on cypress files
dbkr Apr 13, 2022
cfdb2af
Type check cypress files
dbkr Apr 13, 2022
f82aa9a
Rename workflow file again
dbkr Apr 13, 2022
d0cbae9
Don't plus + characters in container name
dbkr Apr 13, 2022
13ee082
Fix yaml
dbkr Apr 13, 2022
58f32b7
Stream logs to file
dbkr Apr 13, 2022
d5d9aa7
Add note to end to end tester to sya what's been ported
dbkr Apr 13, 2022
dd0ff8e
Put docker rm in finally block
dbkr Apr 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ module.exports = {
files: [
"src/**/*.{ts,tsx}",
"test/**/*.{ts,tsx}",
"cypress/**/*.ts",
],
extends: [
"plugin:matrix-org/typescript",
Expand Down
49 changes: 49 additions & 0 deletions .github/workflows/element-build-and-test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Produce a build of element-web with this version of react-sdk
# and any matching branches of element-web and js-sdk, output it
# as an artifact and run integration tests.
name: Element Web - Build and Test
on:
pull_request:
jobs:
build:
runs-on: ubuntu-latest
env:
# This must be set for fetchdep.sh to get the right branch
PR_NUMBER: ${{github.event.number}}
steps:
- uses: actions/checkout@v2
- name: Build
run: scripts/ci/layered.sh && cd element-web && cp element.io/develop/config.json config.json && CI_PACKAGE=true yarn build
- name: Upload Artifact
uses: actions/upload-artifact@v2
with:
name: previewbuild
path: element-web/webapp
# We'll only use this in a triggered job, then we're done with it
retention-days: 1
cypress:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Download build
uses: actions/download-artifact@v3
with:
name: previewbuild
path: webapp
- name: Run Cypress tests
uses: cypress-io/github-action@v2
with:
# The built in Electron runner seems to grind to a halt trying
# to run the tests, so use chrome.
browser: chrome
start: npx serve -p 8080 webapp
- name: Upload Artifact
if: failure()
uses: actions/upload-artifact@v2
with:
name: cypress-results
path: |
cypress/screenshots
cypress/videos
cypress/dockerlogs
23 changes: 0 additions & 23 deletions .github/workflows/layered-build.yaml

This file was deleted.

8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,11 @@ package-lock.json

.vscode
.vscode/

/cypress/videos
/cypress/downloads
/cypress/screenshots
/cypress/synapselogs
# These could have files in them but don't currently
# Cypress will still auto-create them though...
/cypress/fixtures
4 changes: 4 additions & 0 deletions cypress.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"baseUrl": "http://localhost:8080",
"videoUploadOnPasses": false
}
52 changes: 52 additions & 0 deletions cypress/integration/1-register/register.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

/// <reference types="cypress" />
dbkr marked this conversation as resolved.
Show resolved Hide resolved

import { SynapseInstance } from "../../plugins/synapsedocker/index";

describe("Registration", () => {
let synapseId;
let synapsePort;

beforeEach(() => {
cy.task<SynapseInstance>("synapseStart", "consent").then(result => {
synapseId = result.synapseId;
synapsePort = result.port;
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anti-Pattern: Trying to start a web server from within Cypress scripts with cy.exec() or cy.task() .

-- https://docs.cypress.io/guides/references/best-practices#Web-Servers

I assume we've weighed the risks here?

cy.visit("/#/register");
});

afterEach(() => {
cy.task("synapseStop", synapseId);
});

it("registers an account and lands on the home screen", () => {
cy.get(".mx_ServerPicker_change", { timeout: 15000 }).click();
cy.get(".mx_ServerPickerDialog_otherHomeserver").type(`http://localhost:${synapsePort}`);
cy.get(".mx_ServerPickerDialog_continue").click();
// wait for the dialog to go away
cy.get('.mx_ServerPickerDialog').should('not.exist');
cy.get("#mx_RegistrationForm_username").type("alice");
cy.get("#mx_RegistrationForm_password").type("totally a great password");
cy.get("#mx_RegistrationForm_passwordConfirm").type("totally a great password");
cy.get(".mx_Login_submit").click();
cy.get(".mx_RegistrationEmailPromptDialog button.mx_Dialog_primary").click();
cy.get(".mx_InteractiveAuthEntryComponents_termsPolicy input").click();
cy.get(".mx_InteractiveAuthEntryComponents_termsSubmit").click();
cy.url().should('contain', '/#/home');
});
});
23 changes: 23 additions & 0 deletions cypress/plugins/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

/// <reference types="cypress" />

import { synapseDocker } from "./synapsedocker/index";

export default function(on, config) {
synapseDocker(on, config);
}
210 changes: 210 additions & 0 deletions cypress/plugins/synapsedocker/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

/// <reference types="cypress" />

import * as path from "path";
import * as os from "os";
import * as crypto from "crypto";
import * as childProcess from "child_process";
import * as fse from "fs-extra";

// A cypress plugins to add command to start & stop synapses in
// docker with preset templates.

interface SynapseConfig {
configDir: string;
registrationSecret: string;
}

export interface SynapseInstance extends SynapseConfig {
synapseId: string;
port: number;
}

const synapses = new Map<string, SynapseInstance>();

function randB64Bytes(numBytes: number): string {
return crypto.randomBytes(numBytes).toString("base64").replace(/=*$/, "");
}
Comment on lines +40 to +42
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should already have a random string utility we can use?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, true - I guess importing code from the app would be fine, but it feels like it's probably good hygiene to keep them completely separate, especially when the code in question is as simple as it is. Plus this is node-specific and we specifically want base64 encoding for the ed25519 key.


async function cfgDirFromTemplate(template: string): Promise<SynapseConfig> {
const templateDir = path.join(__dirname, "templates", template);

const stats = await fse.stat(templateDir);
if (!stats?.isDirectory) {
throw new Error(`No such template: ${template}`);
}
const tempDir = await fse.mkdtemp(path.join(os.tmpdir(), 'react-sdk-synapsedocker-'));

// change permissions on the temp directory so the docker container can see its contents
await fse.chmod(tempDir, 0o777);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

20% sure this won't work on windows, but also a good chance that nodejs will have stubbed it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nodejs says, "on Windows only the write permission can be changed, and the distinction among the permissions of group, owner or others is not implemented." so sounds like it will work to some extent, or at least no-op.


// copy the contents of the template dir, omitting homeserver.yaml as we'll template that
console.log(`Copy ${templateDir} -> ${tempDir}`);
await fse.copy(templateDir, tempDir, { filter: f => path.basename(f) !== 'homeserver.yaml' });

const registrationSecret = randB64Bytes(16);
const macaroonSecret = randB64Bytes(16);
const formSecret = randB64Bytes(16);

// now copy homeserver.yaml, applying sustitutions
console.log(`Gen ${path.join(templateDir, "homeserver.yaml")}`);
let hsYaml = await fse.readFile(path.join(templateDir, "homeserver.yaml"), "utf8");
hsYaml = hsYaml.replace(/{{REGISTRATION_SECRET}}/g, registrationSecret);
hsYaml = hsYaml.replace(/{{MACAROON_SECRET_KEY}}/g, macaroonSecret);
hsYaml = hsYaml.replace(/{{FORM_SECRET}}/g, formSecret);
await fse.writeFile(path.join(tempDir, "homeserver.yaml"), hsYaml);

// now generate a signing key (we could use synapse's config generation for
// this, or we could just do this...)
// NB. This assumes the homeserver.yaml specifies the key in this location
const signingKey = randB64Bytes(32);
console.log(`Gen ${path.join(templateDir, "localhost.signing.key")}`);
await fse.writeFile(path.join(tempDir, "localhost.signing.key"), `ed25519 x ${signingKey}`);

return {
configDir: tempDir,
registrationSecret,
};
}

// Start a synapse instance: the template must be the name of
// one of the templates in the cypress/plugins/synapsedocker/templates
// directory
async function synapseStart(template: string): Promise<SynapseInstance> {
const synCfg = await cfgDirFromTemplate(template);

console.log(`Starting synapse with config dir ${synCfg.configDir}...`);

const containerName = `react-sdk-cypress-synapse-${crypto.randomBytes(4).toString("hex")}`;

const synapseId = await new Promise<string>((resolve, reject) => {
childProcess.execFile('docker', [
"run",
"--name", containerName,
"-d",
"-v", `${synCfg.configDir}:/data`,
"-p", "8008/tcp",
"matrixdotorg/synapse:develop",
"run",
], (err, stdout) => {
if (err) reject(err);
resolve(stdout.trim());
});
});

// Get the port that docker allocated: specifying only one
// port above leaves docker to just grab a free one, although
// in hindsight we need to put the port in public_baseurl in the
// config really, so this will probably need changing to use a fixed
// / configured port.
const port = await new Promise<number>((resolve, reject) => {
childProcess.execFile('docker', [
"port", synapseId, "8008",
], { encoding: 'utf8' }, (err, stdout) => {
if (err) reject(err);
resolve(Number(stdout.trim().split(":")[1]));
});
});

synapses.set(synapseId, Object.assign({
port,
synapseId,
}, synCfg));

console.log(`Started synapse with id ${synapseId} on port ${port}.`);
return synapses.get(synapseId);
}

async function synapseStop(id) {
const synCfg = synapses.get(id);

if (!synCfg) throw new Error("Unknown synapse ID");

const synapseLogsPath = path.join("cypress", "synapselogs", id);
await fse.ensureDir(synapseLogsPath);

const stdoutFile = await fse.open(path.join(synapseLogsPath, "stdout.log"), "w");
const stderrFile = await fse.open(path.join(synapseLogsPath, "stderr.log"), "w");
await new Promise<void>((resolve, reject) => {
childProcess.spawn('docker', [
"logs",
id,
], {
stdio: ["ignore", stdoutFile, stderrFile],
}).once('close', resolve);
});
await fse.close(stdoutFile);
await fse.close(stderrFile);

await new Promise<void>((resolve, reject) => {
childProcess.execFile('docker', [
"stop",
id,
], err => {
if (err) reject(err);
resolve();
});
});

await new Promise<void>((resolve, reject) => {
childProcess.execFile('docker', [
"rm",
id,
], err => {
if (err) reject(err);
resolve();
});
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be best to put this on a try { } finally { ... }, but not blocking for this PR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


await fse.remove(synCfg.configDir);

synapses.delete(id);

console.log(`Stopped synapse id ${id}.`);
// cypres deliberately fails if you return 'undefined', so
// return null to signal all is well and we've handled the task.
return null;
}

/**
* @type {Cypress.PluginConfig}
*/
// eslint-disable-next-line no-unused-vars
export function synapseDocker(on, config) {
on("task", {
synapseStart, synapseStop,
});

on("after:spec", async (spec) => {
// Cleans up any remaining synapse instances after a spec run
// This is on the theory that we should avoid re-using synapse
// instances between spec runs: they should be cheap enough to
// start that we can have a separate one for each spec run or even
// test. If we accidentally re-use synapses, we could inadvertantly
// make our tests depend on each other.
for (const synId of synapses.keys()) {
console.warn(`Cleaning up synapse ID ${synId} after ${spec.name}`);
await synapseStop(synId);
}
});

on("before:run", async () => {
// tidy up old synapse log files before each run
await fse.emptyDir(path.join("cypress", "synapselogs"));
});
}
3 changes: 3 additions & 0 deletions cypress/plugins/synapsedocker/templates/COPYME/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Meta-template for synapse templates

To make another template, you can copy this directory
Loading