Adding tests and readme updates (microsoft#71)

* Wiring up integration tests using playwright

* Bumping version to b23

* Do not catch and throw exceptions for insertRule

* Visualization tweaks to render canvas on top

* Updating version to 0.6.0

* Run tests in headless mode

* Readme updates

CONTRIBUTING.md

@@ -20,7 +20,7 @@ On Windows, grab an installer from here: https://git-scm.com/download/win and go
 
 On Mac and Linux, it's pre-installed.
 
-### Node.js
+### Node.js (12+)
 
 On Windows, grab an installer from nodejs.org and go with the default options:
 ```
@@ -70,6 +70,11 @@ To build:
 yarn build
 ```
 
+To test:
+```
+yarn test
+```
+
 On Ubuntu, if you run into errors, it may be because you are missing the libfontconfig package
 ```
 sudo apt-get install libfontconfig
@@ -89,6 +94,31 @@ Edit Clarity:
 Go to 'File -> Open Folder' and select the 'clarity' folder that you just cloned.
 ```
 
+Debug Tests
+
+To debug tests from Visual Studio Code, create "launch.json" in your root project folder:
+```
+{
+  "version": "0.2.0",
+  "configurations": [
+      {
+          "name": "Clarity Tests",
+          "type": "node",
+          "request": "launch",
+          "cwd": "${workspaceFolder}/packages/clarity-js",
+          "runtimeExecutable": "${workspaceRoot}/node_modules/.bin/ts-mocha",
+          "runtimeArgs": [
+              "-p",
+              "test/tsconfig.test.json",
+              "${file}"
+          ],
+          "protocol": "inspector"
+      }
+
+  ]
+}
+```
+
 ### Text Editor TSLint Plugin
 
 TSLint plugin will read Clarity's TSLint configuration and highlight any TSLint errors immediately as you edit your code.

README.md

@@ -1,21 +1,20 @@
 # Clarity
-Clarity is a behavioral analytics library written in javascript. It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
+Clarity is an open-source behavioral analytics library written in typescript, with two key goals: privacy & performance. 
 
-Clarity provides you all these insights by:
-* Observing content layout, viewport, and user's interactions with the page
-* Inspecting network requests on the page
-* Logging the event stream in JSON format to a configurable endpoint
+It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
 
-Clarity is a project in active development. While it's not yet ready for production use, we continue making improvements and encourage the community to join us in the process.
+It's the same code that powers Microsoft's hosted behavioral analytics solution: <a href="https://clarity.microsoft.com">https://clarity.microsoft.com</a>. If you would like to see a demo of how it works, checkout <a href="https://clarity.microsoft.com/demo/projects/view/3t0wlogvdz/impressions?date=Last%203%20days">live demo</a>. 
+
+We encourage the community to join us in building the best behavioral analytics library, that puts privacy first and prioritizes performance. 
 
 ## Project Structure
-1. **[clarity-js](https://github.com/microsoft/clarity/tree/master/packages/clarity-js)**: Instrumentation code that goes on the website and tracks user interactions as well as layout changes. It is responsible for batching all captured events, computing metrics and encoding data to minimize bytes sent over the wire.
+1. **[clarity-js](https://github.com/microsoft/clarity/tree/master/packages/clarity-js)**: Instrumentation code that goes on the website and tracks user interactions as well as layout changes.
 
-2. **[clarity-decode](https://github.com/microsoft/clarity/tree/master/packages/clarity-decode)**: Code, which usually runs on the server, decodes incoming data back into its original format. From analytics perspective, all analysis should happen on data coming out of decode module and should never run on encoded data from clarity-js directly.
+2. **[clarity-decode](https://github.com/microsoft/clarity/tree/master/packages/clarity-decode)**: Code, which usually runs on the server, decodes incoming data back into its original format.
 
-3. **[clarity-visualize](https://github.com/microsoft/clarity/tree/master/packages/clarity-visualize)**: This is what makes Clarity visual and intuitive. It takes the decoded data from clarity-decode and turns it back into pixel-perfect session replay, exactly how the user saw it.
+3. **[clarity-visualize](https://github.com/microsoft/clarity/tree/master/packages/clarity-visualize)**: It takes the decoded data from clarity-decode and turns it back into pixel-perfect session replay.
 
-4. **[clarity-devtools](https://github.com/microsoft/clarity/tree/master/packages/clarity-devtools)**: This is a developer tools extension for chromium based browsers. It demonstrates how various components of Clarity come together and enables live session captures against any website, including downloading event data.
+4. **[clarity-devtools](https://github.com/microsoft/clarity/tree/master/packages/clarity-devtools)**: Devtools extension for chromium based browsers to generate live captures against any website.
 
 ## Examples
 Here are some example sessions on popular websites visualized to demonstrate the telemetry captured:
@@ -25,14 +24,8 @@ Here are some example sessions on popular websites visualized to demonstrate the
 2. Cook with Manali (Mobile)
 </br><a href="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif"><img src="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif" title="Clarity - Cook With Manali Example"/></a> 
 
-## Project Goals
-* Enable a generic solution that is able to capture telemetry from third-party websites
-* Encourage participation from open-source community
-* Minimal configuration required by third party web-sites to get started
-* Mobile first
-
 ## Privacy Notice
-Clarity handles sensitive data with care. By default content on the page is masked before upload, so no actual text from the page is sent to the server.
+Clarity handles sensitive data with care. By default sensitive content on the page is masked before uploading to the server.
 
 ## Improving Clarity
 If you haven't already done so, start contributing by following instructions **[here](https://github.com/microsoft/clarity/blob/master/CONTRIBUTING.md)**.

lerna.json

@@ -2,7 +2,7 @@
   "packages": [
     "packages/*"
   ],
-  "version": "0.6.0-beta.22",
+  "version": "0.6.0",
   "npmClient": "yarn",
   "useWorkspaces": true
 }

package.json

@@ -1,7 +1,7 @@
 {
   "name": "clarity",
   "private": true,
-  "version": "0.6.0-beta.22",
+  "version": "0.6.0",
   "repository": "https://github.com/microsoft/clarity.git",
   "author": "Sarvesh Nagpal <sarveshn@microsoft.com>",
   "license": "MIT",
@@ -16,7 +16,8 @@
     "build:js": "yarn workspace clarity-js build",
     "build:decode": "yarn workspace clarity-decode build",
     "build:visualize": "yarn workspace clarity-visualize build",
-    "build:devtools": "yarn workspace clarity-devtools build"
+    "build:devtools": "yarn workspace clarity-devtools build",
+    "test": "yarn workspace clarity-js test"
   },
   "devDependencies": {
     "lerna": "^3.20.2"

packages/clarity-decode/README.md

@@ -1,9 +1,11 @@
 # Clarity
-Clarity is a behavioral analytics library written in javascript. It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
+Clarity is an open-source behavioral analytics library written in typescript, with two key goals: privacy & performance. 
 
-This package takes the input payload from [clarity-js](https://github.com/microsoft/clarity/tree/master/packages/clarity-js) and decodes it back into its original format. From analytics perspective, all analysis should happen on data coming out of decode module and should never run on encoded data from clarity-js directly.
+It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
 
-Clarity is a project in active development. While it's not yet ready for production use, we continue making improvements and encourage the community to join us in the process.
+This package takes the input payload from [clarity-js](https://github.com/microsoft/clarity/tree/master/packages/clarity-js) and decodes it back into its original format. 
+
+We encourage the community to join us in building the best behavioral analytics library, that puts privacy first and prioritizes performance. 
 
 ## Examples
 Here are some example sessions on popular websites visualized to demonstrate the telemetry captured:
@@ -13,14 +15,8 @@ Here are some example sessions on popular websites visualized to demonstrate the
 2. Cook with Manali (Mobile)
 </br><a href="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif"><img src="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif" title="Clarity - Cook With Manali Example"/></a> 
 
-## Project Goals
-* Enable a generic solution that is able to capture telemetry from third-party websites
-* Encourage participation from open-source community
-* Minimal configuration required by third party web-sites to get started
-* Mobile first
-
 ## Privacy Notice
-Clarity handles sensitive data with care. By default content on the page is masked before upload, so no actual text from the page is sent to the server.
+Clarity handles sensitive data with care. By default sensitive content on the page is masked before uploading to the server.
 
 ## Improving Clarity
 If you haven't already done so, start contributing by following instructions **[here](https://github.com/microsoft/clarity/blob/master/CONTRIBUTING.md)**.

packages/clarity-decode/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clarity-decode",
-  "version": "0.6.0-beta.22",
+  "version": "0.6.0",
   "description": "An analytics library that uses web page interactions to generate aggregated insights",
   "author": "Microsoft Corp.",
   "license": "MIT",
@@ -26,7 +26,7 @@
     "url": "https://github.com/Microsoft/clarity/issues"
   },
   "dependencies": {
-    "clarity-js": "^0.6.0-beta.22"
+    "clarity-js": "^0.6.0"
   },
   "devDependencies": {
     "@rollup/plugin-commonjs": "^11.1.0",

packages/clarity-decode/types/core.d.ts

@@ -1,4 +1,6 @@
-import { Data } from "clarity-js";
+import { Core, Data } from "clarity-js";
+
+export import Config = Core.Config;
 
 export interface PartialEvent {
     time: number;

packages/clarity-devtools/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clarity-devtools",
-  "version": "0.6.0-beta.22",
+  "version": "0.6.0",
   "private": true,
   "description": "Adds Clarity debugging support to browser devtools",
   "author": "Microsoft Corp.",
@@ -24,9 +24,9 @@
     "url": "https://github.com/Microsoft/clarity/issues"
   },
   "dependencies": {
-    "clarity-decode": "^0.6.0-beta.22",
-    "clarity-js": "^0.6.0-beta.22",
-    "clarity-visualize": "^0.6.0-beta.22"
+    "clarity-decode": "^0.6.0",
+    "clarity-js": "^0.6.0",
+    "clarity-visualize": "^0.6.0"
   },
   "devDependencies": {
     "@rollup/plugin-node-resolve": "^7.1.3",

packages/clarity-devtools/src/content.ts

@@ -66,9 +66,8 @@ function proxy(): string {
   let closure = (): void => {
     let insertRule = CSSStyleSheet.prototype.insertRule;
     CSSStyleSheet.prototype.insertRule = function(style: string, index: number): number {
-        let value = insertRule.call(this, style, index);
         window.postMessage({ styleIndex: getStyleIndex(this), style, index }, "*");
-        return value;
+        return insertRule.apply(this, arguments);
     };
     function getStyleIndex(sheet: CSSStyleSheet): number {
         for (let i = 0; i < document.styleSheets.length; i++) {

packages/clarity-devtools/static/manifest.json

@@ -3,7 +3,7 @@
   "name": "Clarity Developer Tools",
   "description": "Get insights about how customers use your website.",
   "version": "0.6.0",
-  "version_name": "0.6.0-beta.22",
+  "version_name": "0.6.0",
   "minimum_chrome_version": "50",
   "devtools_page": "devtools.html",
   "icons": {

packages/clarity-js/README.md

@@ -1,12 +1,11 @@
 # Clarity
-Clarity is a behavioral analytics library written in javascript. It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
+Clarity is an open-source behavioral analytics library written in typescript, with two key goals: privacy & performance. 
 
-Clarity provides you all these insights by:
-* Observing content layout, viewport, and user's interactions with the page
-* Inspecting network requests on the page
-* Logging the event stream in JSON format to a configurable endpoint
+It helps you understand how users view and use your website across all modern devices and browsers. Understanding how users navigate, interact and browse your website can provide new insights about your users. Empathizing with your users and seeing where features fail or succeed can help improve your product, grow revenue and improve user retention.
 
-Clarity is a project in active development. While it's not yet ready for production use, we continue making improvements and encourage the community to join us in the process.
+It's the same code that powers Microsoft's hosted behavioral analytics solution: <a href="https://clarity.microsoft.com">https://clarity.microsoft.com</a>. If you would like to see a demo of how it works, checkout <a href="https://clarity.microsoft.com/demo/projects/view/3t0wlogvdz/impressions?date=Last%203%20days">live demo</a>. 
+
+We encourage the community to join us in building the best behavioral analytics library, that puts privacy first and prioritizes performance. 
 
 ## Examples
 Here are some example sessions on popular websites visualized to demonstrate the telemetry captured:
@@ -16,14 +15,8 @@ Here are some example sessions on popular websites visualized to demonstrate the
 2. Cook with Manali (Mobile)
 </br><a href="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif"><img src="https://thumbs.gfycat.com/CoolDependableAdamsstaghornedbeetle-size_restricted.gif" title="Clarity - Cook With Manali Example"/></a> 
 
-## Project Goals
-* Enable a generic solution that is able to capture telemetry from third-party websites
-* Encourage participation from open-source community
-* Minimal configuration required by third party web-sites to get started
-* Mobile first
-
 ## Privacy Notice
-Clarity handles sensitive data with care. By default content on the page is masked before upload, so no actual text from the page is sent to the server.
+Clarity handles sensitive data with care. By default sensitive content on the page is masked before uploading to the server.
 
 ## Improving Clarity
 If you haven't already done so, start contributing by following instructions **[here](https://github.com/microsoft/clarity/blob/master/CONTRIBUTING.md)**.

packages/clarity-js/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clarity-js",
-  "version": "0.6.0-beta.22",
+  "version": "0.6.0",
   "description": "An analytics library that uses web page interactions to generate aggregated insights",
   "author": "Microsoft Corp.",
   "license": "MIT",
@@ -28,21 +28,27 @@
   "devDependencies": {
     "@rollup/plugin-commonjs": "^11.1.0",
     "@rollup/plugin-node-resolve": "^7.1.3",
+    "@types/chai": "^4.2.14",
+    "@types/mocha": "^8.0.4",
     "@types/resize-observer-browser": "^0.1.3",
+    "chai": "^4.2.0",
     "del-cli": "3.0.0",
     "husky": "4.2.3",
     "lint-staged": "10.1.2",
+    "mocha": "^8.2.1",
+    "playwright": "^1.6.2",
     "rollup": "^2.7.3",
     "rollup-plugin-terser": "^5.3.0",
     "rollup-plugin-typescript2": "^0.27.0",
-    "ts-node": "8.8.2",
+    "ts-mocha": "^8.0.0",
     "tslint": "6.1.1",
     "typescript": "^3.8.3"
   },
   "scripts": {
     "build": "yarn build:clean && yarn build:main",
     "build:main": "rollup -c rollup.config.ts",
     "build:clean": "del-cli build/*",
+    "test": "ts-mocha -p test/tsconfig.test.json test/**/*.test.ts",
     "tslint": "tslint --project ./",
     "tslint:fix": "tslint --fix --project ./ --force"
   },

packages/clarity-js/src/core/version.ts

@@ -1,2 +1,2 @@
-let version = "0.6.0-b22";
+let version = "0.6.0";
 export default version;

packages/clarity-js/src/data/metadata.ts

@@ -64,21 +64,25 @@ export function consent(): void {
 
 export function clear(): void {
   // Clear any stored information in the session so we can restart fresh the next time
-  if (sessionStorage) { sessionStorage.removeItem(Constant.StorageKey); }
+  if (supported(window, Constant.SessionStorage)) { sessionStorage.removeItem(Constant.StorageKey); }
 }
 
 export function save(): void {
   let ts = Math.round(Date.now());
   let upgrade = config.lean ? BooleanFlag.False : BooleanFlag.True;
   let upload = typeof config.upload === Constant.String ? config.upload : Constant.Empty;
   if (upgrade && callback) { callback(data, !config.lean); }
-  if (config.track && sessionStorage) {
+  if (config.track && supported(window, Constant.SessionStorage)) {
     sessionStorage.setItem(Constant.StorageKey, [data.sessionId, ts, data.pageNum, upgrade, upload].join(Constant.Separator));
   }
 }
 
+function supported(target: Window | Document, api: string): boolean {
+  try { return !!target[api]; } catch { return false; }
+}
+
 function track(): void {
-  if (config.track) {
+  if (config.track && supported(document, Constant.Cookie)) {
     let expiry = new Date();
     expiry.setDate(expiry.getDate() + Setting.Expire);
     let expires = expiry ? Constant.Expires + expiry.toUTCString() : Constant.Empty;
@@ -97,7 +101,7 @@ function shortid(): string {
 
 function session(): Session {
   let output: Session = { id: shortid(), ts: Math.round(Date.now()), count: 1, upgrade: BooleanFlag.False, upload: Constant.Empty };
-  if (config.track && sessionStorage) {
+  if (config.track && supported(window, Constant.SessionStorage)) {
     let value = sessionStorage.getItem(Constant.StorageKey);
     if (value && value.indexOf(Constant.Separator) >= 0) {
       let parts = value.split(Constant.Separator);
@@ -122,12 +126,14 @@ function user(): string {
 }
 
 function cookie(key: string): string {
-  let cookies: string[] = document.cookie.split(Constant.Semicolon);
-  if (cookies) {
-    for (let i = 0; i < cookies.length; i++) {
-      let pair: string[] = cookies[i].split(Constant.Equals);
-      if (pair.length > 1 && pair[0] && pair[0].trim() === key) {
-        return pair[1];
+  if (supported(document, Constant.Cookie)) {
+    let cookies: string[] = document.cookie.split(Constant.Semicolon);
+    if (cookies) {
+      for (let i = 0; i < cookies.length; i++) {
+        let pair: string[] = cookies[i].split(Constant.Equals);
+        if (pair.length > 1 && pair[0] && pair[0].trim() === key) {
+          return pair[1];
+        }
       }
     }
   }