Skip to content

Commit

Permalink
Initial new platform architecture
Browse files Browse the repository at this point in the history
  • Loading branch information
kimjoar committed Jun 19, 2017
1 parent 86d32da commit 983a7b9
Show file tree
Hide file tree
Showing 133 changed files with 6,842 additions and 36 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,5 @@ selenium
ui_framework/doc_site/build
!ui_framework/doc_site/build/index.html
yarn.lock
.vscode/
ts-tmp/
26 changes: 23 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,9 @@
"mocha": "echo 'use `node scripts/mocha`' && false",
"sterilize": "grunt sterilize",
"uiFramework:start": "grunt uiFramework:start",
"uiFramework:build": "grunt uiFramework:build"
"uiFramework:build": "grunt uiFramework:build",
"ts:build": "tsc",
"ts:start": "tsc --watch"
},
"repository": {
"type": "git",
Expand Down Expand Up @@ -109,6 +111,7 @@
"boom": "2.8.0",
"brace": "0.5.1",
"bunyan": "1.7.1",
"chalk": "1.1.3",
"check-hash": "1.0.1",
"color": "1.0.3",
"commander": "2.8.1",
Expand All @@ -123,6 +126,7 @@
"expiry-js": "0.1.7",
"exports-loader": "0.6.2",
"expose-loader": "0.7.0",
"express": "4.15.2",
"extract-text-webpack-plugin": "0.8.2",
"file-loader": "0.8.4",
"flot-charts": "0.8.3",
Expand Down Expand Up @@ -185,26 +189,40 @@
"rimraf": "2.4.3",
"rison-node": "1.0.0",
"rjs-repack-loader": "1.0.6",
"rxjs": "^5.4.0",
"script-loader": "0.6.1",
"semver": "5.1.0",
"style-loader": "0.12.3",
"tar": "2.2.0",
"tinygradient": "0.3.0",
"trunc-html": "1.0.2",
"trunc-text": "1.0.2",
"type-detect": "4.0.0",
"ui-select": "0.19.6",
"url-loader": "0.5.6",
"uuid": "3.0.1",
"validate-npm-package-name": "2.2.2",
"vision": "4.1.0",
"webpack": "github:elastic/webpack#fix/query-params-for-aliased-loaders",
"wreck": "6.2.0",
"yargs": "7.0.2",
"yauzl": "2.7.0"
},
"devDependencies": {
"@elastic/eslint-config-kibana": "0.6.1",
"@elastic/eslint-import-resolver-kibana": "0.8.1",
"@elastic/eslint-plugin-kibana-custom": "1.0.3",
"@types/chalk": "0.4.31",
"@types/chance": "^0.7.33",
"@types/elasticsearch": "^5.0.13",
"@types/express": "4.0.35",
"@types/jest": "^19.2.3",
"@types/js-yaml": "^3.5.31",
"@types/lodash": "3.10.1",
"@types/node": "6.0.68",
"@types/sinon": "1.16.36",
"@types/supertest": "^2.0.0",
"@types/yargs": "6.6.0",
"angular-mocks": "1.4.7",
"babel-eslint": "7.2.3",
"chai": "3.5.0",
Expand Down Expand Up @@ -243,8 +261,8 @@
"husky": "0.8.1",
"image-diff": "1.6.0",
"istanbul-instrumenter-loader": "0.1.3",
"jest": "20.0.0",
"jest-cli": "20.0.0",
"jest": "^20.0.4",
"jest-cli": "^20.0.4",
"jsdom": "9.9.1",
"karma": "1.7.0",
"karma-chrome-launcher": "2.1.1",
Expand Down Expand Up @@ -276,6 +294,8 @@
"supertest": "3.0.0",
"supertest-as-promised": "2.0.2",
"tree-kill": "1.1.0",
"ts-jest": "^20.0.6",
"typescript": "2.4.0",
"webpack-dev-server": "1.14.1"
},
"engines": {
Expand Down
107 changes: 107 additions & 0 deletions platform/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# New platform

## Dev setup

```
npm run ts:start
```

This builds the code into `./ts-tmp/` for now.

**NB** This will show a couple type errors, e.g. something like:

```
platform/server/elasticsearch/Cluster.ts(28,17): error TS2339: Property 'close' does not exist on type 'Client'.
platform/server/elasticsearch/Cluster.ts(29,23): error TS2339: Property 'close' does not exist on type 'Client'.
platform/server/http/SslConfig.ts(28,28): error TS2339: Property 'constants' does not exist on type 'typeof "crypto"'.
```

This is expected (for now), and it's related to some third-party types.

## VSCode

If you want to see what it looks like with fantastic editor support.

```
$ cat ~/.vscode/settings.json
// Place your settings in this file to overwrite default and user settings.
{
"typescript.tsdk": "./node_modules/typescript/lib",
"typescript.referencesCodeLens.enabled": true
}
```

## Running code

(Make sure to build the code first, e.g. `npm run ts:build` or `npm run ts:start`)

Start the server and plugins:

```
node scripts/platform.js
```

If you update `config/kibana.yml` to e.g. contain `pid.file: ./kibana.pid`
you'll also see it write the PID file. (You can do this while running and just
send a SIGHUP.)

With failure:

```
node scripts/platform.js -c ./config/kibana.dev.yml --port "test"
```

## Running tests

Run Jest:

```
node scripts/jest.js
```

(add `--watch` for re-running on change)


# TODOs

- Better Typescript dev+build setup, included in Webpack et al
- There are still a couple `require`s in the codebase we need to remove.
Some of them seem to be related to `default export` stuff.
- Use `import()` instead of `require` for plugins?
- Types cleanup
- Move types out of "local files" to prepare for external types. This is
already done for a couple things, but not at a proper scale.
- Logging
- Maybe we should look into having the LoggerAdapter implement the
LoggerFactory interface (+ keep the current context). That way you can
easily get a "deeper logger" from the one you already have.
- Log levels in config instead of `silent` et al.
- For the logging that happens before the `upgrade` we should probably just
stream the logs and just stderr them if upgrading fails. So we could start
with a default "StreamLogger" or something like that.
- Fork something like Winston? It feels like all these logging frameworks
are kinda weird and doesn't _really_ fit us, e.g. by having lots of
"default setup" that doesn't fit etc.
- Elasticsearch.js typings needs tons of improvement
- Make `headers` allowed
- `client.close` doesn't exist in the typings
- I think we should fork the existing one into `./packages` and let it
evolve with the rest of the code base.
- Explore ways of handling requests + Elasticsearch.js client
- See `withRequest` in `platform/server/elasticsearch/Cluster.ts`
- Collect coverage from `platform/plugins`
- Need to do way more work on error handling
- REST endpoints incl validation
- Config errors at startup
- Failure in long-running observables
- Maybe explore having "local readmes" that explain the different parts of the
system? E.g. in `./logger` we have a readme that explains how the logger
works. There is an example for the [plugin system][./server/plugins/README.md].
- Http server
- Stopping the HTTP server is async, which means we can have request that come
in at a "bad time", between stopping the http server and stopping Kibana.
- Validate headers too? Can't validate _all_, but you only receive the
headers you _have_ validated.
- Only `get` is implemented so far, and it's quite ugly, but it has tests.
I didn't do more here before we've talked more about how we want these
to work.
Loading

0 comments on commit 983a7b9

Please sign in to comment.