Skip to content

Commit

Permalink
feat: add dgate-cli with new pkg/dgclient
Browse files Browse the repository at this point in the history
  • Loading branch information
bubbajoe committed Apr 20, 2024
1 parent 8fa7f2c commit 391f662
Show file tree
Hide file tree
Showing 41 changed files with 1,594 additions and 335 deletions.
12 changes: 0 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,3 @@ Production Ready Checklist:
- [x] modify json response
- [x] send multiple requests and combine the response

- Add Module Permissions Functionality
- Add Change Versioning
- Add Change Rollback
- Add Server Tags
- Add Transactions
- [ ] - Add transactional support for admin API

## Release Checklist
- Test multiple modules being used at the same time
- [ ] - ??? Add option to specify specific export variables
- [ ] - check for module export conflicts from different modules
- [ ] - check for global variable conflicts
126 changes: 77 additions & 49 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,97 @@
# TODO List

> `???` - maybe not needed
# dgate-cli

- add dgate-cli
- resource management (CRUD for namespace, domain, service, ...)
- server management (start-proxy, stop-proxy, restart, status, logs, stats, etc.)
- cluster management (raft commands, replica commands, etc.) (low priority)
- other commands (backup, restore, etc.) (low priority)
- replace httpie usage in scripts with dgate-cli

## Replace zerolog with slog

## Improve async function performance

There is a pretty significant difference in performance when using async function.


## Add Module Tests

- Test multiple modules being used at the same time
- [ ] - automatically detect export conflicts
- [ ] - Add option to specify export variables when ambiguous (?)
- [ ] - check how global variable conflicts are handled

## Start using Pkl

replace dgate server config with pkl.

## dgate-cli declaritive config

define resources in a declaritive way using pkl.

## Background Jobs

background jobs can be used to execute code periodically or on specific events. custom events can be triggered from inside modules.

For examples, upstream health checks to make sure that the upstream server is still available.
Also to execute code on specific events, like when a new route is added, or when an http requ.

- event listeners: intercept events and return modified/new data
- fetch (http requests made by the proxy)
- request (http requests made to the proxy)
- resource CRUD operations (namespace/domain/service/module/route/collection/document)
- execute cron jobs: @every 1m, @cron 0 0 * * *, @daily, @weekly, @monthly, @yearly

# Metrics

expose metrics for the following:
- proxy server
- request count
- request latency
- request module latency
- request upstream latency
- request error count
- admin server
- request error latency
- request count
- request latency
- request error count
- modules
- request count
- request latency
- request error count
- request error latency

- Add Module Permissions Functionality
- Add Change Versioning
- Add Change Rollback
- Add Server Tags
- Add Transactions
- [ ] - Add transactional support for admin API

## DGate Documentation (dgate.io/docs)

Use Docusaurus to create the documentation for DGate.

## DGate Admin Console
## DGate Admin Console (low priority)

Admin Console is a web-based interface that can be used to manage the state of the cluster. Manage resource, view logs, stats, and more. It can also be used to develop and test modules directly in the browser.

## DGate Runtime (`???`)
## DGate Runtime (low priority)

DGate Runtime is a JavaScript/TypeScript runtime that can be used to test modules. It can be used to test modules before deploying them to the cluster.

## RuntimePool
## Implement an optimal RuntimePool (low priority)

RuntimePool is a pool of runtimes that can be used to execute modules. It can be used to manage the different modules and clean up resources when they are no longer needed or idle for a certain amount of time.

## TCP Proxy/Gateway (L4LB) (`???`)
## TCP Proxy/Gateway (L4LB) (low priority)

Using the same architecture as the HTTP Proxy/Gateway, create a TCP Proxy/Gateway that can be used to proxy TCP connections to upstream servers.

### Custom Protocols API

Allow users to define custom protocols that can be used to proxy TCP connections to upstream servers or handle the connections themselves.
A 'Custom Protocols API'can allow users to define custom protocols that can be used to proxy TCP connections to upstream servers or handle the connections themselves.

The custom protocols can be defined using JavaScript/TypeScript function or using protocol definitions (API) which will allow these values to be passed to the JavaScript/TypeScript code.

Expand Down Expand Up @@ -77,45 +144,6 @@ time based tags
- ?*name:data1,data2
- means that the server must have *one (and only one)* of these tags, for the object to be applied

## Background Jobs (`???`)

background jobs can be used to execute code periodically or on specific events. custom events can be triggered from inside modules.

For examples, upstream health checks to make sure that the upstream server is still available.
Also to execute code on specific events, like when a new route is added, or when an http requ.

- event listeners: intercept events and return modified/new data
- fetch (http requests made by the proxy)
- request (http requests made to the proxy)
- resource CRUD operations (namespace/domain/service/module/route/collection/document)
- execute cron jobs: @every 1m, @cron 0 0 * * *, @daily, @weekly, @monthly, @yearly
Path Parameters Extraction
Domain Parameters Extraction

## Replace zerolog with slog

## Add Module Permissions Functionality

## Improve async function performance

There is a pretty significant difference in performance when using async function.

# Metrics
## Module Permissions

expose metrics for the following:
- proxy server
- request count
- request latency
- request module latency
- request upstream latency
- request error count
- admin server
- request error latency
- request count
- request latency
- request error count
- modules
- request count
- request latency
- request error count
- request error latency
- Allow users to define permissions for modules to access certain dgate resources/apis and/or OS resources.
39 changes: 0 additions & 39 deletions cmd/dgate-cli/cmd.go

This file was deleted.

91 changes: 91 additions & 0 deletions cmd/dgate-cli/commands/collection.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
package commands

import (
"fmt"

"github.com/dgate-io/dgate/pkg/dgclient"
"github.com/dgate-io/dgate/pkg/spec"
"github.com/urfave/cli/v2"
)

func CollectionCommand(client *dgclient.DGateClient) *cli.Command {
return &cli.Command{
Name: "collection",
Aliases: []string{"col"},
Args: true,
ArgsUsage: "<command> <name>",
Usage: "collection commands",
Subcommands: []*cli.Command{
{
Name: "create",
Aliases: []string{"mk"},
Usage: "create a collection",
Action: func(ctx *cli.Context) error {
col, err := createMapFromArgs[spec.Collection](
ctx.Args().Slice(), "name", "schema",
)
if err != nil {
return err
}
fmt.Println(col, client.BaseUrl())
err = client.CreateCollection(col)
if err != nil {
return err
}
return jsonPrettyPrint(col)
},
},
{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete a collection",
Action: func(ctx *cli.Context) error {
col, err := createMapFromArgs[spec.Collection](
ctx.Args().Slice(), "name",
)
if err != nil {
return err
}
err = client.DeleteCollection(
col.Name, col.NamespaceName,
)
if err != nil {
return err
}
return nil
},
},
{
Name: "list",
Aliases: []string{"ls"},
Usage: "list collections",
Action: func(ctx *cli.Context) error {
col, err := client.ListCollection()
if err != nil {
return err
}
return jsonPrettyPrint(col)
},
},
{
Name: "get",
Usage: "get a collection",
Action: func(ctx *cli.Context) error {
col, err := createMapFromArgs[spec.Collection](
ctx.Args().Slice(), "name",
)
if err != nil {
return err
}
col, err = client.GetCollection(
col.Name, col.NamespaceName,
)
if err != nil {
return err
}
return jsonPrettyPrint(col)
},
},
},
}
}
92 changes: 92 additions & 0 deletions cmd/dgate-cli/commands/document.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package commands

import (
"fmt"

"github.com/dgate-io/dgate/pkg/dgclient"
"github.com/dgate-io/dgate/pkg/spec"
"github.com/urfave/cli/v2"
)

func DocumentCommand(client *dgclient.DGateClient) *cli.Command {
return &cli.Command{
Name: "document",
Aliases: []string{"doc"},
Args: true,
ArgsUsage: "<command> <name>",
Usage: "document commands",
Subcommands: []*cli.Command{
{
Name: "create",
Aliases: []string{"mk"},
Usage: "create a document",
Action: func(ctx *cli.Context) error {
doc, err := createMapFromArgs[spec.Document](
ctx.Args().Slice(), "id",
"collection", "data",
)
if err != nil {
return err
}
fmt.Println(doc, client.BaseUrl())
err = client.CreateDocument(doc)
if err != nil {
return err
}
return jsonPrettyPrint(doc)
},
},
{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete a document",
Action: func(ctx *cli.Context) error {
doc, err := createMapFromArgs[spec.Document](
ctx.Args().Slice(), "id",
)
if err != nil {
return err
}
err = client.DeleteDocument(
doc.ID, doc.NamespaceName,
)
if err != nil {
return err
}
return nil
},
},
{
Name: "list",
Aliases: []string{"ls"},
Usage: "list documents",
Action: func(ctx *cli.Context) error {
doc, err := client.ListDocument()
if err != nil {
return err
}
return jsonPrettyPrint(doc)
},
},
{
Name: "get",
Usage: "get a document",
Action: func(ctx *cli.Context) error {
doc, err := createMapFromArgs[spec.Document](
ctx.Args().Slice(), "id",
)
if err != nil {
return err
}
doc, err = client.GetDocument(
doc.ID, doc.NamespaceName,
)
if err != nil {
return err
}
return jsonPrettyPrint(doc)
},
},
},
}
}
Loading

0 comments on commit 391f662

Please sign in to comment.