Proctor is a set of components that allow user to do automated task with configurable access policy.
Bundle repetitive task as a automation and turn it into procs
to make it easier for user to do it themself.
Before we goes deep down into explanation about proctor, you may want to read about Proctor Glossary
This section provide installation for unix environment.
General step
- Install and setup golang
- Clone this repository
- Run
make build
. This will generate binary for proctor cli and service
For proctor service
- Make sure you have running Redis server
- Make sure you have running Postgres server
- Make sure you have running Kubernetes Cluster, for setting up local cluster, please refer here
- Copy
.env.sample
into.env
file. Please refer here for configuration explanation - Make sure you set correct value in
.env
for Kubernetes, Postgresql, and Redis - Export value of
.env
by runningsource .env
- Run
./_output/bin/server s
to start proctor service
For proctor cli
- Run
./_output/bin/cli config PROCTOR_HOST=<proctor-service-host>
to point you proctor cli to local proctor service - Run
./_output/bin/cli
to see complete usage of proctor cli
Here's the overview of proctor components.
Proctor cli is a command line interface that used by client to interact with Proctor service.
Proctor cli read user configuration such as Proctor service host, user email, and user token from ~/.proctor/proctor.yaml
.
Proctor service govern the main process of Proctor such as:
- Create execution context
- Create and read procs metadata
- Create and read procs secret
- Order the execution of procs
- Get execution status and log of running procs
Currently Proctor service use postgres to store execution context of procs.
Metadata store contain all procs metadata, procs that doesn't have metadata on store cannot be executed.
Secret store contain secret value that needed by procs to executed.
Executor is the one that executing the procs, we use Kubernetes Job as executor. Proctor service will send the procs name, needed args then executor will pull necessary image to run the procs. Proctor service will occasionally contact executor to get status of requested procs.
Here's what happen between Proctor components when client want to execute a procs.
- Cli send execution request to service. This request consist of procs name, procs args, and user credentials.
- Service get metadata and secret for requested procs.
- Service create execution context to store data related to procs execution.
- Service tell the executor to run the procs image along with user argument and procs secret.
- Service watch the process run by executor by getting the log and execution status then write it to execution context.
Some route is protected by authentication, authorization or both.
Authenticated user means that user should have account related with proctor.
Authorized user means that user should be part of groups that defined on procs meatadata, for example when procs authorized groups is proctor-user
, and dev
then user need to be a member of both groups.
A request need head these headers to pass auth process:
'Access-Token: <user-access-token>'
'Email-Id: <user-email>'
List of routes that require authentication:
- POST /execution
- GET /execution/{contextId}/status
- GET /execution/logs
- GET /metadata
- POST /metadata
- POST /secret
- POST /schedule
- GET /schedule
- GET /schedule/{scheduleID}
- DELETE /schedule/{scheduleID}
List of routes that require authorization:
- POST /execution
- POST /schedule
Proctor doesn't come with built in auth implementation, it's using configurable plugin mechanism.
Proctor service support plugin for authentication and authorization using go plugin. It create limitation that proctor service can only be used on Linux and MacOS (Until the time go plugin support other OS).
By using authentication and authorization as a plugin you can customize this process using your organization current user management.
Creating plugin for auth (authentication and authorization) means creating a struct that implement Auth interface then initialize it. Begin using your custom plugin by building it with below command:
go build -race -buildmode=plugin -o ./_output/bin/plugin/auth.so <your-plugin-file>
Plug your custom plugin to proctor service by pointing the PROCTOR_AUTH_PLUGIN_EXPORTED
environment variable to variable exported by plugin and PROCTOR_AUTH_PLUGIN_BINARY
to auth.so
file generated by command above.
After you finish it then proctor service will use your plugin.
If you like to jump in directly to example implementation then you can explore our default auth plugin here. This implementation is using Gate as its user management.
You can read here to learn more about creating procs.
PROCTOR_APP_PORT
is port on which service will runPROCTOR_LOG_LEVEL
defines log levels of service. Available options are:debug
,info
,warn
,error
,fatal
,panic
PROCTOR_REDIS_ADDRESS
is hostname and port of redis store for jobs configuration and metadataPROCTOR_REDIS_MAX_ACTIVE_CONNECTIONS
defines maximum active connections to redis. Maximum idle connections is half of this configPROCTOR_LOGS_STREAM_READ_BUFFER_SIZE
andPROCTOR_LOGS_STREAM_WRITE_BUFFER_SIZE
is the buffer size for websocket connection while streaming logsPROCTOR_KUBE_CONFIG
needs to be set only if service is running outside a kubernetes cluster- If unset, service will execute jobs in the same kubernetes cluster where it is run
- When set to "out-of-cluster", service will fetch kube config based on current-context from
.kube/config
file in home directory
- If a job doesn't reach completion, it is terminated after
PROCTOR_KUBE_JOB_ACTIVE_DEADLINE_SECONDS
PROCTOR_KUBE_JOB_RETRIES
is the number of retries for a kubernetes job (on failure)PROCTOR_DEFAULT_NAMESPACE
is the namespace under which jobs will be run in kubernetes cluster. By default, K8s has namespace "default". If you set another value, please create namespace in K8s before deployingproctord
PROCTOR_KUBE_CONTEXT
is used the name of context you want to use when running out of cluster.PROCTOR_KUBE_CLUSTER_HOST_NAME
is address/ip address to api-server of kube cluster. It is used for fetching logs of a pod using httpsPROCTOR_KUBE_CA_CERT_ENCODED
is the CA cert file encoded in base64. This is used for establishing authority while talking to kubernetes api-server on a public https callPROCTOR_KUBE_BASIC_AUTH_ENCODED
is the base64 encoded authentication of kubernetes. Enocdeusername:password
to base64 and set this config.- Before streaming logs of jobs,
PROCTOR_KUBE_POD_LIST_WAIT_TIME
is the time to wait until jobs and pods are in active/successful/failed state PROCTOR_POSTGRES_USER
,PROCTOR_POSTGRES_PASSWORD
,PROCTOR_POSTGRES_HOST
andPROCTOR_POSTGRES_PORT
is the username and password to the postgres database you wish to connect to- Set
PROCTOR_POSTGRES_DATABASE
toproctord_development
for development purpose - Create database
PROCTOR_POSTGRES_DATABASE
PROCTOR_POSTGRES_MAX_CONNECTIONS
defines maximum open and idle connections to postgresPROCTOR_POSTGRES_CONNECTIONS_MAX_LIFETIME
is the lifetime of a connection in minutesPROCTOR_NEW_RELIC_APP_NAME
andPROCTOR_NEW_RELIC_LICENCE_KEY
are used to send profiling details to newrelic. Provide dummy values if you don't want profilingPROCTOR_MIN_CLIENT_VERSION
is minimum client version allowed to communicate with proctordPROCTOR_SCHEDULED_JOBS_FETCH_INTERVAL_IN_MINS
is the interval at which the scheduler fetches updated jobs from databasePROCTOR_MAIL_USERNAME
,PROCTOR_MAIL_PASSWORD
,PROCTOR_MAIL_SERVER_HOST
,PROCTOR_MAIL_SERVER_PORT
are the creds required to send notification to users on scheduled jobs executionPROCTOR_JOB_POD_ANNOTATIONS
is used to set any kubernetes pod specific annotations.PROCTOR_SENTRY_DSN
is used to set sentry DSN.