Skip to content

Commit

Permalink
[Docs] Another batch of docs improvements #1497 (#1498)
Browse files Browse the repository at this point in the history
  • Loading branch information
peterschmidt85 authored Aug 5, 2024
1 parent bb324fc commit b28ca4a
Show file tree
Hide file tree
Showing 41 changed files with 1,274 additions and 627 deletions.
90 changes: 47 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,55 +14,43 @@

</div>

`dstack` is an open-source container orchestration engine for AI.
It accelerates the development, training, and deployment of AI models, and simplifies the management of clusters.
`dstack` is a lightweight alternative to Kubernetes, designed specifically for managing the development, training, and
deployment of AI models at any scale.

#### Cloud and on-prem
`dstack` is easy to use with any cloud provider (AWS, GCP, Azure, OCI, Lambda, TensorDock, Vast.ai, RunPod, etc.) or
any on-prem clusters.

`dstack` is easy to use with any cloud or on-prem servers.
Supported cloud providers include AWS, GCP, Azure, OCI, Lambda, TensorDock, Vast.ai, RunPod, and CUDO.
For using `dstack` with on-prem servers, see [fleets](https://dstack.ai/docs/fleets#__tabbed_1_2).
If you already use Kubernetes, `dstack` can be used with it.

#### Accelerators

`dstack` supports `NVIDIA GPU` and `Google Cloud TPU` out of the box.

## Major news ✨

- [2024/07] [dstack 0.18.7: Fleets, RunPod Volumes, dstack apply, and more](https://github.com/dstackai/dstack/releases/tag/0.18.7) (Release)
- [2024/07] [dstack 0.18.8: GCP volumes](https://github.com/dstackai/dstack/releases/tag/0.18.8) (Release)
- [2024/07] [dstack 0.18.7: Fleets, RunPod volumes, dstack apply, and more](https://github.com/dstackai/dstack/releases/tag/0.18.7) (Release)
- [2024/05] [dstack 0.18.4: Google Cloud TPU, and more](https://github.com/dstackai/dstack/releases/tag/0.18.4) (Release)
- [2024/05] [dstack 0.18.3: OCI, and more](https://github.com/dstackai/dstack/releases/tag/0.18.3) (Release)
- [2024/05] [dstack 0.18.2: On-prem clusters, private subnets, and more](https://github.com/dstackai/dstack/releases/tag/0.18.2) (Release)
- [2024/04] [dstack 0.18.0: RunPod, multi-node tasks, and more](https://github.com/dstackai/dstack/releases/tag/0.18.0) (Release)

## Installation

Before using `dstack` through CLI or API, set up a `dstack` server.

### Install the server

The easiest way to install the server, is via `pip`:

```shell
pip install "dstack[all]" -U
```

### Configure backends
### 1. Configure backends

If you have default AWS, GCP, Azure, or OCI credentials on your machine, the `dstack` server will pick them up automatically.
If you want the `dstack` server to run containers or manage clusters in your cloud accounts (or use Kubernetes),
create the [~/.dstack/server/config.yml](https://dstack.ai/docs/reference/server/config.yml.md) file and configure backends.

Otherwise, you need to manually specify the cloud credentials in `~/.dstack/server/config.yml`.
### 2. Start the server

See the [server/config.yml reference](https://dstack.ai/docs/reference/server/config.yml.md#examples)
for details on how to configure backends for all supported cloud providers.

### Start the server

To start the server, use the `dstack server` command:
Once the `~/.dstack/server/config.yml` file is configured, proceed to start the server:

<div class="termy">

```shell
$ pip install "dstack[all]" -U
$ dstack server

Applying ~/.dstack/server/config.yml...
Expand All @@ -76,42 +64,58 @@ The server is running at http://127.0.0.1:3000/
> **Note**
> It's also possible to run the server via [Docker](https://hub.docker.com/r/dstackai/dstack).
### Add on-prem servers
The `dstack` server can run anywhere: on your laptop, a dedicated server, or in the cloud. Once it's up, you
can use either the CLI or the API.

### 3. Set up the CLI

To point the CLI to the `dstack` server, configure it
with the server address, user token, and project name:

```shell
$ pip install dstack
$ dstack config --url http://127.0.0.1:3000 \
--project main \
--token bbae0f28-d3dd-4820-bf61-8f4bb40815da

Configuration is updated at ~/.dstack/config.yml
```

### 4. Create on-prem fleets

If you'd like to use `dstack` to run workloads on your on-prem servers,
see [on-prem fleets](https://dstack.ai/docs/fleets#__tabbed_1_2) command.
> If you want the `dstack` server to run containers on your on-prem servers,
use [fleets](../fleets.md#__tabbed_1_2).

## How does it work?

### 1. Define run configurations
> Before using `dstack`, [install](https://dstack.ai/docs/installation/index.md) the server and configure backends.
`dstack` supports three types of run configurations:
### 1. Define configurations

`dstack` supports the following configurations:

* [Dev environments](https://dstack.ai/docs/dev-environments.md) &mdash; for interactive development using a desktop IDE
* [Tasks](https://dstack.ai/docs/tasks.md) &mdash; for any kind of batch jobs or web applications (supports distributed jobs)
* [Services](https://dstack.ai/docs/services.md)&mdash; for production-grade deployment (supports auto-scaling and authorization)

Each type of run configuration allows you to specify commands for execution, required compute resources, retry policies, auto-scaling rules, authorization settings, and more.
* [Tasks](https://dstack.ai/docs/tasks.md) &mdash; for scheduling jobs (incl. distributed jobs) or running web apps
* [Services](https://dstack.ai/docs/services.md) &mdash; for deployment of models and web apps (with auto-scaling and authorization)
* [Fleets](https://dstack.ai/docs/fleets.md) &mdash; for managing cloud and on-prem clusters
* [Volumes](https://dstack.ai/docs/concepts/volumes.md) &mdash; for managing persisted volumes
* [Gateways](https://dstack.ai/docs/concepts/volumes.md) &mdash; for configuring the ingress traffic and public endpoints

Configuration can be defined as YAML files within your repo.

### 2. Run configurations

Run any defined configuration either via `dstack` CLI or API.

`dstack` automatically handles provisioning, interruptions, port-forwarding, auto-scaling, network, volumes,
run failures, out-of-capacity errors, and more.
### 2. Apply configurations

### 3. Manage fleets
Apply the configuration either via the `dstack apply` CLI command or through a programmatic API.

Use [fleets](https://dstack.ai/docs/fleets.md) to provision and manage clusters and instances, both in the cloud and on-prem.
`dstack` automatically manages provisioning, job queuing, auto-scaling, networking, volumes, run failures,
out-of-capacity errors, port-forwarding, and more &mdash; across clouds and on-prem clusters.

## More information

For additional information and examples, see the following links:

* [Docs](https://dstack.ai/docs)
* [Examples](examples)
* [Examples](https://dstack.ai/docs/examples)
* [Changelog](https://github.com/dstackai/dstack/releases)
* [Discord](https://discord.gg/u8SmfwPpMd)

Expand Down
2 changes: 1 addition & 1 deletion docs/assets/stylesheets/extra.css
Original file line number Diff line number Diff line change
Expand Up @@ -598,7 +598,7 @@ code .md-code__nav:hover .md-code__button {
}

.md-typeset h5 {
font-size: 16px;
font-size: 18px;
}

.md-typeset h3 {
Expand Down
41 changes: 23 additions & 18 deletions docs/docs/concepts/gateways.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# Gateways

Gateways handle the ingress traffic of running services.
They provide [services](services.md) with HTTPS domains, handle authentication, distribute load, and perform auto-scaling.
In order to run a service, you need to have at least one gateway set up.
Gateways manage the ingress traffic of running services and provide them with an HTTPS endpoint mapped to your domain,
handling authentication, load distribution, and auto-scaling.

To run a service, you need at least one gateway set up.

> If you're using [dstack Sky :material-arrow-top-right-thin:{ .external }](https://sky.dstack.ai){:target="_blank"},
the gateway is already set up for you.
> the gateway is already set up for you.
## Configuration
## Define a configuration

First, create a YAML file in your project folder. Its name must end with `.dstack.yml` (e.g. `.dstack.yml` or `gateway.dstack.yml`
are both acceptable).
Expand All @@ -16,10 +17,14 @@ are both acceptable).

```yaml
type: gateway
# A name of the gateway
name: example-gateway

# Gateways are bound to a specific backend and region
backend: aws
region: eu-west-1

# This domain will be used to access the endpoint
domain: example.com
```
Expand All @@ -28,18 +33,17 @@ domain: example.com
A domain name is required to create a gateway.
!!! info "Reference"
See the [.dstack.yml reference](../reference/dstack.yml/gateway.md)
for all supported configuration options and examples.
See [.dstack.yml](../reference/dstack.yml/gateway.md) for all the options supported by
gateways, along with multiple examples.
## Creating and updating gateways
## Create or update a gateway
To create or update the gateway, simply call the [`dstack apply`](../reference/cli/index.md#dstack-apply) command:

<div class="termy">

```shell
$ dstack apply . -f examples/deployment/gateway.dstack.yml
The example-gateway doesn't exist. Create it? [y/n]: y
BACKEND REGION NAME HOSTNAME DOMAIN DEFAULT STATUS
Expand All @@ -49,36 +53,37 @@ The example-gateway doesn't exist. Create it? [y/n]: y

</div>

## Updating DNS records
## Update DNS records

Once the gateway is assigned a hostname, go to your domain's DNS settings
and add an `A` DNS record for `*.<gateway domain>` (e.g., `*.example.com`) pointing to the gateway's hostname.

This will allow you to access runs and models using this domain.
## Manage gateways

## Managing gateways

### Listing gateways
### List gateways

The [`dstack gateway list`](../reference/cli/index.md#dstack-gateway-list) command lists existing gateways and their status.

### Deleting gateways
### Delete a gateway

To delete a gateway, pass gateway configuration to [`dstack delete`](../reference/cli/index.md#dstack-delete):

<div class="termy">

```shell
$ dstack delete . -f examples/deployment/gateway.dstack.yml
$ dstack delete -f examples/deployment/gateway.dstack.yml
```

</div>

[//]: # (TODO: Ellaborate on default`)
[//]: # (TODO: Ellaborate on default)

[//]: # (TODO: ## Accessing endpoints)

## What's next?

1. See [services](../services.md) on how to run services
2. Check the [`.dstack.yml` reference](../reference/dstack.yml/gateway.md) for more details and examples

!!! info "Reference"
See [.dstack.yml](../reference/dstack.yml/gateway.md) for all the options supported by
gateways, along with multiple examples.
55 changes: 34 additions & 21 deletions docs/docs/concepts/volumes.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Volumes

Volumes allow you to persist data between runs. `dstack` simplifies managing volumes and lets you mount them to a specific
directory when working with dev environments, tasks, and services.
Volumes allow you to persist data between runs. `dstack` allows to create and attach volumes to
dev environments, tasks, and services.

!!! info "Experimental"
Volumes are currently experimental and work with the `aws`, `gcp`, and `runpod` backends.
Support for other backends is coming soon.

## Configuration
## Define a configuration

First, create a YAML file in your project folder. Its name must end with `.dstack.yml` (e.g. `.dstack.yml` or `vol.dstack.yml`
are both acceptable).
Expand All @@ -16,9 +16,14 @@ are both acceptable).

```yaml
type: volume
# A name of the volume
name: my-new-volume

# Volumes are bound to a specific backend and region
backend: aws
region: eu-central-1

# Required size
size: 100GB
```
Expand All @@ -28,13 +33,13 @@ If you use this configuration, `dstack` will create a new volume based on the sp

!!! info "Registering existing volumes"
If you prefer not to create a new volume but to reuse an existing one (e.g., created manually), you can
[specify its ID via `volume_id`](../reference/dstack.yml/volume.md#register-volume). In this case, `dstack` will register the specified volume so that you can use it with dev environments, tasks, and services.
[specify its ID via `volume_id`](../reference/dstack.yml/volume.md#existing-volume). In this case, `dstack` will register the specified volume so that you can use it with dev environments, tasks, and services.

!!! info "Reference"
See the [.dstack.yml reference](../reference/dstack.yml/dev-environment.md)
for all supported configuration options and examples.
See [.dstack.yml](../reference/dstack.yml/volume.md) for all the options supported by
volumes, along with multiple examples.

## Creating and registering volumes
## Create, register, or update a volume

To create or register the volume, simply call the `dstack apply` command:

Expand All @@ -43,6 +48,7 @@ To create or register the volume, simply call the `dstack apply` command:
```shell
$ dstack apply -f volume.dstack.yml
Volume my-new-volume does not exist yet. Create the volume? [y/n]: y
NAME BACKEND REGION STATUS CREATED
my-new-volume aws eu-central-1 submitted now
Expand All @@ -54,7 +60,7 @@ Volume my-new-volume does not exist yet. Create the volume? [y/n]: y

Once created, the volume can be attached with dev environments, tasks, and services.

## Attaching volumes
## Attach a volume

Dev environments, tasks, and services let you attach any number of volumes.
To attach a volume, simply specify its name using the `volumes` property and specify where to mount its contents:
Expand All @@ -63,7 +69,12 @@ To attach a volume, simply specify its name using the `volumes` property and spe

```yaml
type: dev-environment
# A name of the dev environment
name: vscode-vol
ide: vscode
# Map the name of the volume to any path
volumes:
- name: my-new-volume
path: /volume_data
Expand All @@ -79,9 +90,9 @@ and its contents will persist across runs.
to `/workflow` (and sets that as the current working directory). Right now, `dstack` doesn't allow you to
attach volumes to `/workflow` or any of its subdirectories.

## Managing volumes
## Manage volumes

### Listing volumes
### List volumes

The [`dstack volume list`](../reference/cli/index.md#dstack-gateway-list) command lists created and registered volumes:

Expand All @@ -91,7 +102,7 @@ NAME BACKEND REGION STATUS CREATED
my-new-volume aws eu-central-1 active 3 weeks ago
```
### Deleting volumes
### Delete volumes
When the volume isn't attached to any active dev environment, task, or service, you can delete it using `dstack delete`:
Expand All @@ -104,15 +115,17 @@ If you've registered an existing volume, it will be de-registered with `dstack`

## FAQ

??? info "Using volumes across backends"
Since volumes are backed up by cloud network disks, you can only use them within the same cloud. If you need to access
data across different backends, you should either use object storage or replicate the data across multiple volumes.
##### Can I use volumes across backends?

Since volumes are backed up by cloud network disks, you can only use them within the same cloud. If you need to access
data across different backends, you should either use object storage or replicate the data across multiple volumes.

##### Can I use volumes across regions?

Typically, network volumes are associated with specific regions, so you can't use them in other regions. Often,
volumes are also linked to availability zones, but some providers support volumes that can be used across different
availability zones within the same region.

??? info "Using volumes across regions"
Typically, network volumes are associated with specific regions, so you can't use them in other regions. Often,
volumes are also linked to availability zones, but some providers support volumes that can be used across different
availability zones within the same region.
##### Can I attach volumes to multiple runs or instances?

??? info "Attaching volumes to multiple runs and instances"
You can mount a volume in multiple runs.
This feature is currently supported only by the `runpod` backend.
You can mount a volume in multiple runs. This feature is currently supported only by the `runpod` backend.
Loading

0 comments on commit b28ca4a

Please sign in to comment.