Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

distribution: architecture, security, cost estimation, creation, operations, and more #1284

Merged
merged 5 commits into from
Jul 28, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixup!
  • Loading branch information
slimsag committed Jul 27, 2020
commit 92be500d45c83e193e5e58325c91d39392af76d0
32 changes: 32 additions & 0 deletions handbook/engineering/distribution/managed/cost_estimation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Managed instances: cost estimation

The following describes how to **estimate** the infrastructure costs incurred for a managed instance. Managed instances are always created in completely isolated GCP projects, and as such it is easy to see the exact infrastructure cost breakdown for a given customer.

| Cost estimate | Description |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------|
| $388/mo min. varies | [VM instance](https://cloud.google.com/compute/vm-instance-pricing#n1_standard_machine_types), `n1-standard-16` min. in `us-central`. Consult [resource estimator](https://docs.sourcegraph.com/admin/install/resource_estimator) |
| $85/mo min. / $0.340/GB | Data disk, SSD/regional, 250G minimum - exact size depends on customer's repository sizes. |
| $34/mo | Boot disk, SSD/regional, 100G fixed size. |
| $85/mo / $0.026/GB | 13 weekly snapshots (taken weekly, retained for 90d) |
| $78/mo / $0.026/GB | 12 monthly upgrade data disk snapshots (taken once per month as part of upgrade, retained for 1yr). |
| $36/mo | [Cloud Load Balancing](https://cloud.google.com/vpc/network-pricing#lb), two rules required. |
| $12/mo | [External IP address](https://cloud.google.com/vpc/network-pricing#ipaddress), four required. |
| $1/mo + $0.045/GB | [Cloud NAT](https://cloud.google.com/vpc/network-pricing#nat-pricing), one required. |
| $6/mo + $0.75/million requests | [Cloud Armor](https://cloud.google.com/vpc/network-pricing#armor-pricing), one policy, one rule, and user-activity requests. |
| $1.42/GB | [Cloud Proxy](https://cloud.google.com/vpc/network-pricing#proxy-instance-charge), 3 are required. |
| $0.12/GB | [Internet egress traffic](https://cloud.google.com/vpc/network-pricing#internet_egress) from `us-central1`. |

Or, more succinctly:

| Cost estimate | Description |
|----------------|--------------------------------------------------------------------------|
| $282/mo | **data and snapshot storage costs for the first 250GB** |
| $56/mo | **network infrastructure costs** |
| +$388/mo min. | **n1-standard-16, or your VM instance type based on resource estimator** |
| +$1/GB storage | **For any additional storage above 250GB required.** |
| +$1.54/GB | **For each GB of network traffic to/from the instance.** |


Thus, the smallest managed instance supporting around ~300 repositories and ~100 users costs around $726/mo.

**IMPORTANT:** This is a _rough estimate_ to the best of our ability, infrastructure pricing estimates are not trivial to do extremely accurately and actual costs are always subject to change. The goal of this is to give you a rough estimate of costs, not an exact amount.
41 changes: 41 additions & 0 deletions handbook/engineering/distribution/managed/creation_process.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Creating a managed instance

Creating a new managed instance involves following the steps below.

1. Ask @stephen or @beyang to create a new GCP project `sourcegraph-managed-$COMPANY` and grant you IAM **Editor** role access.
1. Ask @beyang to enable billing in the GCP project.
1. Create GCP service account credentials:
- From console.cloud.google.com select the project > **APIs & Services** > **Credentials** > **Create credentials** > **Service account**
- Service account name: `deploy`
- Service account description: blank
- On the **Service account permissions (optional)** page add the **Compute Admin**, **Storage admin**.
- **Done** > ignore **Grant users access to this service account (optional)** and choose **Done**
- Select **Edit** (pencil) on the service account we just created
- **Add key** > **Create new key** > **JSON** > **Create**
1. Upload the service account key and create admin credentials in 1password:
- Open the 1password [Managed instances vault](https://my.1password.com/vaults/l35e5xtcfsk5suuj4vfj76hqpy/allitems) (ask @stephen, @gonza, or @beyang to grant you access)
- **Add** > **Document** > enter **$COMPANY service account** as the title > Upload the service account JSON file previously downloaded > **Save**
- **Add** > **Password** > enter **$COMPANY sourcegraph-admin** as the title > Change **length** to 40 and turn on symbols and digits >> **Save**
1. In GCP, enable the **Compute Engine API**:
- Under **APIs & Services** > **Library** search for "Compute"
- Select **Compute Engine API** and choose **Enable**
1. `export GOOGLE_APPLICATION_CREDENTIALS=~/Downloads/sourcegraph-managed-company-220df65550d4.json`
1. Clone and `cd deploy-sourcegraph-managed/`
1. `VERSION=v3.17.2 ./create-deployment.sh $COMPANY/` and **commit the result.**
1. Open and edit `deploy-sourcegraph-managed/$COMPANY/gcp-tfstate/gcp-tfstate.tf` according to the comments within, commit the result.
1. In `gcp-tfstate` run `terraform init && terraform apply && git add . && git commit -m 'initialize GCP tfstate bucket'`
1. Open and edit `infrastructure.tf` according to the comments within and commit the result.
1. In `deploy-sourcegraph-managed/$COMPANY` run `terraform init && terraform plan && terraform apply`
1. Access the instance over SSH and confirm all containers are healthy (instructions below).
1. In the infrastructure repository, [create a DNS entry](https://github.com/sourcegraph/infrastructure/blob/master/dns/sourcegraph-managed.tf) that points `$COMPANY.sourcegraph.com` to the `default-global-address` IP (see "Finding the external load balancer IP" below) and follow the process there to `asdf exec terraform apply` it.
1. Confirm all containers come up healthy (`docker ps` should report them as such)
1. Create a PR for review.
1. Access the Sourcegraph web UI (instructions for port-forwarding below)
1. Navigate to Grafana and confirm the instance looks healthy.
1. Set up the initial admin account (for use by the Sourcegraph team only)
- Email: `managed+$COMPANY@sourcegraph.com` (note `+` sign not `-`)
- Username: `sourcegraph-admin`
- Password: Use the password previously created and stored in 1password.
1. Configure `externalURL` in the site configuration, and use SSH to restart the server (`sudo su`, `shutdown -r`) wait for it to come back up and access it again.
1. In the global user settings, set `"alerts.showPatchUpdates": false`
1. In the GCP web UI under **Network services** > **Load balancers** > select the load balancer > watch the SSL certificate status. It may take some time for it to become active (~1h41m) / for Google to see the DNS change from Cloudflare. Confirm it is active by following ["Access through the GCP load balancer as a user would"](#access-through-the-gcp-load-balancer-as-a-user-would).
Loading