From 8e7271abbd89be3575710323b72e114dada5d551 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Tue, 18 Aug 2020 17:44:16 +0200 Subject: [PATCH 01/18] engineering: Add GCP projects overview --- handbook/engineering/environments.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 handbook/engineering/environments.md diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md new file mode 100644 index 00000000000..7949515f44b --- /dev/null +++ b/handbook/engineering/environments.md @@ -0,0 +1,25 @@ +# Environment + +This page describes our different cloud environments. + +## Google Cloud + +We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control. + +- **Sourcegraph**: Services for sourcegraph.com. +- **sourcegraph-code-intel**: Services for Code Intel code execution. +- **sourcegraph-managed-**: There are multiple `-managed` projects, one for each our managed deployments. +- **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging. +- **Sourcegraph Auxiliary**: Testing clusters and deployments. +- **Universities**: Sourcegraph instances for universities. (Deprecated) +- **sourcegraph-interviews**: Shared project for interviews +- **Engineers Projects (Folder)**: Contains multiple engineers project. All projects must be prefixed with `$name-`. + +--- + +- **sourcegraph-testing**: Testing clusters and deployments. (Delete or not?) +- **sourcegraph-calend**: ??? +- **sourcegraph-orgtool**: ??? +- **Quickstart**: ??? +- **Gmail**: ??? +- **Release**: ??? From 11428d8fb7867e385519b71fecaa20dda7e5a3b4 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Tue, 18 Aug 2020 17:46:33 +0200 Subject: [PATCH 02/18] fixup! engineering: Add GCP projects overview --- handbook/engineering/environments.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 7949515f44b..191cf714690 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -6,6 +6,9 @@ This page describes our different cloud environments. We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control. + +### Projects + - **Sourcegraph**: Services for sourcegraph.com. - **sourcegraph-code-intel**: Services for Code Intel code execution. - **sourcegraph-managed-**: There are multiple `-managed` projects, one for each our managed deployments. From 59908b5a8581b8dc73bf4221b2d898d640fa6cfb Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Tue, 18 Aug 2020 17:48:26 +0200 Subject: [PATCH 03/18] Split dynamic folders --- handbook/engineering/environments.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 191cf714690..99e569d084e 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -7,7 +7,7 @@ This page describes our different cloud environments. We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control. -### Projects +### Root Projects - **Sourcegraph**: Services for sourcegraph.com. - **sourcegraph-code-intel**: Services for Code Intel code execution. @@ -16,7 +16,11 @@ We utilize multiple Google Cloud projects and folders to organize our workloads - **Sourcegraph Auxiliary**: Testing clusters and deployments. - **Universities**: Sourcegraph instances for universities. (Deprecated) - **sourcegraph-interviews**: Shared project for interviews -- **Engineers Projects (Folder)**: Contains multiple engineers project. All projects must be prefixed with `$name-`. + +### Folders +Dynamic or temporary projects are created in the generic folders listed below. + +- **Engineers Projects**: Contains multiple engineers project. All projects must be prefixed with `$name-`. --- From 3b26bea1ecb5f78815391a55feb285257dd1c023 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Tue, 18 Aug 2020 17:49:01 +0200 Subject: [PATCH 04/18] fixup! Split dynamic folders --- handbook/engineering/environments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 99e569d084e..222cdd79f3f 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -18,7 +18,7 @@ We utilize multiple Google Cloud projects and folders to organize our workloads - **sourcegraph-interviews**: Shared project for interviews ### Folders -Dynamic or temporary projects are created in the generic folders listed below. +Dynamic or temporary projects are created in the folders listed below. - **Engineers Projects**: Contains multiple engineers project. All projects must be prefixed with `$name-`. From d4dd463b8d7fbaa9d4f5e55bfbe670c96d0f4558 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Mon, 24 Aug 2020 11:01:11 +0200 Subject: [PATCH 05/18] Update handbook/engineering/environments.md Co-authored-by: Robert Lin --- handbook/engineering/environments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 222cdd79f3f..9f861e62496 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -20,7 +20,7 @@ We utilize multiple Google Cloud projects and folders to organize our workloads ### Folders Dynamic or temporary projects are created in the folders listed below. -- **Engineers Projects**: Contains multiple engineers project. All projects must be prefixed with `$name-`. +- **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). --- From 2f06e2863e0c4076bf5d2e1cc629a090fa6e4491 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Mon, 24 Aug 2020 12:45:08 +0200 Subject: [PATCH 06/18] fixup! Split dynamic folders --- handbook/engineering/environments.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 9f861e62496..1251895f0c0 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -21,12 +21,3 @@ We utilize multiple Google Cloud projects and folders to organize our workloads Dynamic or temporary projects are created in the folders listed below. - **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). - ---- - -- **sourcegraph-testing**: Testing clusters and deployments. (Delete or not?) -- **sourcegraph-calend**: ??? -- **sourcegraph-orgtool**: ??? -- **Quickstart**: ??? -- **Gmail**: ??? -- **Release**: ??? From aab17cb65b8aba8e9e727f45ce33e563ca5e61d9 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Mon, 24 Aug 2020 15:09:35 +0200 Subject: [PATCH 07/18] Add -calend and -orgtool projects --- handbook/engineering/environments.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 1251895f0c0..37c98783833 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -15,9 +15,11 @@ We utilize multiple Google Cloud projects and folders to organize our workloads - **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging. - **Sourcegraph Auxiliary**: Testing clusters and deployments. - **Universities**: Sourcegraph instances for universities. (Deprecated) -- **sourcegraph-interviews**: Shared project for interviews +- **sourcegraph-interviews**: Shared project for interviews. +- **sourcegraph-calend**: Owned by @sqs. +- **sourcegraph-orgtool**: Owned by @sqs. ### Folders Dynamic or temporary projects are created in the folders listed below. -- **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). +- **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). Engineers are expected to remove all their resources once they are done testing. From 7db4536347dc34343be7cf71b7c97a84d2a6d356 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Mon, 24 Aug 2020 17:21:30 +0200 Subject: [PATCH 08/18] Update handbook/engineering/environments.md Co-authored-by: Nick Snyder --- handbook/engineering/environments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 37c98783833..518415b56d9 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -20,6 +20,7 @@ We utilize multiple Google Cloud projects and folders to organize our workloads - **sourcegraph-orgtool**: Owned by @sqs. ### Folders + Dynamic or temporary projects are created in the folders listed below. - **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). Engineers are expected to remove all their resources once they are done testing. From 5b4a3c4a355a6d212aff2bdc53e0e2f32438e3b6 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 12:29:59 +0200 Subject: [PATCH 09/18] fixup! Update handbook/engineering/environments.md --- handbook/engineering/environments.md | 37 +++++++++++++++++++--------- 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 518415b56d9..7ff32555c29 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -4,23 +4,36 @@ This page describes our different cloud environments. ## Google Cloud -We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control. - +We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control for our engineers, as well as limit the scope of roles and service accounts across projects. ### Root Projects +These projects contain per-project permissions. -- **Sourcegraph**: Services for sourcegraph.com. -- **sourcegraph-code-intel**: Services for Code Intel code execution. -- **sourcegraph-managed-**: There are multiple `-managed` projects, one for each our managed deployments. -- **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging. -- **Sourcegraph Auxiliary**: Testing clusters and deployments. -- **Universities**: Sourcegraph instances for universities. (Deprecated) -- **sourcegraph-interviews**: Shared project for interviews. -- **sourcegraph-calend**: Owned by @sqs. -- **sourcegraph-orgtool**: Owned by @sqs. +- **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging +- **Universities**: Sourcegraph instances for universities +- **sourcegraph-interviews**: Shared project for interviews ### Folders Dynamic or temporary projects are created in the folders listed below. -- **Engineers Projects**: Contains projects used by individual engineers. All projects must be prefixed with `$name-` (the name of the owner). Engineers are expected to remove all their resources once they are done testing. +### Engineers Projects +Contains projects used by individual engineers. Engineers are expected to remove all their resources once they are done testing. All projects must be prefixed with `$name-` (the name of the owner). + +### Sourcegraph Cloud +Sourcegraph Cloud projects. + +- **Sourcegraph**: Services for sourcegraph.com +- **Sourcegraph Auxiliary**: Testing clusters, deployments and VMs +- **sourcegraph-code-intel**: Services for Code Intel code execution +- **Sourcegraph CI**: Services for our CI cluster and temporary CI resources + +### Other Projects +Misc Projects with per-project permissions. + +- **sourcegraph-calend**: Owned by @sqs +- **sourcegraph-orgtool**: Owned by @sqs + +### Managed Instances + +Multiple `sourcegraph-managed-$name` projects, one for each our managed deployments to guarantee separation of privileges and access control. From 324154db5358fe8f61d14ae0b1fa10a57ffb95d1 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:21:06 +0200 Subject: [PATCH 10/18] Update handbook/engineering/environments.md Co-authored-by: Nick Snyder --- handbook/engineering/environments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 7ff32555c29..9f1457a136b 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -7,6 +7,7 @@ This page describes our different cloud environments. We utilize multiple Google Cloud projects and folders to organize our workloads and manage access control for our engineers, as well as limit the scope of roles and service accounts across projects. ### Root Projects + These projects contain per-project permissions. - **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging From a9cefcaa2ccd8b23ec8b5c702a3707249a5e7da8 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:21:17 +0200 Subject: [PATCH 11/18] Update handbook/engineering/environments.md Co-authored-by: Nick Snyder --- handbook/engineering/environments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 9f1457a136b..77ce27b5192 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -18,6 +18,7 @@ These projects contain per-project permissions. Dynamic or temporary projects are created in the folders listed below. + ### Engineers Projects Contains projects used by individual engineers. Engineers are expected to remove all their resources once they are done testing. All projects must be prefixed with `$name-` (the name of the owner). From 92afaa94b7ecdf63bf3b77fc2eadf2bfb81990c1 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:21:28 +0200 Subject: [PATCH 12/18] Update handbook/engineering/environments.md Co-authored-by: Nick Snyder --- handbook/engineering/environments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 77ce27b5192..4d0866bf317 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -22,6 +22,7 @@ Dynamic or temporary projects are created in the folders listed below. ### Engineers Projects Contains projects used by individual engineers. Engineers are expected to remove all their resources once they are done testing. All projects must be prefixed with `$name-` (the name of the owner). + ### Sourcegraph Cloud Sourcegraph Cloud projects. From 3648cb6d8400ae640319641c567330141866096f Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:21:35 +0200 Subject: [PATCH 13/18] Update handbook/engineering/environments.md Co-authored-by: Nick Snyder --- handbook/engineering/environments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 4d0866bf317..442766835b6 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -31,6 +31,7 @@ Sourcegraph Cloud projects. - **sourcegraph-code-intel**: Services for Code Intel code execution - **Sourcegraph CI**: Services for our CI cluster and temporary CI resources + ### Other Projects Misc Projects with per-project permissions. From ab3fbc7dba1f8fefef62962e3c4fe7fe241f6b1c Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:34:19 +0200 Subject: [PATCH 14/18] Fix newlines --- handbook/engineering/environments.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 442766835b6..ec2ef4c4544 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -16,14 +16,12 @@ These projects contain per-project permissions. ### Folders -Dynamic or temporary projects are created in the folders listed below. - - ### Engineers Projects -Contains projects used by individual engineers. Engineers are expected to remove all their resources once they are done testing. All projects must be prefixed with `$name-` (the name of the owner). +Contains projects used by individual engineers. Engineers are expected to remove all their resources once they are done testing. All projects must be prefixed with `$name-` (the name of the owner). ### Sourcegraph Cloud + Sourcegraph Cloud projects. - **Sourcegraph**: Services for sourcegraph.com @@ -31,8 +29,8 @@ Sourcegraph Cloud projects. - **sourcegraph-code-intel**: Services for Code Intel code execution - **Sourcegraph CI**: Services for our CI cluster and temporary CI resources - ### Other Projects + Misc Projects with per-project permissions. - **sourcegraph-calend**: Owned by @sqs From dd3dac36e0582eb84e449b3581d9de7e73f6cbe6 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:35:26 +0200 Subject: [PATCH 15/18] Clarify per-project permissions --- handbook/engineering/environments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index ec2ef4c4544..710d312a8b9 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -31,7 +31,7 @@ Sourcegraph Cloud projects. ### Other Projects -Misc Projects with per-project permissions. +Misc Projects with permissions set at the project level. - **sourcegraph-calend**: Owned by @sqs - **sourcegraph-orgtool**: Owned by @sqs From 423ddf56c544aaa0558eef9cbc126211c53baaf9 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:37:32 +0200 Subject: [PATCH 16/18] End lists with punctuation --- handbook/engineering/environments.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 710d312a8b9..2c7fc4ef4c9 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -10,9 +10,9 @@ We utilize multiple Google Cloud projects and folders to organize our workloads These projects contain per-project permissions. -- **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging -- **Universities**: Sourcegraph instances for universities -- **sourcegraph-interviews**: Shared project for interviews +- **TelligentSourcegraph**: Data pipeline and storage for pings and Cloud event logging. +- **Universities**: Sourcegraph instances for universities. +- **sourcegraph-interviews**: Shared project for interviews. ### Folders @@ -24,17 +24,17 @@ Contains projects used by individual engineers. Engineers are expected to remove Sourcegraph Cloud projects. -- **Sourcegraph**: Services for sourcegraph.com -- **Sourcegraph Auxiliary**: Testing clusters, deployments and VMs -- **sourcegraph-code-intel**: Services for Code Intel code execution -- **Sourcegraph CI**: Services for our CI cluster and temporary CI resources +- **Sourcegraph**: Services for sourcegraph.com. +- **Sourcegraph Auxiliary**: Testing clusters, deployments and VMs. +- **sourcegraph-code-intel**: Services for Code Intel code execution. +- **Sourcegraph CI**: Services for our CI cluster and temporary CI resources. ### Other Projects Misc Projects with permissions set at the project level. -- **sourcegraph-calend**: Owned by @sqs -- **sourcegraph-orgtool**: Owned by @sqs +- **sourcegraph-calend**: Owned by @sqs. +- **sourcegraph-orgtool**: Owned by @sqs. ### Managed Instances From 03e77daa648260b516876308f42b348a0ab43e77 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:38:59 +0200 Subject: [PATCH 17/18] Add to index --- handbook/engineering/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/handbook/engineering/index.md b/handbook/engineering/index.md index 02462ec9474..781b8572f2c 100644 --- a/handbook/engineering/index.md +++ b/handbook/engineering/index.md @@ -22,6 +22,7 @@ - [Incidents](incidents.md) - [Releases](releases/index.md) - [Release issue template](releases/release_issue_template.md) +- [Cloud environments](environments.md) - [Deployments](deployments.md) - [On-call](on_call/index.md) - [Prometheus](prometheus.md) From f81423296e56077ecb86fc6eff9bc990b9df71f4 Mon Sep 17 00:00:00 2001 From: Gonzalo Peci Date: Fri, 28 Aug 2020 18:43:33 +0200 Subject: [PATCH 18/18] Explicit code-intel isolation --- handbook/engineering/environments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/handbook/engineering/environments.md b/handbook/engineering/environments.md index 2c7fc4ef4c9..7f4494d8b2b 100644 --- a/handbook/engineering/environments.md +++ b/handbook/engineering/environments.md @@ -26,7 +26,7 @@ Sourcegraph Cloud projects. - **Sourcegraph**: Services for sourcegraph.com. - **Sourcegraph Auxiliary**: Testing clusters, deployments and VMs. -- **sourcegraph-code-intel**: Services for Code Intel code execution. +- **sourcegraph-code-intel**: Services for Code Intel code execution that are separated from our production project for extra isolation. - **Sourcegraph CI**: Services for our CI cluster and temporary CI resources. ### Other Projects