Skip to content

docs: reorganize template docs #10297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
f1e03c4
docs: rework our "templates" section
bpmct Jul 28, 2023
bfee554
wikistuff
bpmct Jul 28, 2023
80215cf
fix formatting
bpmct Jul 28, 2023
7119be1
add diagram
bpmct Jul 30, 2023
2c84f2e
reorganize some things
bpmct Jul 31, 2023
3279747
docs: improve workspaces and templates doc (#9139)
marcpaq Oct 16, 2023
89d07ae
Resolved conflicts from main branch
matifali Oct 16, 2023
9488d6a
Merge branch 'main' into templates-docs
matifali Oct 16, 2023
4447e68
make fmt
matifali Oct 16, 2023
86bff3f
fix merge conflict
matifali Oct 16, 2023
85cf21b
make fmt
matifali Oct 16, 2023
acc2ea6
make gen
matifali Oct 16, 2023
92856a5
update
matifali Oct 16, 2023
21eab5f
Merge branch 'main' into templates-docs
matifali Oct 16, 2023
93b2d38
lint
matifali Oct 17, 2023
ac2e51a
Discard changes to coderd/database/queries.sql.go
matifali Oct 17, 2023
30128dd
Discard changes to cli/templates.go
matifali Oct 17, 2023
5be19b5
Discard changes to cli/templateversionarchive.go
matifali Oct 17, 2023
62b5ef1
Discard changes to cli/templateversions.go
matifali Oct 17, 2023
65d7aed
Update docker-in-workspaces.md
matifali Oct 17, 2023
bc96319
replace ```sh with ```shell
matifali Oct 17, 2023
f86b428
open-in-coder
matifali Oct 17, 2023
77b0ac4
fmt
matifali Oct 17, 2023
5469513
mention coder_metadata in icons.md
matifali Oct 17, 2023
d43e370
resource_metadata
matifali Oct 17, 2023
20bd888
use shell
matifali Oct 17, 2023
b5b74b0
modules.md
matifali Oct 17, 2023
279ebc8
mention coder registry module
matifali Oct 17, 2023
3762b02
workspace.md
matifali Oct 17, 2023
d0c4292
resource_metadata
matifali Oct 17, 2023
c50b95e
remove duplication
matifali Oct 17, 2023
8911200
address comments
matifali Oct 17, 2023
15bf4a0
cleanup
matifali Oct 17, 2023
c107198
fmt
matifali Oct 17, 2023
33d6574
fix broken links
matifali Oct 17, 2023
27e43a8
fix numbering
matifali Oct 17, 2023
882610f
mention module registry
matifali Oct 17, 2023
ba152f8
add example
matifali Oct 17, 2023
51b80dd
demote heading
matifali Oct 17, 2023
6f84abc
remove top level entry from manifest
matifali Oct 17, 2023
e58588b
fmt
matifali Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/admin/provisioners.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Provisioners
# External provisioners

By default, the Coder server runs
[built-in provisioner daemons](../cli/server.md#provisioner-daemons), which
Expand Down
Binary file modified docs/images/autostart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/autostop.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/creating-workspace-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/schedule.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/template-variables.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/build-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/choosing-edit-template.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/coder-login-web.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/coder-session-token.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/create-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/create-workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/edit-files.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/edit-source-code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/new-workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/publish.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/select-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/source-code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/starter-templates.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-tour.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-variables.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/update.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/use-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/workspace-apps.png
Binary file added docs/images/templates/workspace-ready.png
Binary file added docs/images/workspace-update.png
99 changes: 65 additions & 34 deletions docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -139,55 +139,86 @@
},
{
"title": "Templates",
"description": "Learn about templates, which define the infrastructure underlying workspaces",
"description": "Templates define the infrastructure for workspaces",
"path": "./templates/index.md",
"icon_path": "./images/icons/picture.svg",
"children": [
{
"title": "Resource Persistence",
"description": "Learn how resource persistence works in Coder",
"path": "./templates/resource-persistence.md",
"icon_path": "./images/icons/infinity.svg"
"title": "Working with templates",
"description": "Creating, editing, and updating templates",
"path": "./templates/creating.md"
},
{
"title": "Provider Authentication",
"description": "Learn how to authenticate the provisioner",
"path": "./templates/authentication.md",
"icon_path": "./images/icons/key.svg"
},
{
"title": "Change Management",
"description": "Learn how to source-control templates with git and CI",
"path": "./templates/change-management.md",
"icon_path": "./images/icons/git.svg"
"title": "Your first template",
"description": "A tutorial for creating and editing your first template",
"path": "./templates/tutorial.md"
},
{
"title": "Resource Metadata",
"description": "Learn how to expose resource data to users",
"path": "./templates/resource-metadata.md",
"icon_path": "./images/icons/table-rows.svg"
"title": "Guided tour",
"description": "Create a template from scratch",
"path": "./templates/tour.md"
},
{
"title": "Agent Metadata",
"description": "Learn how to expose live agent information to users",
"path": "./templates/agent-metadata.md",
"icon_path": "./images/icons/table-rows.svg"
"title": "Setting up templates",
"description": "Best practices for writing templates",
"path": "./templates/best-practices.md",
"children": [
{
"title": "Change management",
"description": "Versioning templates with git and CI",
"path": "./templates/change-management.md",
"icon_path": "./images/icons/git.svg"
},
{
"title": "Provider authentication",
"description": "Authenticate the provisioner",
"path": "./templates/authentication.md",
"icon_path": "./images/icons/key.svg"
},
{
"title": "Resource persistence",
"description": "How resource persistence works in Coder",
"path": "./templates/resource-persistence.md",
"icon_path": "./images/icons/infinity.svg"
},
{
"title": "Terraform modules",
"description": "Reuse code across Coder templates",
"path": "./templates/modules.md"
}
]
},
{
"title": "Parameters",
"description": "Use parameters to customize templates",
"path": "./templates/parameters.md",
"icon_path": "./images/icons/code.svg"
"title": "Customizing templates",
"description": "Give information and options to workspace users",
"path": "./templates/customizing.md",
"children": [
{
"title": "Agent metadata",
"description": "Show operational metrics in the workspace",
"path": "./templates/agent-metadata.md"
},
{
"title": "Resource metadata",
"description": "Show information in the workspace about template resources",
"path": "./templates/resource-metadata.md"
},
{
"title": "Parameters",
"description": "Prompt the user for additional information about a workspace",
"path": "./templates/parameters.md"
}
]
},
{
"title": "Open in Coder",
"description": "Learn how to add an \"Open in Coder\" button to your repos",
"description": "Add an \"Open in Coder\" button to your repos",
"path": "./templates/open-in-coder.md",
"icon_path": "./images/icons/key.svg"
},
{
"title": "Docker in Workspaces",
"description": "Use docker inside containerized templates",
"title": "Docker in workspaces",
"description": "Use Docker inside containerized templates",
"path": "./templates/docker-in-workspaces.md",
"icon_path": "./images/icons/docker.svg"
},
Expand All @@ -198,9 +229,9 @@
"state": "alpha"
},
{
"title": "Terraform Modules",
"description": "Reuse code across Coder templates",
"path": "./templates/modules.md"
"title": "Troubleshooting templates",
"description": "Fix common template problems",
"path": "./templates/troubleshooting.md"
},
{
"title": "Process Logging",
Expand Down Expand Up @@ -337,7 +368,7 @@
"icon_path": "./images/icons/scale.svg"
},
{
"title": "Provisioners",
"title": "External Provisioners",
"description": "Run provisioners isolated from the Coder server",
"path": "./admin/provisioners.md",
"icon_path": "./images/icons/queue.svg",
Expand Down
4 changes: 4 additions & 0 deletions docs/platforms/jfrog.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ The admin-level access token is used to provision user tokens and is never expos
developers or stored in workspaces.
</blockquote>

<blockquote class="info">
You can skip the whole page and use [JFrog module](https://registry.coder.com/modules/jfrog) for easy JFrog Artifactory integration.
</blockquote>

## Provisioner Authentication

The most straight-forward way to authenticate your template with Artifactory is
Expand Down
73 changes: 38 additions & 35 deletions docs/templates/agent-metadata.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Agent Metadata
# Agent metadata

![agent-metadata](../images/agent-metadata.png)

With Agent Metadata, template admins can expose operational metrics from their
workspaces to their users. It is the dynamic complement of
[Resource Metadata](./resource-metadata.md).
You can show live operational metrics to workspace users with agent metadata. It
is the dynamic complement of [resource metadata](./resource-metadata.md).

See the
[Terraform reference](https://registry.terraform.io/providers/coder/coder/latest/docs/resources/agent#metadata).
You specify agent metadata in the
[`coder_agent`](https://registry.terraform.io/providers/coder/coder/latest/docs/resources/agent).

## Examples

Expand All @@ -16,9 +15,10 @@ All of these examples use
for the script declaration. With heredoc strings, you can script without messy
escape codes, just as if you were working in your terminal.

Some of the below examples use the [`coder stat`](../cli/stat.md) command. This
is useful for determining CPU/memory usage inside a container, which can be
tricky otherwise.
Some of the examples use the [`coder stat`](../cli/stat.md) command. This is
useful for determining CPU and memory usage of the VM or container that the
workspace is running in, which is more accurate than resource usage about the
workspace's host.

Here's a standard set of metadata snippets for Linux agents:

Expand Down Expand Up @@ -86,60 +86,63 @@ resource "coder_agent" "main" {
}
```

## Utilities
## Useful utilities

[top](https://linux.die.net/man/1/top) is available in most Linux distributions
and provides virtual memory, CPU and IO statistics. Running `top` produces
output that looks like:
You can also show agent metadata for information about the workspace's host.

[top](https://manpages.ubuntu.com/manpages/jammy/en/man1/top.1.html) is
available in most Linux distributions and provides virtual memory, CPU and IO
statistics. Running `top` produces output that looks like:

```text
%Cpu(s): 65.8 us, 4.4 sy, 0.0 ni, 29.3 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st
MiB Mem : 16009.0 total, 493.7 free, 4624.8 used, 10890.5 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 11021.3 avail Mem
```

[vmstat](https://linux.die.net/man/8/vmstat) is available in most Linux
distributions and provides virtual memory, CPU and IO statistics. Running
`vmstat` produces output that looks like:
[vmstat](https://manpages.ubuntu.com/manpages/jammy/en/man8/vmstat.8.html) is
available in most Linux distributions and provides virtual memory, CPU and IO
statistics. Running `vmstat` produces output that looks like:

```text
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 19580 4781680 12133692 217646944 0 2 4 32 1 0 1 1 98 0 0
```

[dstat](https://linux.die.net/man/1/dstat) is considerably more parseable than
`vmstat` but often not included in base images. It is easily installed by most
package managers under the name `dstat`. The output of running `dstat 1 1` looks
like:
[dstat](https://manpages.ubuntu.com/manpages/jammy/man1/dstat.1.html) is
considerably more parseable than `vmstat` but often not included in base images.
It is easily installed by most package managers under the name `dstat`. The
output of running `dstat 1 1` looks like:

```text
--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
1 1 98 0 0|3422k 25M| 0 0 | 153k 904k| 123k 174k
```

## DB Write Load
## Managing the database load

Agent metadata can generate a significant write load and overwhelm your database
if you're not careful. The approximate writes per second can be calculated using
the following formula (applied once for each unique metadata interval):
Agent metadata can generate a significant write load and overwhelm your Coder
database if you're not careful. The approximate writes per second can be
calculated using the formula:

```text
num_running_agents * write_multiplier / metadata_interval
(metadata_count * num_running_agents * 2) / metadata_avg_interval
```

For example, let's say you have:
For example, let's say you have

- 10 running agents
- each with 4 metadata snippets
- where two have an interval of 4 seconds, and the other two 6 seconds
- each with 6 metadata snippets
- with an average interval of 4 seconds

You can expect `(10 * 6 * 2) / 4`, or 30 writes per second.

One of the writes is to the `UNLOGGED` `workspace_agent_metadata` table and the
other to the `NOTIFY` query that enables live stats streaming in the UI.

You can expect at most `(10 * 2 / 4) + (10 * 2 / 6)` or ~8 writes per second.
The actual writes per second may be a bit lower due to batching of metadata.
Adding more metadata with the same interval will not increase writes per second,
but it may still increase database load slightly.
## Next Steps

We use a `write_multiplier` of `2` because each metadata write generates two
writes. One of the writes is to the `UNLOGGED` `workspace_agent_metadata` table
and the other to the `NOTIFY` query that enables live stats streaming in the UI.
- [Resource metadata](./resource-metadata.md)
- [Parameters](./parameters.md)
47 changes: 29 additions & 18 deletions docs/templates/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,42 @@
</p>
</blockquote>

Coder's provisioner process needs to authenticate with cloud provider APIs to
provision workspaces. You can either pass credentials to the provisioner as
parameters or execute Coder in an environment that is authenticated with the
cloud provider.
The Coder server's
[provisioner](https://registry.terraform.io/providers/coder/coder/latest/docs/data-sources/provisioner)
process needs to authenticate with other provider APIs to provision workspaces.
There are two approaches to do this:

We encourage the latter where supported. This approach simplifies the template,
keeps cloud provider credentials out of Coder's database (making it a less
valuable target for attackers), and is compatible with agent-based
authentication schemes (that handle credential rotation and/or ensure the
credentials are not written to disk).
- Pass credentials to the provisioner as parameters.
- Preferred: Execute the Coder server in an environment that is authenticated
with the provider.

Cloud providers for which the Terraform provider supports authenticated
environments include
We encourage the latter approach where supported:

- Simplifies the template.
- Keeps provider credentials out of Coder's database, making it a less valuable
target for attackers.
- Compatible with agent-based authentication schemes, which handle credential
rotation or ensure the credentials are not written to disk.

Generally, you can set up an environment to provide credentials to Coder in
these ways:

- A well-known location on disk. For example, `~/.aws/credentials` for AWS on
POSIX systems.
- Environment variables.

It is usually sufficient to authenticate using the CLI or SDK for the provider
before running Coder, but check the Terraform provider's documentation for
details.

These platforms have Terraform providers that support authenticated
environments:

- [Google Cloud](https://registry.terraform.io/providers/hashicorp/google/latest/docs)
- [Amazon Web Services](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
- [Microsoft Azure](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
- [Kubernetes](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs)

Additional providers may be supported; check the
Other providers might also support authenticated environments. Check the
[documentation of the Terraform provider](https://registry.terraform.io/browse/providers)
for details.

The way these generally work is via the credentials being available to Coder
either in some well-known location on disk (e.g. `~/.aws/credentials` for AWS on
posix systems), or via environment variables. It is usually sufficient to
authenticate using the CLI or SDK for the cloud provider before running Coder
for this to work, but check the Terraform provider documentation for details.
7 changes: 7 additions & 0 deletions docs/templates/best-practices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Template best practices

We recommend a few ways to manage workspace resources, authentication, and
versioning.

<children>
</children>
14 changes: 6 additions & 8 deletions docs/templates/change-management.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Template Change Management

We recommend source controlling your templates as you would other code.
[Install Coder](../install/) in CI/CD pipelines to push new template versions.
We recommend source-controlling your templates as you would other code. You can
[install Coder](../install/) in CI/CD pipelines to push new template versions.

```console
# Install the Coder CLI
Expand All @@ -21,14 +21,12 @@ export CODER_TEMPLATE_DIR=.coder/templates/kubernetes
export CODER_TEMPLATE_VERSION=$(git rev-parse --short HEAD)

# Push the new template version to Coder
coder login --url $CODER_URL --token $CODER_SESSION_TOKEN
coder templates push --yes $CODER_TEMPLATE_NAME \
--directory $CODER_TEMPLATE_DIR \
--name=$CODER_TEMPLATE_VERSION # Version name is optional
```

> Looking for an example? See how we push our development image and template
> [via GitHub actions](https://github.com/coder/coder/blob/main/.github/workflows/dogfood.yaml).

> To cap token lifetime on creation,
> [configure Coder server to set a shorter max token lifetime](../cli/server.md#--max-token-lifetime)
To cap token lifetime on creation,
[configure Coder server to set a shorter max token lifetime](../cli/server.md#--max-token-lifetime).
For an example, see how we push our development image and template
[with GitHub actions](https://github.com/coder/coder/blob/main/.github/workflows/dogfood.yaml).
Loading