Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add template-files composite action #127

Merged
merged 27 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
4f4abe8
Adding initial template-files action
kenodegard Sep 13, 2023
08fced3
Include requirements.txt for cache key
kenodegard Sep 14, 2023
6d4a7ac
Use github context instead of envvar
kenodegard Sep 14, 2023
c5d2fd5
Undo caching
kenodegard Sep 14, 2023
2aa4444
Improve argument validation
kenodegard Sep 14, 2023
e57ceac
Less verbose pip install
kenodegard Sep 14, 2023
7f3ab1d
Improve error message
kenodegard Sep 14, 2023
20c5202
Improve errors
kenodegard Sep 14, 2023
9049e74
Add rich colors
kenodegard Sep 14, 2023
0120e76
Preserve trailing newlines
kenodegard Sep 14, 2023
cea2c34
Format
kenodegard Sep 14, 2023
ed50ef7
Gracefully terminate if no config file
kenodegard Sep 19, 2023
af5f413
Rework into functions for error handling
kenodegard Sep 19, 2023
f3ceab2
Format
kenodegard Sep 19, 2023
55324f5
Add option to remove files
kenodegard Sep 19, 2023
7ef616f
Better error handling
kenodegard Sep 19, 2023
7356f38
Fix jsonschema type
kenodegard Sep 19, 2023
1056e74
Add requirements.txt
kenodegard May 7, 2024
0b3b626
Convert GHA version tag to commit hash
kenodegard May 7, 2024
7393d79
Suggestions from code review
kenodegard May 7, 2024
df6127c
Correct src/dst variables
kenodegard May 7, 2024
ca16921
Merge remote-tracking branch 'upstream/main' into template-files
kenodegard May 7, 2024
a1d67e3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 7, 2024
8f068a2
Specify requirements file for setup-python cache
kenodegard May 7, 2024
37239ba
Manual caching
kenodegard May 7, 2024
f2a3cf2
Merge branch 'main' into template-files
kenodegard May 8, 2024
5006bd1
Correct regex
kenodegard May 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions template-files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Template Files

A composite GitHub Action to template (or copy) files from other repositories and
commits them to the specified PR.

## GitHub Action Usage

In your GitHub repository include this action in your workflows:

```yaml
- uses: conda/actions/template-files
with:
# [optional]
# the path to the configuration file
config: .github/templates/config.yml

# [optional]
# the path to the template stubs
stubs: .github/templates/

# [optional]
# the GitHub token with API access
token: ${{ github.token }}
```

Define what files to template in a configuration file, e.g., `.github/templates/config.yml`:

```yaml
user/repo:
# copy to same path
- path/to/file
- src: path/to/file

# copy to different path
- src: path/to/other
dst: path/to/another

# templating
- src: path/to/template
with:
name: value
```
217 changes: 217 additions & 0 deletions template-files/action.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
"""Copy files from external locations as defined in `sync.yml`."""
from __future__ import annotations

import os
import sys
from argparse import ArgumentParser, ArgumentTypeError, Namespace
from pathlib import Path
from typing import Any

import yaml
from github import Auth, Github, UnknownObjectException
from github.Repository import Repository
from jinja2 import Environment, FileSystemLoader
from jsonschema import validate
from rich.console import Console

print = Console(color_system="standard", soft_wrap=True).print
perror = Console(
color_system="standard",
soft_wrap=True,
stderr=True,
style="bold red",
).print


def validate_file(value: str) -> Path | None:
try:
path = Path(value).expanduser().resolve()
path.read_text()
return path
except (IsADirectoryError, PermissionError) as err:
# IsADirectoryError: value is a directory, not a file
# PermissionError: value is not readable
raise ArgumentTypeError(f"{value} is not a valid file: {err}")
except FileNotFoundError:
# FileNotFoundError: value does not exist
return None


def validate_dir(value: str) -> Path:
try:
path = Path(value).expanduser().resolve()
path.mkdir(parents=True, exist_ok=True)
ignore = path / ".ignore"
ignore.touch()
ignore.unlink()
return path
except (FileExistsError, PermissionError) as err:
# FileExistsError: value is a file, not a directory
# PermissionError: value is not writable
raise ArgumentTypeError(f"{value} is not a valid directory: {err}")


def parse_args() -> Namespace:
# parse CLI for inputs
parser = ArgumentParser()
parser.add_argument("--config", type=validate_file, required=True)
parser.add_argument("--stubs", type=validate_dir, required=True)
return parser.parse_args()


def read_config(args: Namespace) -> dict:
# read and validate configuration file
config = yaml.load(
args.config.read_text(),
Loader=yaml.SafeLoader,
)
validate(
config,
schema={
"type": "object",
"patternProperties": {
r"\w+/\w+": {
"type": "array",
"items": {
"type": ["string", "object"],
"minLength": 1,
"properties": {
"src": {"type": "string"},
"dst": {"type": "string"},
"remove": {"type": "boolean"},
"with": {
"type": "object",
"patternProperties": {
r"\w+": {"type": "string"},
},
},
},
},
}
},
},
)
return config


def iterate_config(
config: dict,
gh: Github,
env: Environment,
source: Repository,
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
) -> int:
# iterate over configuration and template files
errors = 0
for repository, files in config.items():
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
try:
destination = gh.get_repo(repository)
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
except UnknownObjectException as err:
perror(f"❌ Failed to fetch {repository}: {err}")
errors += 1
continue

for file in files:
src: str | None
dst: Path | None
remove: bool
context: dict[str, Any]

if isinstance(file, str):
src = file
dst = Path(file)
remove = False
context = {}
elif isinstance(file, dict):
src = file.get("src", None)
dst = None if (tmp := file.get("dst", src)) is None else Path(tmp)
remove = file.get("remove", False)
context = file.get("with", {})
else:
perror(f"❌ Invalid file definition ({file}), expected str or dict")
errors += 1
continue

if remove:
if dst is None:
perror(f"❌ Invalid file definition ({file}), expected dst")
errors += 1
continue

try:
dst.unlink()
except FileNotFoundError:
# FileNotFoundError: dst does not exist
print(f"⚠️ {dst} has already been removed")
except PermissionError as err:
# PermissionError: not possible to remove dst
perror(f"❌ Failed to remove {dst}: {err}")
errors += 1
else:
print(f"✅ Removed {dst}")
continue

try:
content = destination.get_contents(src).decoded_content.decode()
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
except UnknownObjectException as err:
perror(f"❌ Failed to fetch {src} from {repository}: {err}")
errors += 1
continue

# inject stuff about the source and destination
context["repo"] = context["dst"] = context["destination"] = source
context["src"] = context["source"] = destination
kenodegard marked this conversation as resolved.
Show resolved Hide resolved

template = env.from_string(content)
dst.parent.mkdir(parents=True, exist_ok=True)
dst.write_text(template.render(**context))

print(f"✅ Templated {repository}/{src} as {dst}")

return errors


def main():
errors = 0

args = parse_args()
if not args.config:
print("⚠️ No configuration file found, nothing to update")
sys.exit(0)

config = read_config(args)

# initialize Jinja environment and GitHub client
env = Environment(
loader=FileSystemLoader(args.stubs),
jezdez marked this conversation as resolved.
Show resolved Hide resolved
# {{ }} is used in MermaidJS
# ${{ }} is used in GitHub Actions
# { } is used in Python
# %( )s is used in Python
block_start_string="[%",
block_end_string="%]",
variable_start_string="[[",
variable_end_string="]]",
comment_start_string="[#",
comment_end_string="#]",
keep_trailing_newline=True,
jezdez marked this conversation as resolved.
Show resolved Hide resolved
)
gh = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))

# get current repository
repository = os.environ["GITHUB_REPOSITORY"]
try:
source = gh.get_repo(repository)
except UnknownObjectException as err:
perror(f"❌ Failed to fetch {repository}: {err}")
errors += 1

if not errors:
errors += iterate_config(config, gh, env, source)
kenodegard marked this conversation as resolved.
Show resolved Hide resolved

if errors:
perror(f"Got {errors} error(s)")
sys.exit(errors)


if __name__ == "__main__":
main()
38 changes: 38 additions & 0 deletions template-files/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
name: Template Files
description: Template (or copy) files from other repositories and commits them to the specified PR.
author: Anaconda Inc.
branding:
icon: book-open
color: green

inputs:
config:
description: Configuration path defining what files to template/copy.
default: .github/templates/config.yml
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
stubs:
description: >-
Path to where stub files are located in the current repository.
default: .github/templates/
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
token:
description: >-
A token with ability to comment, label, and modify the commit status
(`pull_request: write` and `statuses: write` for fine-grained PAT; `repo` for classic PAT)
default: ${{ github.token }}

runs:
using: composite
steps:
- uses: actions/setup-python@v4
kenodegard marked this conversation as resolved.
Show resolved Hide resolved
with:
python-version: '3.11'
kenodegard marked this conversation as resolved.
Show resolved Hide resolved

- name: install dependencies
shell: bash
run: pip install --quiet jinja2 jsonschema pygithub pyyaml rich
kenodegard marked this conversation as resolved.
Show resolved Hide resolved

- name: sync & template files
shell: bash
run: python ${{ github.action_path }}/action.py --config ${{ inputs.config }} --stubs ${{ inputs.stubs }}
env:
GITHUB_TOKEN: ${{ github.token }}
Loading