Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): bulk export #450

Merged
merged 40 commits into from
Dec 31, 2020
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
62a2eb5
refactor(cli): move findBaseDirs to tanka package
Duologic Dec 19, 2020
809bbcd
refactor(cli): use tanka.FindEnvironments()
Duologic Dec 19, 2020
c96ff10
refactor(cli): remove ErrNoSpec logic
Duologic Dec 19, 2020
652ba03
refactor(cli): move setupConfiguration to env.go
Duologic Dec 19, 2020
a468ebd
feat(cli): go relative with `tk env list <path>`
Duologic Dec 19, 2020
cb7bd94
chore(git): don't ignore cmd/tk as a dir
Duologic Dec 19, 2020
72869d7
fix(cli): relative path for predict
Duologic Dec 19, 2020
79f5333
fix: EnvsOnlyEvaluator explicitly removes Data, we should not add it
Duologic Dec 20, 2020
fc09689
feat(api): ParseEnvs evaluates multiple envs in parallel
Duologic Dec 20, 2020
2b5757e
fix: make Parallel an option with sane default
Duologic Dec 20, 2020
3f6653f
refactor(cli): load manifests for export in separate block
Duologic Dec 20, 2020
f6acfaf
feat(cli): export with dir template from Environment object
Duologic Dec 20, 2020
3fde4b5
refactor(cli): move export logic to pkg/tanka
Duologic Dec 20, 2020
a963666
feat(api): export multiple paths/envs
Duologic Dec 21, 2020
a033202
fix(api): cleaner ParseEnvs error handling
Duologic Dec 21, 2020
5793774
chore(cli): remove unused code
Duologic Dec 21, 2020
5aeeb75
feat(cli): export multiple environments
Duologic Dec 21, 2020
fbb193e
feat(cli): manifest file to map exported file to environment
Duologic Dec 21, 2020
d9ae637
fix(api): return envs if err nil in FindEnvironments
Duologic Dec 21, 2020
638c253
fix(cli): make recursive export optional
Duologic Dec 22, 2020
26de70f
refactor: group flags in flags.go
Duologic Dec 22, 2020
d007048
refactor: directly assign flags to opts
Duologic Dec 22, 2020
f72c571
doc(api): add docstrings on ExportEnvOpts
Duologic Dec 22, 2020
f72a09a
fix(cli): single --format option
Duologic Dec 23, 2020
269e6e3
fix: prune in EnvsOnlyEvaluator
Duologic Dec 24, 2020
8d61a2c
fix: naming is hard
Duologic Dec 24, 2020
31bbf27
fix: remove DefaultExportEnvOpts
Duologic Dec 24, 2020
0ed1506
fix: use process.StrExps
Duologic Dec 24, 2020
340c776
fix: include ParseOpts from cli
Duologic Dec 24, 2020
88af061
fix: coding nits
Duologic Dec 24, 2020
7441ccc
fix: error formatting
Duologic Dec 24, 2020
5db036e
fix: remove Data before re-serialization
Duologic Dec 24, 2020
70c1f4a
refactor: consolidate writing files in export.go
Duologic Dec 24, 2020
80eec44
fix: tk env list with optional path (backwards compat)
Duologic Dec 24, 2020
a583ca8
fix(api): prevent race condition
Duologic Dec 26, 2020
79fc752
fix(api): prevent another race condition + hanging worker
Duologic Dec 26, 2020
af5fd23
fix(cli): ensure relative paths are absolute from root
Duologic Dec 26, 2020
25161b9
post rebase
Duologic Dec 30, 2020
a83bb20
fix: remove noisy log statement
Duologic Dec 30, 2020
9085e6c
fix(cli): don't change original export format
Duologic Dec 31, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix(api): prevent another race condition + hanging worker
  • Loading branch information
Duologic committed Dec 30, 2020
commit 79fc752c2bfed9a55ceb01516330218e5c115d26
18 changes: 12 additions & 6 deletions pkg/tanka/environments.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package tanka

import (
"fmt"
"log"
"os"
"path/filepath"
"sync"
Expand Down Expand Up @@ -88,11 +89,11 @@ func ParseParallel(paths []string, opts ParseParallelOpts) (envs []*v1alpha1.Env
for i := 0; i < numParallel; i++ {
wg.Add(1)
go func() {
err := parseWorker(envsChan)
if err != nil {
allErrors = append(allErrors, err)
defer wg.Done()
errs := parseWorker(envsChan)
if errs != nil {
allErrors = append(allErrors, errs...)
}
wg.Done()
}()
}

Expand Down Expand Up @@ -132,13 +133,18 @@ type parseJob struct {
env *v1alpha1.Environment
}

func parseWorker(envsChan <-chan parseJob) error {
func parseWorker(envsChan <-chan parseJob) (errs []error) {
for req := range envsChan {
log.Printf("Parsing %s\n", req.path)
_, env, err := ParseEnv(req.path, req.opts)
if err != nil {
return fmt.Errorf("%w: %s", err, req.path)
Duologic marked this conversation as resolved.
Show resolved Hide resolved
errs = append(errs, fmt.Errorf("%s:\n %w", req.path, err))
continue
}
*req.env = *env
}
if len(errs) != 0 {
return errs
}
return nil
}