- Do not use gpg signatures
- Remove stunnel, dstat, libncurses5-dev from base AMI
- Update dependencies
- Skip use1-az3 availability zone when creating subnets
- build_ami: use 7th generation instances; force manage_iam to true
- Use Ubuntu 24.04 by default
- Add more context sensitve help for config
- Remove stunnel, dstat, libncurses5-dev from base AMI
- Update dependencies
- Skip use1-az3 availability zone when creating subnets
- build_ami: use 7th generation instances
- build-ami: force manage_iam to true
- Use ebs-gp3 for ubuntu AMI parameter store lookup
- Use Ubuntu 24.04 by default
- Add more context sensitve help for config
- s3 buckets: Compute size metrics for all storage classes
- aegea ssh: download correct SSM plugin
- aegea launch: use t3a.micro instance type by default
- Use Amazon Linux 2023 when specifying Amazon Linux
- Update paramiko dependency range
- Remove aegea deploy and git utils
- Bump dependency versions
- Test and documentation infrastructure improvements
- Bump dependency versions
- Test infrastructure improvements
launch: Add logic to set and discover Amazon Linux 2022 AMI Update base_config.yml
- Add default config value for build_ami.tags
build-ami: correctly ingest tag value from config chain
Note that this change necessitates the use of a new syntax for the build_ami.tags section of the YAML config file. Instead of a list of strings, use a mapping.
- build-ami: correctly ingest tag value from config chain (part 2)
- build-ami: correctly ingest tag value from config chain
- Bump dependency versions
- Implement full IMDSv2 support
Add version file to gitignore
- Use setuptools-scm to manage version
- Begin CloudTrail log reader (aegea cloudtrail)
- Documentation and test improvements
- Fix loading of default value for encryption config
- ensure_s3_bucket: Set bucket default encryption
- ssh: Avoid using DescribeSubnets to get instance AZ
- aegea ssh: Derive SSH user name and ID consistently. Please note: this change alters the algorithm that derives auto-provisioned Linux user IDs from IAM principal names, which can cause changes to EFS permissions and in other situations where UID mappings are used.
- launch: add manage_iam option, –no-manage-iam flag to access it
- Namespace rootfs.skel by command
- Scan all config file dirs for rootfs.skel (#63)
- Accommodate vpc eventual consistency
- Allow storage and iam_role to be passed via config files
- Only mount accessible efs shares (#61)
- launch: fix logic for detection of existing EFS home
- launch: correctly handle EFS home in multiple VPCs
- launch: cancel SIR on waiter error
- batch: deploy Lambda helper from a writable location
- logs: set default log horizon to 24h
- aegea ssh: Restore python 3.6 compatibility
- Documentation improvements
- ssh: print legible help when instance is not running
- Add autoprovisioned user to docker group
- Add aegea-ssh shim for vscode
- Fix logic error in set_aws_profile
launch: select subnet for spot instances
- Use correct ARN resource element for principal name
- launch: use cloudinit to provision current user by default
- ssh: use ec2-instance-connect, unbundle keymaker
- Avoid using non-aegea AMIs by default
- Support IPv6 in aegea VPCs
- Use only aegea-managed or default VPCs
- rm: support deleting subnets and vpcs
- Sort subnets by vpc_id
- aegea build-ami: use default image name if not supplied
- batch: use custom log group if set
- Drop direct dependency on awscli to facilitate awscli v2 compatibility
- Use SSM for locating Ubuntu AMI
- aegea now supports arm64 instances
- Recover bdm aliases
- Enable resizing of root volume
- Fix incomplete release
- Update release script
- Add ripgrep and fd to base AMI
- batch watch: Sleep on exit to capture more logs
- Avoid assumptions about source profile IAM usernames
- Sort ECR images by date
- build-ami: adjust timeouts and error message
- Replace deprecated btrfs-tools with btrfs-progs
- Use c5.xlarge to build AMIs; clarify Batch error message
- aegea sfn ls: add status filter
- Add zstd to base config
- Bump dependency versions
- aegea batch: begin fargate support
- Use subprocess instead of distro to check for Ubuntu presence (#56)
- Begin managed policy concatenator; use concatenated policy in aegea launch
- Change default ecs run container image to Ubuntu 20.04 LTS
- ecs stop: fix argument reference
- Add ecs stop
- ecs run: add tag support
Fixup for 294931ecd22f90c1086f0d9994a003dc816da1ff
- Fixup for 294931ecd22f90c1086f0d9994a003dc816da1ff
- ecs run: accept raw numeric cpu, mem values
- Fix broken release
- ecs run: Align default container memory reservation to Fargate task
- Specify default user for batch submit
- Pass through user setting in ecs and batch
- Test improvements
- Use instance id in lieu of public_dns_name when adding host key
- launch: poll instance state; do not assume public DNS name. Fixes #55
- Support ECS EFS volumes
- Enable ssh agent forwarding in sudo shells by default
- Update CRAN mirror for Ubuntu 20.04
- Use bless-provided username when connecting to containers
- aegea ssh: support bless with oidc
- Add ecs ssh
- ecs: give parsers unique names
- batch: give parsers unique names
- ensure_vpc: set tags at create time
- rm: support EIGWs
Bump dependencies
- launch: assign tags at launch time where possible
- version: Print versions of boto3, botocore
- IAM: avoid trying to write policy every time
- Do not assume a private AMI is present
- Add aegea s3 versions, aegea s3 restore
- build-ami: Disable apt-daily-upgrade.service
- Restrict default batch job IAM policies
- build-ami: Make Linux shell profile env file sh compatible
- Fix hostname config when using SSM with Bless
- Save instance public key under correct hostname
Fixup for build-ami
Fixup 2
Revert “Fixup for build_ami”
- Fixup for build_ami
batch submit –wdl –watch: Return WDL output
- ecs watch: make compatible with new ECS task IDs
- Fix s3 buckets options config
- batch submit: Begin WDL support
- Fix batch watch, begin customizable job log printing
- ecr retag: Add repo ID sanity check
- Use ssm for build-ami
- Begin aegea run
- Disable apt-daily.service
- Begin aegea s3 select
- Add aegea ecr retag
- batch submit: set DEBIAN_FRONTEND=noninteractive
- Accelerate aegea buckets ls
- Cap ThreadPoolExecutor workers at 8
Revert “Use setuptools_scm”
- Fixup for build-docker-image
- Use setuptools_scm
Fixup for build-docker-image
- Batch: allow container to be unset
- Add support for client endpoint config
- Speed up APT install for docker builder instance
- Add new regions to VPC base config
- Add aegea sfn stop
- Cosmetic improvements to sfn history
- Add sfn history
- batch: add SSM policy and name tags for CE instances
- Add info logging for batch ssh
- batch ssh: Fix default ssh_args
- Fix aegea ssh
- batch: ask to terminate job on Ctrl-C
- Manage Batch memory quota
- Update boto3 and awscli dependencies to match Ubuntu LTS
- Add AmazonSSMManagedInstanceCore to aegea.launch role
- batch describe: use common helper to pull in description cache
Fix typo in deb package URL
- Use HTTPS to download SM plugin
Use unauthenticated S3 session to download public S3 URL Update readme to mention SSH SSM integration
- ssh: use SSM Session Manager by default
- sfn watch: Return deserialized output for pretty-printing
- Resolve SSH port lazily and without mutable kwarg
- batch: add job description helper lambda
- Print the command that would be run with aegea batch –dry-run (#53)
- sfn describe: allow state machines to be described
- sfn watch: Print Lambda name if available
Add aegea sfn watch
- batch submit: don’t require command override if job definition is set
- Initial release of the aegea sfn family of functions
Fixup for 3b43abdf558cc700dc35218190c54a477783a275
Ignore empty sfn input/output
Add aegea sfn describe
Begin aegea sfn
- build-docker-image: Parameterize Docker image tag
- Update policies in preparation for SSM support
Fixup for 2ef2186e0749e205153374aa6a106379d4e62090
- ebs attach: Fix mkfs defaults
- build-docker-image: add –no-cache option
- logs: add –print-s3-urls
- Parallelize aegea batch ls
- aegea ecs tasks: List all tasks
Stop Ubuntu MOTD spam, part 2
- Stop Ubuntu MOTD spam
- build-docker-image: use Docker cache
- batch submit –execute: Follow s3 redirects for staging bucket url (#51)
- aegea batch terminate: allow multiple job IDs, custom reason
Fix typo
- Batch: configurable staging bucket; use HEAD Bucket
- Fixup for 2600524a76ac1a0373d619ba245955eb40661e92
- Make nvme discovery more defensive
- aegea rm: support removing EC2 launch templates by id
- aegea ecs: do not require ecs:CreateCluster if cluster is present
- Log performance improvements (#50)
- Use CloudWatch log export for aegea logs
- Use CloudWatch Logs Insights for aegea grep
- Add log group name completer
- aegea launch: add Bless support
- Fix bugs in default ephemeral device handling logic
- aegea ssh: Support configurable use_kms_auth
- SpotFleetBuilder: Use AmazonEC2SpotFleetTaggingRole
- aegea scp: Do not crash if no colon is found
- aegea ssh, aegea scp: Add bless support
- aegea ecs: reuse task definitions
- aegea cost: add group by tag support
- aegea batch: format ephemeral storage on host
- Always encrypt EBS volumes
- Fix dockerd configuration for default AMI
- Add aegea cost-forecast
- Begin aegea cost
- Update pricing code to use pricing API
- Add instance type and service name completers
- aegea rm: Fix IAM policy deletion logic
- aegea security-groups: render port ranges correctly
- aegea rds ls: add ARN
- Use getservbyname for SSH port
- Recognize only dash-separated commands
- Test improvements
- aegea launch: add –efs-home and update EFS infra code
- aegea ecs run: set mount_instance_storage to None
- aegea batch update-compute-environment: support zero values
- Fix job definition reuse regression introduced in b00296b
- Centralize sort_by handling
- Fix release of v2.8.0
- aegea batch: ebs cleanup: make resilient to open WD handles
- aegea rds ls lists clusters; add aegea rds instances
- Use AWS_PROFILE, unset AWS_DEFAULT_PROFILE
- aegea iam: do not crash if access is denied to list attached policies
aegea batch: allow ebs shellcode to deal with incorrect usage
- aegea batch: fix install issues in ebs shellcode
- Workaround for Python bug 33666: os.errno was removed in Python 3.7
- Update AMI builder settings
- aegea batch: tab completers, paginators for queues, CEs
- aegea batch: avoid setting resourceRequirements unless needed
- aegea batch: manually construct job definition paginator
- Add helper to get ECS container metadata
Support –profile and –region CLI options
- aegea ssh: fix command building
- Fix zone detection, try 2
- Add aegea lambda update_config
- Tag EBS volumes with managedBy and batch job ID tags
- Refactor DNS default zone management
- Set dev tree version back to placeholder value (0.0.0)
- aegea ebs detach: continue on unmount failure
- aegea ebs: Always print create response even if attach fails
- aegea ebs attach: Fall back to Xen device name
- aegea ebs create: make return value invariant on options
- aegea ebs: Use FS labels to track EBS volumes on non-NVMe instances
- Correctly process ebs_vol_mgr_shellcode string
- aegea iam users: print access keys
- aegea ecs run: Allow Fargate executor to fetch ECR images
- Add aegea batch update-compute-environment
- aegea batch watch: Forward exit code from job
- aegea.util.aws.ensure_iam_role: Fix trust policy handling bug for new roles
- aegea batch: Use ephemeral storage
- aegea batch submit: EBS shellcode fixes
- Updates to aegea ebs and aegea batch submit to better support EBS volume management
- Expand aegea ebs functionality
- aegea ecs run: utilize 4G scratch space
- aegea ecs run: forward exit code from container
- aegea ecs run: set trust policy; allow IAM policies to be updated
- aegea ecs run: fix –execute env var expectations
- aegea ecs watch: Fix for breaking change in ECS API
- aegea logs: fix bug where log_stream was ignored
- aegea launch: Improve help in DNS error message
- aegea ssh: turn on ServerAliveInterval by default
- aegea launch: prefer AMIs built by current user or by Aegea
- aegea ecs run improvements
- Print simple defaults in help messages; consolidate help formatting
- Add aegea ecs
- aegea top: don’t crash on access deny; parallelize query
- aegea batch: include parameter hash in job definiton
Reset job definition namespace
- Fix for v2.3.3 (release only committed changes)
- aegea batch: Look for a matching job definition before creating one
- Avoid crashing when no access is given to MFA status
- aegea launch: Match subnet if AZ is specified
- Allow empty principal in aegea secrets put
- Implement aegea lambda update
- Implement aegea configure set
- Expand aegea –version to print platform details
- Test fixes
- Fix logic error in selecting private DNS zone in aegea launch
- Debug and optimize EC2 pricing API client
- Allow passing of options to scp
- Fix linter errors
- Avoid CVE-2018-1000805
- Move chalice dependency to extras
- Update version range for tweak dependency
- aegea logs: use get_log_events instead of filter_log_events for speed
- Begin aegea config
- Bump keymaker dependency
- Add volume type to batch submit command (#41)
- Fix logic bug in aegea ssh username discovery
- aegea build-ami: Ubuntu 18.04 compatibility
- Get correct IAM username for cross-account SSH
- Bump dependencies
- Bump deps
- Fixups for aegea deploy
- Buildbox usability updates
- Fix Python compat issue in key_fingerprint
- Fix queue naming in aegea-deploy-pilot
- Continue secrets migration
- Fix splitting of deploy systemd unit names
- Begin switching aegea secrets to secretsmanager
- Add Lambda listing parsers
- Bump deps and add common deps per @cschin request
- Fix permissions in cloudinit rootfs.skel input
- Accommodate IAM eventual consistency in instance profiles
- Bump dependencies
- Bump pip ami build dependencies
- Add aegea scp
- Beautify batch shellcode
- aegea launch: add support for EBS volumes via --storage
- aegea --log-level: Remove unneeded NOTSET level
- Expand documentation
- Fix version generation
- aegea batch submit: Use S3 to stage execute payload
- Enable newline formatting and excise comments in ebs shellcode
- kill processes using the filesystem before unmounting (#34)
- aegea batch watch: fix logic error when job fails before starting
- Disable custom Batch AMIs by default
- Make sure version is updated when rolling release
- Fix broken release
- Bump tweak dependency with upstream fix
- Undo changes that had to do with tweak breakage
- fix another typo that was breaking job launch (#33)
- fix batch: newlines and percent characters have special meaning (#32)
- Further ameliorate the volume attach/detach polling issues (#31)
- Limit time we wait for aws detach to succeed before deleting volume (#30)
- Make exception catching more specific
Fix for batch API breaking changes (#25)
- Set default nofile to 100000; lint fixes
- aegea batch submit: Add ability to specify ulimits nofile to conatiner and also adding sensible default (#24)
- Change aegea-deploy service to serve as template, add custom make targets, using one queue per (org, name, branch, instanceid)
- Add iam-role argument to build
- aegea batch watch: Do not crash if log stream does not exist yet
Merge pull request #22 from wholebiome/build-timeout Extend timeout for AMI builds Added timeout to loop, default much longer Fix tests
- Add file missed in 0c99863
- Fix logic error in parameter naming
- Temporarily disable batch custom AMI
- Minor refactor in batch
- Ensure default selection of batch instances has instance storage
- Begin aegea lambda ls, aegea rm --lambda
- Tab complete log levels
- Avoid using pkgutil for introspection
- Batch bug fixes and begin support for custom Batch ECI AMIs
- Install process robustness improvements
- Install documentation improvements
- Don't symlink aegea in bin to avoid pip uninstall bugs
- Resume interrupted release
- Resume interrupted release
- Installation documentation and robustness improvements
- Batch API and mission-specific improvements
- aegea batch: automatic setup of builder IAM policies
- aegea batch submit --job-role: automatic setup of job IAM roles
- aegea batch submit --storage: EBS volume manager
- Autocomplete column titles in listing subcommands where a resource is available
- Autoconfigure a VPC if all VPCs including the default VPC were deleted
- Asset loader: offload rootfs.skel to S3 when user-data exceeds 16K limit
- Arvados updates
- Make missions dir doc link relative (#9)
- Display statusReason in aegea batch ls and aegea batch watch
- Add automatic configuration for route53 private DNS
- Various improvements to aegea batch
- Work around autoloader import issue seen on some Python 2.7 versions
- aegea build_ami: improve progress and error messages
- Fix makefile shell assumption
- Batch WIP
- Test and release infra improvements
- Batch docs
- aegea-build-image-for-mission now builds ECR images by default
- Integration work for Batch
- Add ELB SG configurator, aegea-rebuild-public-elb-sg
- Add awscli to deps
- ELB deploy: set default target group name properly
- Make sure wheel is installed before attempting setup
- Aegea batch submit: Begin CWL support
- Aegea batch watch: amend log line dup fix
- Improvements to aegea batch
- Aegea EFS refactor
- Aegea batch
- Add IP Ranges API
- Add aegea buckets cors placeholder
- Aegea bucket lifecycle
- Test and release infrastructure improvements
- Fogdog mission: add environment placeholder
- Begin timestamp backport
- Propagate base AMI metadata in build_image
- Implement aegea rds snapshot
- Only use pager with pretty-printed tables
- Add Amazon Linux AMI locator
- Use -w0 for auto col width table formatter
- aegea zones update: support multiple updates
- Cosmetic and documentation fixes
- aegea-build-ami-for-mission: skip make if no Makefile
- Begin FogDog mission
- Arvados config support; improve config file handling
- Don't fail cloud-init on account of expected ssh failure
- Run ssh-add from aegea launch
- aegea elb create bugfix
- Fix ELB behavior when TG is present
- Simplify arg forwarding in build_ami
- Support running core aegea on Ubuntu 14.04 vendored Python
- Improve freeform cloud-config-data passing
- Fix pager; introduce --auto-col-width table formatter
- List security groups in elb listing
- Break out and begin buildout of aegea ebs subcommand
- Begin improving rds listings
- Improve DNS zone repr
- New protocol to check out local tracking branch in aegea deploy
- aegea elb create: configurable health check path
- Key cloud-init files manifest by file path to avoid duplicates
- ELB provisioning and listing improvements
- Aegea deploy fixups
- Online documentation improvements
- aegea zones: begin ability to edit records from command line
- Begin support for recursive git clone deploy keys (#4)
- Pretty-print dicts and lists as json in tables
- Logic fixes in elb create command
- Initial support for arvados mission
- Begin work on missions
- aegea-deploy-pilot: admit dashes in branch name via service name
- Fix bug where tweak overwrote config file supplied via environment
- Online documentation improvements
- Aegea build_image renamed to build_ami
- Aegea tag, untag
- Doc improvements
- Ubuntu 14.04 compatibility and role improvements
- docker-event-relay reliability improvements
- Remove snapd from default loadout
- aegea volumes: display attachment instance names
- aegea-deploy-pilot: Deploy on SIGUSR1
- Initial support for flow logs
- Pretty-print and perform whois lookups for aegea security_groups
- aegea ls security_groups: break out protocol into its own column
- Print security group rules in aegea ls security_groups
- List security groups in aegea ls
- Print zone ID in aegea zones
- Aegea deploy reliability improvements: use per-pid queues
- Aegea launch reliability improvements: Back off on polling the EC2 API
- Update release script
- Config updates
- Sort properly while formatting datetimes
- Continue ALB support
- Add babel and format relative dates
- Add aegea elb create
- Changes in support of app deploy infrastructure
- Add R default mirror config
- IAM principal lists now report attached policies
Continue release script
Continue release script
- Introduce rds subcommand
- Rollup: many changes
- Rollup: many changes
- aegea audit implementation (except section 4)
- numerous image improvements
- Rollup: many changes
- Rollup: many changes
- Begin tracking version history
- Expand test suite