Skip to content

Commit

Permalink
Remove google specifics. Fix some minor issues.
Browse files Browse the repository at this point in the history
  • Loading branch information
@david-mcmahon
david-mcmahon committed Oct 11, 2016
1 parent cf9dc25 commit 84b3b91
Show file tree
Hide file tree
Showing 8 changed files with 119 additions and 193 deletions.
9 changes: 5 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ This repo contains the release infrastructure for

## Intro

Kubernetes releases are done by the Kubernetes team at Google due to
Live Kubernetes releases are done by the Kubernetes team at Google due to
permissions and other restrictions. This may expand eventually to allow
other Kubernetes contributors to generate releases.

The current process runs by default in *mock* mode and anyone should
The current tooling runs by default in *mock* mode and anyone should
be able to run it in this mode to see exactly how the process works.
In *mock* mode all the code paths are followed for a release, but nothing
is pushed to repositories.
Expand Down Expand Up @@ -48,6 +48,8 @@ run the script with --nomock to perform a real release.
The tool was designed to require minimal inputs.
The only information the tool needs is to know where you want to create a
release with one optional flag `[--official]` \(used on release-\* branches only\).
The [--official] flag indicates that a standard patch release will be created
on a branch. Without the [--official] flag, a beta would be created.

Try an alpha release:
```
Expand Down Expand Up @@ -78,7 +80,7 @@ v9.9.9 tag on the release-9.9 branch, create a release-9.9.9 branch):
$ anago release-9.9.9
```

## Official Releases
## Live Releases

Anago is currently locked down to only run for a specific set of individuals.
when ```--nomock``` is specified.
Expand All @@ -94,7 +96,6 @@ your favorite editor.
### Tools

* [prin](https://github.com/kubernetes/release/blob/master/prin) : What tags/releases is my PR IN?
* [mailer](https://github.com/kubernetes/release/blob/master/mailer) : Generic mail interface (due to Google's deprecation of sendmail)
* [find_green_build](https://github.com/kubernetes/release/blob/master/find_green_build) : Ask Jenkins for a good build to use
* [script-template](https://github.com/kubernetes/release/blob/master/script-template) : Generate a script template in the kubernetes/release ecosystem
* [relnotes](https://github.com/kubernetes/release/blob/master/relnotes) : Scrape github for release notes \(See below for more info\)
Expand Down
56 changes: 36 additions & 20 deletions anago
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ PROG=${0##*/}
#+ $PROG [--yes] [--nomock] [--noclean] [--official]
#+ [--buildversion=<jenkins build version>]
#+ [--basedir=<alt base work dir>] <branch>
#+ [--security_layer=/path/to/pointer/to/script]
#+ $PROG [--helpshort|--usage|-?]
#+ $PROG [--help|-man]
#+
Expand Down Expand Up @@ -79,6 +80,10 @@ PROG=${0##*/}
#+ build version
#+ [--basedir=dir] - Specify an alternate base directory
#+ (default: /usr/local/google/$USER)
#+ [--security_layer=] - A file containing a path to a script to
#+ source/include:
#+ FLAGS_security_layer=/path/to/script
#+ Default: $HOME/.kubernetes-releaserc
#+ [--help | -man] - display man page for this script
#+ [--usage | -?] - display in-line usage
#+
Expand Down Expand Up @@ -161,10 +166,10 @@ common::cleanexit () {
# Simple ACL check to limit nomock runs to a short list of release folks
check_acls () {
case "$USER" in
filipg|robertbailey|fabioy|djmm|pwittroc|stclair|mtaufen) ;;
*) logecho "Releases restricted to certain users!"
return 1
;;
$ACL_LIST) ;;
*) logecho "Live releases restricted to certain users!"
return 1
;;
esac
}

Expand All @@ -174,13 +179,10 @@ check_acls () {
# @param package - A space separated list of packages to verify exist
#
check_prerequisites () {
local useratgoog="$USER@google.com"
local userat="$USER@$DOMAIN_NAME"
local tempfile=/tmp/$PROG-cp.$$

# TODO: stub out for non-google
# Verify some hours of LOAS
logecho -n "Checking LOAS state: "
logrun -s common::loascheck 1 || return 1
security_layer::auth_check 2

if ! common::set_cloud_binaries; then
logecho "Releasing Kubernetes requires gsutil and gcloud. Please download,"
Expand All @@ -190,11 +192,11 @@ check_prerequisites () {
return 1
fi

# TODO: Users outside google? Guess domain?
# TODO: THe real test here is to verify that whatever auth has access to
# TODO: Users outside google? Ask/derive domain?
# TODO: The real test here is to verify that whatever auth has access to
# do releasey things
gcloud_auth_list=$($GCLOUD auth list 2>/dev/null)
for user in k8s.production.user@gmail.com $useratgoog; do
for user in $G_AUTH_USER $userat; do
logecho -n "Checking cloud auth for $user: "
if [[ "$gcloud_auth_list" =~ -\ $user ]]; then
logecho -r "$OK"
Expand All @@ -208,11 +210,11 @@ check_prerequisites () {
fi
done
# Ensure $USER is active to start
if ! [[ "$gcloud_auth_list" =~ -\ $useratgoog\ ACTIVE ]]; then
logecho "$useratgoog is not the active gcloud user!"
if ! [[ "$gcloud_auth_list" =~ -\ $userat\ ACTIVE ]]; then
logecho "$userat is not the active gcloud user!"
logecho "Set with:"
logecho
logecho "$ gcloud config set account $useratgoog"
logecho "$ gcloud config set account $userat"
return 1
fi
GCLOUD_ACCOUNT=$user
Expand Down Expand Up @@ -377,7 +379,7 @@ prepare_tree () {
# the NON-master branch itself, versionize the docs
if [[ "$PARENT_BRANCH" == master && "$current_branch" != "master" ]]; then
logecho -n "Versionizing docs for ${RELEASE_VERSION[$label]}: "
logrun -s $TREE_ROOT/build/versionize-docs.sh $current_branch
logrun -s $TREE_ROOT/build/versionize-docs.sh $current_branch || return 1
logecho -n "Committing: "
logrun git commit -am \
"Versioning docs and examples for ${RELEASE_VERSION[$label]} on $current_branch."
Expand Down Expand Up @@ -524,9 +526,10 @@ announce () {
# Due to announcements landing on public mailing lists requiring membership,
# post from the invoking user (for now until this is productionized further)
# and use reply-to to ensure replies go to the right place.
mailer --to="$mailto" --cc="$USER" --file="$announcement_text" --html \
--replyto="K8s-Anago<cloud-kubernetes-release@google.com>" \
--from="K8s-Anago<$USER@google.com>" --subject="$subject" || return 1
common::sendmail "$mailto" "K8s-Anago<$USER@$DOMAIN_NAME>" \
"K8s-Anago<cloud-kubernetes-release@google.com>" \
"$subject" "$USER" \
"$announcement_text" --html || return 1

logrun rm -f $announcement_text
}
Expand Down Expand Up @@ -821,18 +824,31 @@ common::logfileinit $LOGFILE 10
# BEGIN script
common::timestamp begin

# Additional functionality
common::security_layer

##############################################################################
common::stepheader "CHECK CREDENTIALS"
##############################################################################
gitlib::check_credentials

# Domain check
if [[ $(hostname) =~ \.([^\.]+\.com)$ ]]; then
DOMAIN_NAME=${FLAGS_domain:-${BASH_REMATCH[1]}}
if ! [[ -n $DOMAIN_NAME ]]; then
common::exit 1 "Unable to determine your domain." \
"Pass it in on the command-line" \
"with --domain=<yourdomain.com>"
fi
fi

# Simple check to validate who can do actual releases
((FLAGS_nomock)) && check_acls

##############################################################################
common::stepheader "CHECK PREREQUISITES"
##############################################################################
common::check_packages jq docker-engine pandoc sendgmr \
common::check_packages jq docker-engine pandoc ${PREREQUISITE_PACKAGES[*]} \
|| common::exit 1 "Exiting..."
check_prerequisites || common::exit 1 "Exiting..."

Expand Down
3 changes: 3 additions & 0 deletions find_green_build
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ common::cleanexit () {
# BEGIN script
common::timestamp begin

# Force verbose flag
FLAGS_verbose=1

# Initialize and save up to 10 (rotated logs)
MYLOG=/tmp/$PROG.log
common::logfileinit $MYLOG 10
Expand Down
121 changes: 64 additions & 57 deletions lib/common.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -633,67 +633,26 @@ common::sha () {
# * $HOME/${PROG}rc (FLAGS_security_layer=/path/to/source)
# SECURITY_LAYER global defaulted here. Set to 1 in external source
common::security_layer () {
local rcfile=$HOME/.kubernetes-releaserc
SECURITY_LAYER=0
# Quietly source the pointer
source $HOME/.${PROG}rc >/dev/null 2>&1 || return 0

# Source the include
if [[ $(hostname -d) =~ google.com ]]; then
if [[ -f $rcfile ]]; then
source $rcfile >/dev/null 2>&1
else
logecho "$WARNING: This session is incomplete. go/$PROG"
logecho
fi
fi

# If not there attempt to set it from env
FLAGS_security_layer=${FLAGS_security_layer:-":"}

[[ -s $FLAGS_security_layer ]] || return 0
source $FLAGS_security_layer
source $FLAGS_security_layer >/dev/null 2>&1
}

###############################################################################
# Check state of LOAS
#
# @param hour - hours remaining on prodcert
# @optparam user - Check a specific user
# Returns:
# 0 if LOAS is active and will still be active for more than N hours.
# 1 if LOAS is inactive or will expire in less than N hours.
common::loascheck () {
local hour=${1:-0}
local user=${2:-}
local becomeuser=''
local athostname=''
local isroleacct=0
local tty_session=1

# Role account?
id |fgrep -q role-accts && isroleacct=1

# Interactive/tty session?
tty -s || tty_session=0

[[ -n "$user" ]] && becomeuser="become $user --"

if ! $becomeuser \
prodcertstatus --quiet --check_remaining_hours=$hour &>/dev/null; then
logecho
logecho "EXCEPTION: LOAS credentials for ${user:-$USER} will expire" \
"in $hour hours. Run:"

if [[ -n "$user" ]] || (($isroleacct)); then
# If run via cron, assume we need to instruct user where to run
# prodaccess
(($tty_session)) || athostname="@$HOSTNAME"

logecho "$ become -t ${user:-$LOGNAME}$athostname -- prodaccess --sslenroll"
else
logecho "$ prodaccess"
fi
return 1
else
# Issue a WARNING if we get close to a reasonable threshold
# The latest version of prodcertstatus spews useless text into stderr
# and we want to display the useful part of the stderr so explicitly
# strip it out
# We also have to do a dance with stderr/stdout
$becomeuser prodcertstatus --check_remaining_hours=4 2>&1 1>/dev/null |\
egrep -v 'Reusing existing SSO cookie.'
return 0
fi
}

###############################################################################
# Check packages for a K8s release
Expand Down Expand Up @@ -733,10 +692,11 @@ common::check_packages () {
"Run the following and try again:"
logecho
for prereq in ${missing[@]}; do
if [[ $prereq == "sendgmr" ]] && [[ $distro == "Ubuntu" ]]; then
logecho "sudo goobuntu-add-repo $prereq && sudo apt-get update"
if [[ -n ${PREREQUISITE_INSTRUCTIONS[$prereq]} ]]; then
logecho "# See ${PREREQUISITE_INSTRUCTIONS[$prereq]}"
else
logecho "$ sudo $packagemgr install $prereq"
fi
logecho "sudo $packagemgr install $prereq"
done
return 1
fi
Expand Down Expand Up @@ -849,6 +809,53 @@ common::set_cloud_binaries () {
fi
}

###############################################################################
# sendmail/mailer front end.
# @optparam html - Send html formatted
# @param to - To
# @param from - From
# @param reply_to - Reply To
# @param subject - Subject
# @param cc - cc
# @param file - file to send
#
common::sendmail () {
local cc_arg
local html=0

while [[ "$#" -gt 0 ]]; do
case "$1" in
-h) html=1; shift ;;
*) continue ;;
esac
done

local to="$1"
local from="$2"
local reply_to="$3"
local subject="$4"
local cc="$5"
local file="$6"

(
cat <<EOF+
To: "$to"
From: "$from"
Subject: "$subject"
Cc: "$cc"
Reply-To: "$reply_to"
EOF+
((html)) && echo "Content-Type: text/html"
cat $file
) |/usr/sbin/sendmail -t
}

# Stubs for security_layer functions
security_layer::auth_check () {
logecho "Skipping $FUNCNAME..."
return 0
}

# Set a common::trap() to capture ^C's and other unexpected exits and do the
# right thing in common::trapclean().
common::trap common::trapclean ERR SIGINT SIGQUIT SIGTERM SIGHUP
Expand Down
1 change: 0 additions & 1 deletion lib/gitlib.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ GHCURL="curl -s --fail --retry 10 -u ${GITHUB_TOKEN:-$FLAGS_github_token}:x-oaut
JCURL="curl -g -s --fail --retry 10"
K8S_GITHUB_API='https://api.github.com/repos/kubernetes/kubernetes'
K8S_GITHUB_URL='https://github.com/kubernetes/kubernetes'
JENKINS_URL="http://kubekins.dls.corp.google.com/job"

# Regular expressions for bash regex matching
# 0=entire branch name
Expand Down
6 changes: 0 additions & 6 deletions lib/releaselib.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,6 @@ release::set_build_version () {
local main_job="kubernetes-e2e-gce$branch_suffix"
local job_path=/tmp/buildresults-cache.$$
local -a JOB

# Would be nice to pull/generate these jobs dynamically filtered through
# a pattern like gce,gke so new/changed testing jobs don't require
# updating here, but it turns out way too much hardcoding would need
# to be done to make this 'dynamic', so just list these explicitly.
#all_jobs=$($JCURL -s "http://kubekins.dls.corp.google.com/view/Critical%20Builds/api/json" |jq -r '.jobs[] | .name' | egrep -v '-soak-|-build')
local -a gce_jobs=("kubernetes-e2e-gce-serial$branch_suffix"
"kubernetes-e2e-gce-slow$branch_suffix"
"kubernetes-kubemark-5-gce$branch_suffix"
Expand Down
Loading

0 comments on commit 84b3b91

Please sign in to comment.