[aws-eks] OpenID Connect provider association with AWS account

[lkoniecz]

Freature refers to fine-grained IAM roles for service accounts. In this approach, pods are pods first class citizens in IAM. To make the feature work, OpenID provider which comes with EKS needs to be associated with AWS account. More one here: https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html

eksctl does that in one command. Unfortunatelly there is no native cloudformation support so it cannot be set up automatically.

The goal would be to add a parameter to EKS cluster constructor, which would let programmer decide whether to associate the EKS OpenID with AWS IAM.

Use Case

I'd like to use AWS native support for IAM roles for pods rather than 3rd party solutions like role-credential interceptors (kube2iam for instance).

Proposed Solution

kubectl_enabled flag (introduced in CDK 1.4, if I recall correctly), changes the approach to the way how EKS is provisioned. You could follow the same route and perform custom logic after cluster is deployed. The logic would need to perform following steps (more about this here: https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html):

• retrieve OID provider URL from the cluster
• create open id identity provider (https://docs.aws.amazon.com/cli/latest/reference/iam/create-open-id-connect-provider.html). The only difficulty is to get the thumbprint (more about this here: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html)

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[vlesierse]

Currently I'm investigating the possibilities to implement this feature and my suggestion is to implement this in the ClusterResourceProvider.

When the cluster is active we can obtain the oidc issuer url from cluster.identity.oidc.issuer and use this information to obtain the CA thumbprint. I've got most of the code ready locally to do this.

To enable this behaviour we can either add a property to the ClusterProps like associateOIdCProvider or do this lazy when you use cluster.addServiceAccount(<Name>, <IAM Role>)

Think I'll have an initial PR ready early next week.

[eladb]

Sounds great. Would love this contribution!

[hlascelles]

Until this PR is closed off...

Native CloudFormation support still doesn't exist, but if you would like to create an OIDC provider using pure YAML / JSON (using inlined lambdas) we've written that up here: https://bambooengineering.io/configuring-eks-for-iam-oidc-using-cloudformation/

The full demo with templates is in this repo.

[vlesierse]

Thank you for this direction I’ll definitely take a look. Was planning to continue this weekend and will explore this options.

[vlesierse]

PR #6062 has now a working implementation to support this. It add a new ServiceAccount construct which you can use to add IAM policies. Using the service account with you Pod will result in a least privilege access policies to AWS resources.

Working example:

const account = new ServiceAccount(this, 'ALBIngressController', {
cluster: cluster,
name: 'alb-ingress-controller',
namespace: 'kube-system'
});
account.addToPolicy(
new PolicyStatement({
  resources: ['*'],
  actions: [
    "acm:DescribeCertificate",
    "acm:ListCertificates",
    "acm:GetCertificate",
    "ec2:AuthorizeSecurityGroupIngress",
    "ec2:CreateSecurityGroup",
    "ec2:CreateTags",
    "ec2:DeleteTags",
    "ec2:DeleteSecurityGroup",
    "ec2:DescribeAccountAttributes",
    "ec2:DescribeAddresses",
    "ec2:DescribeInstances",
    "ec2:DescribeInstanceStatus",
    "ec2:DescribeInternetGateways",
    "ec2:DescribeNetworkInterfaces",
    "ec2:DescribeSecurityGroups",
    "ec2:DescribeSubnets",
    "ec2:DescribeTags",
    "ec2:DescribeVpcs",
    "ec2:ModifyInstanceAttribute",
    "ec2:ModifyNetworkInterfaceAttribute",
    "ec2:RevokeSecurityGroupIngress",
    "elasticloadbalancing:AddListenerCertificates",
    "elasticloadbalancing:AddTags",
    "elasticloadbalancing:CreateListener",
    "elasticloadbalancing:CreateLoadBalancer",
    "elasticloadbalancing:CreateRule",
    "elasticloadbalancing:CreateTargetGroup",
    "elasticloadbalancing:DeleteListener",
    "elasticloadbalancing:DeleteLoadBalancer",
    "elasticloadbalancing:DeleteRule",
    "elasticloadbalancing:DeleteTargetGroup",
    "elasticloadbalancing:DeregisterTargets",
    "elasticloadbalancing:DescribeListenerCertificates",
    "elasticloadbalancing:DescribeListeners",
    "elasticloadbalancing:DescribeLoadBalancers",
    "elasticloadbalancing:DescribeLoadBalancerAttributes",
    "elasticloadbalancing:DescribeRules",
    "elasticloadbalancing:DescribeSSLPolicies",
    "elasticloadbalancing:DescribeTags",
    "elasticloadbalancing:DescribeTargetGroups",
    "elasticloadbalancing:DescribeTargetGroupAttributes",
    "elasticloadbalancing:DescribeTargetHealth",
    "elasticloadbalancing:ModifyListener",
    "elasticloadbalancing:ModifyLoadBalancerAttributes",
    "elasticloadbalancing:ModifyRule",
    "elasticloadbalancing:ModifyTargetGroup",
    "elasticloadbalancing:ModifyTargetGroupAttributes",
    "elasticloadbalancing:RegisterTargets",
    "elasticloadbalancing:RemoveListenerCertificates",
    "elasticloadbalancing:RemoveTags",
    "elasticloadbalancing:SetIpAddressType",
    "elasticloadbalancing:SetSecurityGroups",
    "elasticloadbalancing:SetSubnets",
    "elasticloadbalancing:SetWebACL",
    "iam:CreateServiceLinkedRole",
    "iam:GetServerCertificate",
    "iam:ListServerCertificates",
    "waf:GetWebACL",
    "waf-regional:GetWebACLForResource",
    "waf-regional:GetWebACL",
    "waf-regional:AssociateWebACL",
    "waf-regional:DisassociateWebACL",
    "tag:GetResources",
    "tag:TagResources"
  ]
}));

cluster.addResource('ALBIngressControllerRBAC',
  {
    "apiVersion": "rbac.authorization.k8s.io/v1",
    "kind": "ClusterRole",
    "metadata": {
      "labels": {
        "app.kubernetes.io/name": account.serviceAccountName
      },
      "name": account.serviceAccountName
    },
    "rules": [
      {
        "apiGroups": [
          "",
          "extensions"
        ],
        "resources": [
          "configmaps",
          "endpoints",
          "events",
          "ingresses",
          "ingresses/status",
          "services"
        ],
        "verbs": [
          "create",
          "get",
          "list",
          "update",
          "watch",
          "patch"
        ]
      },
      {
        "apiGroups": [
          "",
          "extensions"
        ],
        "resources": [
          "nodes",
          "pods",
          "secrets",
          "services",
          "namespaces"
        ],
        "verbs": [
          "get",
          "list",
          "watch"
        ]
      }
    ]
  },
  {
    "apiVersion": "rbac.authorization.k8s.io/v1",
    "kind": "ClusterRoleBinding",
    "metadata": {
      "labels": {
        "app.kubernetes.io/name": account.serviceAccountName
      },
      "name": account.serviceAccountName
    },
    "roleRef": {
      "apiGroup": "rbac.authorization.k8s.io",
      "kind": "ClusterRole",
      "name": account.serviceAccountName
    },
    "subjects": [
      {
        "kind": "ServiceAccount",
        "name": account.serviceAccountName,
        "namespace": "kube-system"
      }
    ]
  });
cluster.addChart('ALBIngressController', {
  chart: 'aws-alb-ingress-controller',
  repository: 'http://storage.googleapis.com/kubernetes-charts-incubator',
  namespace: 'kube-system',
  values: {
    autoDiscoverAwsRegion: true,
    autoDiscoverAwsVpcID: true,
    clusterName: cluster.clusterName,
    rbac: {
      create: false,
      serviceAccountName: account.serviceAccountName
    }
  }
});