How to connect to AWS EKS using IAM role?

Questions and Answers
1

Hi,

It is typical to split AWS organization in multiple accounts to improve security management. We follow this practice and isolated Qovery managed cluster to a separate account (account B). We also have a Developer role in account B determining the developer permissions inside the account.

In this guide Qovery asks to create an IAM user and assign it to Admins group in order to connect to Kubernetes cluster Qovery. Is it possible to connect using IAM roles?

We are trying to connect via Terraform but we get “Kubernetes cluster unreachable: the server has asked for the client to provide credentials”.

provider "aws" {
  region = "eu-west-1"

  assume_role {
    role_arn = "arn:aws:iam::111111111111:role/Developer"
  }
}

data "aws_eks_cluster" "aws-production" {
  name = var.qovery_aws_cluster_name
}

data "aws_eks_cluster_auth" "aws-production" {
  name = var.qovery_aws_cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.aws-production.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.aws-production.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.aws-production.token
}

This approached worked fine with manually deployed clusters. Is there anything specific Qovery doing to limit the access to the cluster? How to configure it to allow access from IAM roles?

2

Hello @prki ,

I’ll have a look at your issue with the team and get back to you.

Regards,
Charles-Edouard

4

Hello @prki ,

I checked with the team and this looks like an AWS configuration.

We found some information on this blog post and in these documentation: Terrafom and AWS

According to the AWS documentation, you need to do something like this to get your EKS token:

aws eks get-token \
    --cluster-name my-eks-cluster \
    --role-arn arn:aws:iam::111122223333:role/eksctl-EKS-Linux-Cluster-v1-24-cluster-ServiceRole-j1k7AfTIQtnM

Can you try this and let us know if it helped you solve your problem?

Regards,
Charles-Edouard

5

Hi Charles,

Yes, I tried using get-token with role-arn without success. I gave the role AdministratorAccess policy for testing purposes but it still can’t connect to EKS via kubectl. Is there anything additional to be done to give IAM role access to EKS?

provider "kubernetes" {
  host                   = data.aws_eks_cluster.aws-production.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.aws-production.certificate_authority[0].data)
  exec {
    api_version = "client.authentication.k8s.io/v1"
    command     = "aws"
    args = [
      "eks", "get-token",
      "--cluster-name", var.qovery_aws_cluster_name,
      "--role-arn", var.aws_role
    ]
  }
}
6

Hello @prki :wave:

Can you add your Developer IAM role to the aws-auth config map of your cluster ? (you can use eksctl to do it: Manage IAM users and roles - eksctl). Then try to re-apply your terraform.

To give you more details, we are using the following tool to automatically inject the users from iam to this aws-auth config map (the users in the Admins group): GitHub - Qovery/iam-eks-user-mapper: A tool to automatically give AWS IAM users access to your Kubernetes cluster.

7

I will try adding the role to aws-auth config map.

Is it safe to modify it and it won’t be reverted by Qovery? Ideally we would like a way to do it programatically in Terraform, perhaps as part of cluster config in Qovery?

8

Yes it’s safe to modify it, be careful to not delete the existing mappings that already exist by running a wrong eksctl delete command => what you can do to be sure is to get the aws-auth configmap content before making your modifications, so you’ll be safe to rollback if needed.

Qovery won’t revert it at next cluster deployment, it’s a one shot modification.

At of now we don’t have this option cluster side, we can maybe think of improve this part cc @Julien_Dan

9

Thanks @Melvin_Zottola, adding the role to aws-auth config map worked.

Admins group syncing is already part of cluster advanced configuration, it would be nice if it also had an option to enable access for the roles specified.

1 Like
10

I @prki , thanks for the feedback I’ve opened an item on our public roadmap to track this

1 Like
11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.