Create EKS Node IAM Role

Kubernetes agent, kubelet, runs on every Amazon EKS node and makes calls to multiple AWS APIs. To grant each node sufficient permissions to perform these actions, you need to create an IAM role with the following IAM policies attached to it:

  • AmazonEKSWorkerNodePolicy
  • AmazonEC2ContainerRegistryReadOnly
  • AmazonEKS_CNI_Policy
  • CloudWatchAgentServerPolicy

You need the CloudWatchAgentServerPolicy IAM policy to successfully Enable Logging for your EKS cluster, install the CloudWatch agent on cluster nodes, and allow sending metrics to CloudWatch.

In this section you will create this IAM role for the nodes of your EKS cluster using AWS CloudFormation and GitOps.

Check Your Environment

When working with AWS CloudFormation stacks to manage resources, you not only need sufficient permissions on AWS CloudFormation, but also on the underlying resources that are defined in the template.

To create an IAM role with proper IAM policies attached to it for the nodes of your EKS cluster using AWS CloudFormation you need permission to

  • Deploy AWS CloudFormation stacks.
  • Create IAM roles.
  • Attach managed IAM policies to IAM roles.

Note

If you do not have the above permissions, contact your AWS administrator to grant sufficient permissions to your IAM user or deploy the below AWS CloudFormation stack for you.

Procedure

  1. Go to your GitOps repository, inside your rok-tools management environment:

    root@rok-tools:/# cd ~/ops/deployments
    
  2. Choose a name for the EKS node IAM role:

    root@rok-tools:~/ops/deployments# export EKS_NODE_ROLE=eksNodeRole
    
  3. Generate an AWS CloudFormation stack:

    root@rok-tools:~/ops/deployments# j2 rok/eks/eks-node-iam-role.yaml.j2 -o rok/eks/eks-node-iam-role.yaml
    

    Alternatively, save the AWS CloudFormation Template provided below or download eks-node-iam-role.yaml.j2 and use it locally.

  4. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Configure AWS CF stack for EKS node role"
    
  5. Deploy the AWS CloudFormation stack to create the EKS node IAM role:

    root@rok-tools:~/ops/deployments# aws cloudformation deploy \
    >    --stack-name ${EKS_NODE_ROLE?} \
    >    --template-file rok/eks/eks-node-iam-role.yaml \
    >    --capabilities CAPABILITY_NAMED_IAM
    
    Waiting for changeset to be created..
    Waiting for stack create/update to complete
    Successfully created/updated stack - eksNodeRole
    
    • If the above command fails with:

      Αn error occurred (AccessDenied) when calling the DescribeStacks operation:
      User: arn:aws:iam::409688176173:user/user is not authorized to perform:
      cloudformation:DescribeStacks on resource: arn:aws:cloudformation:eu-central-1:409688176173:stack/eksClusterRole/*
      

      your IAM user does not have sufficient permissions to perform an action necessary to deploy an AWS CloudFormation stack. To proceed, contact your AWS administrator to grant sufficient permissions to your IAM user or deploy the AWS CloudFormation stack for you.

    • If the above command fails with:

      Failed to create/update the stack. Run the following command
      to fetch the list of events leading up to the failure
      aws cloudformation describe-stack-events --stack-name eksNodeRole
      

      describe the events of the CloudFormation stack to identify the root cause of the failure:

      root@rok-tools:~/ops/deployments# aws cloudformation describe-stack-events --stack-name ${EKS_CLUSTER_ROLE}
      

      A stack event like the following:

      {
         "StackId": "arn:aws:cloudformation:eu-central-1:409688176173:stack/eksNodeRole/599bc930-7b3f-11eb-ac1c-029efe3a90a0",
         "EventId": "eksNodeRole-CREATE_FAILED-2021-03-02T10:09:27.457Z",
         "StackName": "eksNodeRole",
         "LogicalResourceId": "eksNodeRole",
         "PhysicalResourceId": "",
         "ResourceType": "AWS::IAM::Role",
         "Timestamp": "2021-03-02T10:09:27.457000+00:00",
         "ResourceStatus": "CREATE_FAILED",
         "ResourceStatusReason": "eksNodeRole already exists",
         "ResourceProperties": "{\"ManagedPolicyArns\":[\"arn:aws:iam::aws:policy/AmazonEKSClusterPolicy\"],\"RoleName\":\"eksNodeRole\",\"AssumeRolePolicyDocument\":{\"Version\":\"2012-10-17\",\"Statement\":[{\"Action\":[\"sts:AssumeRole\"],\"Effect\":\"Allow\",\"Principal\":{\"Service\":[\"eks.amazonaws.com\"]}}]}}"
      }
      

      means that one or more resources that the AWS CloudFormation stack defines already exist, leading to name conflicts.

      A stack event like the following:

      {
         "StackId": "arn:aws:cloudformation:eu-central-1:409688176173:stack/eksNodeRole/415eef80-7b46-11eb-b047-06980f530fec",
         "EventId": "eksNodeRole-CREATE_FAILED-2021-03-02T10:58:54.216Z",
         "StackName": "eksNodeRole",
         "LogicalResourceId": "eksNodeRole",
         "PhysicalResourceId": "",
         "ResourceType": "AWS::IAM::Role",
         "Timestamp": "2021-03-02T10:58:54.216000+00:00",
         "ResourceStatus": "CREATE_FAILED",
         "ResourceStatusReason": "API: iam:CreateRole User: arn:aws:iam::409688176173:user/user is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::409688176173:role/eksNodeRole",
         "ResourceProperties": "{\"ManagedPolicyArns\":[\"arn:aws:iam::aws:policy/AmazonEKSClusterPolicy\"],\"RoleName\":\"eksClusterRole\",\"AssumeRolePolicyDocument\":{\"Version\":\"2012-10-17\",\"Statement\":[{\"Action\":[\"sts:AssumeRole\"],\"Effect\":\"Allow\",\"Principal\":{\"Service\":[\"eks.amazonaws.com\"]}}]}}"
      }
      

      means that your IAM user does not have sufficient permissions to create the resources that the AWS CloudFormation stack defines.

      To proceed, contact your AWS administrator to grant your IAM user sufficient permissions or deploy the AWS CloudFormation stack for you.

    After a while the deployment of the AWS CloudFormation stack completes successfully, ensuring that the EKS node IAM role exists.

Verify

  1. Verify that the EKS node IAM role exists:

    root@rok-tools:/# aws iam get-role --role-name ${EKS_NODE_ROLE?}
    
  2. Verify that the EKS node IAM role has the AmazonEKSWorkerNodePolicy, AmazonEC2ContainerRegistryReadOnly, AmazonEKS_CNI_Policy and CloudWatchAgentServerPolicy IAM policies attached to it:

    root@rok-tools:/# aws iam list-attached-role-policies --role-name ${EKS_NODE_ROLE?}
    {
       "AttachedPolicies": [
          {
                "PolicyName": "AmazonEKSWorkerNodePolicy",
                "PolicyArn": "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
          },
          {
                "PolicyName": "AmazonEC2ContainerRegistryReadOnly",
                "PolicyArn": "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
          },
          {
                "PolicyName": "AmazonEKS_CNI_Policy",
                "PolicyArn": "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
          },
          {
                "PolicyName": "CloudWatchAgentServerPolicy",
                "PolicyArn": "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
          }
       ]
    }
    
  3. Verify that the stack’s configuration has not drifted from the expected configuration, i.e., actual resources have not deviated from their definition in the AWS CloudFormation template:

    root@rok-tools:/# aws cloudformation detect-stack-drift --stack-name ${EKS_CLUSTER_ROLE?}
    
    root@rok-tools:/# aws cloudformation describe-stack-resource-drifts \
    >    --stack-name ${EKS_NODE_ROLE?} \
    >    | jq -r .StackResourceDrifts[0].StackResourceDriftStatus
    IN_SYNC
    

Summary

You have successfully created an IAM role for the nodes of your new EKS cluster using AWS CloudFormation and GitOps.

What’s Next

The next step is to select a Virtual Private Cloud (VPC) for your new EKS cluster.