Deploy FluentD

This guide will walk you through setting up FluentD to send logs to CloudWatch Logs.

Fast Forward

If you already have deployed FluentD, expand this box to fast-forward.

  1. Proceed to the Verify section.

See also

Procedure

  1. Go to your GitOps repository, inside your rok-tools management environment:

    root@rok-tools:~# cd ~/ops/deployments

  2. Restore the required context from previous sections:

    root@rok-tools:~/ops/deployments# source <(cat deploy/env.{envvars-aws,\ > eks-cluster,fluentd-eks-iam})
    root@rok-tools:~/ops/deployments# export AWS_DEFAULT_REGION AWS_ACCOUNT_ID \ > EKS_CLUSTER FLUENTD_EKS_IAM_ROLE

  3. Render the ConfigMap patch template with the variables you have specified:

    root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yaml

  4. Render the ServiceAccount patch template with the variables you have specified:

    root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/sa.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/sa.yaml

  5. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Configure FluentD CloudWatch agent"

  6. Apply the manifests:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/amazon-cloudwatch/overlays/deploy/

Verify

  1. Verify that FluentD DaemonSet is up-and-running. Verify fields READY and UP-TO-DATE are equal to field DESIRED:

    root@rok-tools:~/ops/deployments# kubectl get ds -n amazon-cloudwatch NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE fluentd-cloudwatch 2 2 2 2 2 <none> 1m

  2. Verify that you have enabled logging for your containers and worker nodes by checking that the corresponding log groups have been created in Amazon CloudWatch Logs:

    root@rok-tools:~/ops/deployments# aws logs describe-log-groups \ > --log-group-name-prefix /aws/containerinsights/${EKS_CLUSTER?} \ > --query logGroups[].[logGroupName] --output text /aws/containerinsights/arrikto-cluster/application /aws/containerinsights/arrikto-cluster/dataplane /aws/containerinsights/arrikto-cluster/host

Summary

You have successfully deployed FluentD on your EKS cluster.

What’s Next

Check out the rest of the EKS maintenance operations that you can perform on your cluster.

PreviousNext