Deploy Fluent Bit

This guide will walk you through setting up Fluent Bit to send logs to Amazon CloudWatch Logs.

Procedure

  1. Go to your GitOps repository, inside your rok-tools management environment:

    root@rok-tools:~# cd ~/ops/deployments
  2. Restore the required context from previous sections:

    root@rok-tools:~/ops/deployments# source <(cat deploy/env.{envvars-aws,\ > eks-cluster,fluent-bit-eks-iam})
    root@rok-tools:~/ops/deployments# export AWS_DEFAULT_REGION AWS_ACCOUNT_ID \ > EKS_CLUSTER FLUENT_BIT_EKS_IAM_ROLE
  3. Render the ConfigMap patch template using the environment variables you specified:

    root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yaml
  4. Render the ServiceAccount patch template with the variables you have specified:

    root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/sa.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/sa.yaml
  5. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Deploy Fluent Bit"
  6. Apply the manifests:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/amazon-cloudwatch/overlays/deploy/

    Note

    We use the default Fluent Bit optimized configuration that is aligned with Fluent Bit best practices.

    Note

    By default, the retention of log groups that Fluent Bit creates on Amazon CloudWatch Logs is set to Never expire.

Verify

  1. Verify that the Fluent Bit DaemonSet is ready. Verify that fields READY and UP-TO-DATE are equal to field DESIRED:

    root@rok-tools:~/ops/deployments# kubectl get ds -n amazon-cloudwatch fluent-bit NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE fluent-bit 2 2 2 2 2 <none> 2m
  2. Verify that you have enabled logging for your containers and worker nodes. Ensure that the corresponding log groups have been created in Amazon CloudWatch Logs:

    root@rok-tools:~/ops/deployments# aws logs describe-log-groups \ > --log-group-name-prefix /aws/containerinsights/${EKS_CLUSTER?} \ > --query logGroups[].[logGroupName] --output text /aws/containerinsights/arrikto-cluster/application /aws/containerinsights/arrikto-cluster/dataplane /aws/containerinsights/arrikto-cluster/host

Summary

You have successfully deployed Fluent Bit on your EKS cluster and configured it to forward logs to Amazon CloudWatch Logs.

What’s Next

Check out the rest of the maintenance operations that you can perform on your cluster.