Deploy Fluent Bit¶
This guide will walk you through setting up Fluent Bit to send logs to Amazon CloudWatch Logs.
Fast Forward
If you already have deployed Fluent Bit, expand this box to fast-forward.
- Proceed to the Verify section.
Overview
What You’ll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing EKS cluster.
- An existing IAM role for Fluent Bit.
Procedure¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsRestore the required context from previous sections:
root@rok-tools:~/ops/deployments# source <(cat deploy/env.{envvars-aws,\ > eks-cluster,fluent-bit-eks-iam})root@rok-tools:~/ops/deployments# export AWS_DEFAULT_REGION AWS_ACCOUNT_ID \ > EKS_CLUSTER FLUENT_BIT_EKS_IAM_ROLERender the
ConfigMap
patch template using the environment variables you specified:root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/configmap.yamlRender the
ServiceAccount
patch template with the variables you have specified:root@rok-tools:~/ops/deployments# j2 rok/amazon-cloudwatch/overlays/deploy/patches/sa.yaml.j2 \ > -o rok/amazon-cloudwatch/overlays/deploy/patches/sa.yamlCommit your changes:
root@rok-tools:~/ops/deployments# git commit -am "Deploy Fluent Bit"Apply the manifests:
root@rok-tools:~/ops/deployments# rok-deploy --apply rok/amazon-cloudwatch/overlays/deploy/Note
We use the default Fluent Bit optimized configuration that is aligned with Fluent Bit best practices.
Note
By default, the retention of log groups that Fluent Bit creates on Amazon CloudWatch Logs is set to
Never expire
.
Verify¶
Verify that the Fluent Bit
DaemonSet
is ready. Verify that fields READY and UP-TO-DATE are equal to field DESIRED:root@rok-tools:~/ops/deployments# kubectl get ds -n amazon-cloudwatch fluent-bit NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE fluent-bit 2 2 2 2 2 <none> 2mVerify that you have enabled logging for your containers and worker nodes. Ensure that the corresponding log groups have been created in Amazon CloudWatch Logs:
root@rok-tools:~/ops/deployments# aws logs describe-log-groups \ > --log-group-name-prefix /aws/containerinsights/${EKS_CLUSTER?} \ > --query logGroups[].[logGroupName] --output text /aws/containerinsights/arrikto-cluster/application /aws/containerinsights/arrikto-cluster/dataplane /aws/containerinsights/arrikto-cluster/host
Summary¶
You have successfully deployed Fluent Bit on your EKS cluster and configured it to forward logs to Amazon CloudWatch Logs.
What’s Next¶
Check out the rest of the maintenance operations that you can perform on your cluster.