Fluentd
Installing Fluentd: A Step-by-Step Guide
Prince Onyeanuna
February 3, 2025
Published on
February 3, 2025
Read More
Divine Odazie
19 Jan, 2025

Now that you've been introduced to Fluentd, there's no need to let that information get stale. Let's get to work.

This guide provides step-by-step instructions on how to install Fluentd on different platforms. Choose the installation guide that suits your operating system.

For Windows or MacOS users, you can use the installation instructions from the source, Docker, or Kubernetes.

General prerequisites

Before installing Fluentd, you want to ensure that your system meets the following requirements:

  • System Requirements: Sufficient CPU and memory to handle the expected data load.
  • Permissions: Administrative privileges to install packages and modify system configurations.

Installation on Ubuntu (Debian-based systems)

Fluentd provides stable DEB packages under two distributions:

  • fluent-package (maintained by the Fluentd project)
  • calyptia-fluentd (maintained by Chronosphere).

Step 1: Install Fluentd via the Apt repository

Below are several commands for various system versions. Choose the one compactable with your system and run it.

For Fluent-package

For Calyptia-Fluentd

After running the command, you should get an output similar to this:

==============================
fluent-package Installation Script
==============================
This script requires superuser access to install apt packages.
You will be prompted for your password by sudo.
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  7636  100  7636    0     0   191k      0 --:--:-- --:--:-- --:--:--  196k

Step 2: Launch Fluentd

The next step is to start the Fluentd service using systemd. The following command will help you do this and check the current status of Fluentd:

sudo systemctl start fluentd.service  
sudo systemctl status fluentd.service  

For Calyptia-Fluentd, replace fluentd.service with calyptia-fluentd.service.

You should see the following output:

launch-fluentd

Step 3: Post sample logs

The last step is to send a test log to verify the installation. You can do this using the command below:

curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test  
tail -n 1 /var/log/fluent/fluentd.log  

Your output should look similar to this:

2024-12-04 07:19:13.420429094 +0000 debug.test: {"json":"message"}

Install from source

This installation guide involves cloning the Fluentd GitHub repository, installing dependencies, and running Fluentd.

The following steps will guide you along the way:

Step 1: Download the source code

Get the latest source from the Fluentd GitHub repository:

git clone https://github.com/fluent/fluentd.git  
cd fluentd  

Step 2: Install dependencies

Ensure Ruby and required libraries are installed, then run the following command:

gem install bundler  
bundle install  

Step 3: Build and run Fluentd

bundle exec fluentd --setup ./fluent  
bundle exec fluentd -c ./fluent/fluent.conf  

Installation with Docker

Using Docker is a quick and isolated way to run Fluentd. This approach is suitable for all OS types as well.

The following steps will guide you in installing Fluentd using Docker:

Step 1: Pull the Fluentd image

The following command will pull the official Fluentd image from Docker Hub:

docker pull fluent/fluentd:v1.14-1  

You should get a similar output:

v1.14-1: Pulling from fluent/fluentd
6097bfa160c1: Pull complete
fbc3d36e35e4: Pull complete
26054f1c5f42: Pull complete
ea86cadf5e05: Pull complete
b05ff3fb7de5: Pull complete
Digest: sha256:3e0bd37d74c95cff04a5a70ca4bb7470c6f807075cb914c1267c89f5801e1333
Status: Downloaded newer image for fluent/fluentd:v1.14-1
docker.io/fluent/fluentd:v1.14-1

Step 2: Run Fluentd

Run Fluentd using the command below:

docker run -d -p 8888:8888 -v $(pwd)/fluent.conf:/fluentd/etc/fluent.conf fluent/fluentd:v1.14-1  

Your output will be your Docker container ID:

4c8367720d3b8a2f5881a98493b16dfb4d440bcc3e70f9619b0e9a8b0e31d3a7

Step 3: Test the setup

To post a test log to Fluentd, run the following command:

curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test  
docker logs <container_id>  

You should get a similar output like this:

Docker-fluentd-installation

Installation on Kubernetes

This guide will walk you through installing Fluentd as a DaemonSet in Kubernetes.

Prerequisites

Before installing Fluentd on Kubernetes, ensure you have the following:

  • Kubernetes cluster: A running Kubernetes cluster
  • kubectl: Kubernetes command-line tool installed and configured
  • Cluster access: Administrative access to the Kubernetes cluster
  • Log destination: (Optional) A configured log destination like Elasticsearch, CloudWatch, or another log aggregation service

Step 1: Create RBAC resources

Fluentd requires specific permissions to collect logs from your Kubernetes cluster.

Apply the following RBAC (Role-Based Access Control) configuration:

apiVersion: v1
kind: ServiceAccount
metadata:
 name: fluentd
 namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 name: fluentd
rules:
- apiGroups: [""]
 resources: ["pods", "nodes", "namespaces"]
 verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: fluentd
roleRef:
 kind: ClusterRole
 name: fluentd
 apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
 name: fluentd
 namespace: kube-system

Apply the configuration:

kubectl apply -f fluentd-rbac.yaml

Step 2: Create Fluentd ConfigMap

Configure Fluentd's log collection and forwarding behavior using a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
 name: fluentd-config
 namespace: kube-system
data:
 fluent.conf: |-
   <source>
     @type tail
     @id in_tail_container_logs
     path /var/log/containers/*.log
     pos_file /var/log/fluentd-containers.log.pos
     tag kubernetes.*
     read_from_head true
     <parse>
       @type json
       time_key time
       time_format %Y-%m-%dT%H:%M:%S.%NZ
     </parse>
   </source>

   <match **>
     @type stdout
   </match>

Apply the configuration:

kubectl apply -f fluentd-configmap.yaml

Step 3: Deploy Fluentd DaemonSet

Deploy Fluentd as a DaemonSet to ensure log collection on every node:

apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: fluentd
 namespace: kube-system
 labels:
   k8s-app: fluentd-logging
   version: v1
spec:
 selector:
   matchLabels:
     k8s-app: fluentd-logging
     version: v1
 template:
   metadata:
     labels:
       k8s-app: fluentd-logging
       version: v1
   spec:
     tolerations:
     - key: node-role.kubernetes.io/control-plane
       operator: Exists
       effect: NoSchedule
     - key: node-role.kubernetes.io/master
       operator: Exists
       effect: NoSchedule
     serviceAccount: fluentd
     serviceAccountName: fluentd
     containers:
     - name: fluentd
       image: fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch8-1
       imagePullPolicy: IfNotPresent
       env:
       - name: K8S_NODE_NAME
         valueFrom:
           fieldRef:
             fieldPath: spec.nodeName
       resources:
         limits:
           memory: 512Mi
           cpu: 500m
         requests:
           cpu: 100m
           memory: 200Mi
       volumeMounts:
       - name: varlog
         mountPath: /var/log
         readOnly: true
       - name: varlibdockercontainers
         mountPath: /var/lib/docker/containers
         readOnly: true
       - name: fluentd-config
         mountPath: /fluentd/etc/fluent.conf
         subPath: fluent.conf
       securityContext:
         privileged: true
     terminationGracePeriodSeconds: 30
     volumes:
     - name: varlog
       hostPath:
         path: /var/log
     - name: varlibdockercontainers
       hostPath:
         path: /var/lib/docker/containers
     - name: fluentd-config
       configMap:
         name: fluentd-config

Apply the DaemonSet:

kubectl apply -f fluentd-daemonset.yaml

Step 4: Verify installation

Check the status of Fluentd pods:

kubectl get pods -n kube-system | grep fluentd

You should see an output similar to this:

fluentd-rj8t2                            1/1     Running   0              2m9s

View Fluentd logs:

kubectl logs -n kube-system -l k8s-app=fluentd-logging

Final thoughts

In this guide, we’ve walked through installing Fluentd across multiple environments—from native Ubuntu setups using fluent-package or calyptia-fluentd, to Docker containers and Kubernetes clusters. Each method ensures Fluentd is ready to collect, process, and route logs efficiently, regardless of your infrastructure.

Installation is the critical first step in building a unified logging layer. With Fluentd now running, you’re positioned to centralize logs from applications, services, and infrastructure into a cohesive pipeline.

Like this article? Sign up for our newsletter below and become one of over 1000 subscribers who stay informed on the latest developments in the world of DevOps. Subscribe now!

Stay ahead with the latest updates, exclusive insights, and tailored solutions by joining our newsletter.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
What is your email addres?
Featured posts
Fluentd Architecture and Key Concepts
This is some text inside of a div block.
This is some text inside of a div block.

Stay ahead with the latest updates, exclusive insights, and tailored solutions by joining our newsletter.

We care about your data in our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
ABOUT THE AUTHOR