Setting Up a Complete Monitoring Stack with Prometheus, Grafana, and Node Exporter on Kubernetes – Step-by-Step Guide

Monitoring your Kubernetes clusters is crucial to maintain reliability, detect issues early, and optimize performance. In this guide, you will learn how to set up a complete monitoring stack using Prometheus, Grafana, and Node Exporter on Kubernetes, step-by-step, with practical commands and explanations you can implement immediately.

Why Prometheus and Grafana?

• Prometheus: Powerful open-source monitoring and alerting toolkit designed for reliability and scalability.
• Grafana: Visualization and dashboarding platform to visualize your metrics from Prometheus clearly.
• Node Exporter: Collects Linux system metrics for Prometheus to scrape.

This stack enables you to monitor CPU, memory, disk usage, and Kubernetes metrics efficiently, making it ideal for DevOps workflows.

Prerequisites

• A Kubernetes cluster (Minikube, DigitalOcean Kubernetes, or any managed cluster)
• kubectl installed and configured
• Helm installed on your local system

Step 1: Add Prometheus Community Helm Repository

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

Step 2: Install Prometheus using Helm

We will install Prometheus and configure it to scrape metrics from the Node Exporter.

helm install prometheus prometheus-community/prometheus \
  --namespace monitoring --create-namespace

Verify that Prometheus is running:

kubectl get pods -n monitoring

To access Prometheus UI:

kubectl port-forward svc/prometheus-server -n monitoring 9090:80

Visit http://localhost:9090 in your browser.

Step 3: Install Node Exporter

Node Exporter is already included within the Prometheus Helm chart, but you can deploy it separately if needed for advanced configurations:

helm install node-exporter prometheus-community/prometheus-node-exporter \
  --namespace monitoring

Step 4: Install Grafana using Helm

Now, install Grafana to visualize your Prometheus metrics:

helm install grafana grafana/grafana \
  --namespace monitoring

Get the Grafana admin password:

kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Port forward Grafana:

kubectl port-forward svc/grafana -n monitoring 3000:80

Access Grafana at http://localhost:3000 using username admin and the retrieved password.

Step 5: Connect Prometheus as a Data Source in Grafana

1. Navigate to Configuration > Data Sources in Grafana.
2. Click Add data source and select Prometheus.
3. Set the URL to http://prometheus-server.monitoring.svc.cluster.local if within cluster, or use http://localhost:9090 if testing locally.
4. Click Save & Test.

Step 6: Import Dashboards

You can import community dashboards or create your own. For example:

1. Go to Dashboards > Import in Grafana.
2. Use dashboard ID 1860 for Node Exporter Full Dashboard.
3. Select your Prometheus data source and import.

You will now see CPU, memory, disk, and network metrics visualized for your Kubernetes nodes.

Step 7: Setting Up Alerts (Optional)

Prometheus and Grafana allow you to configure alerts to notify your team via Slack, Email, or PagerDuty when metrics breach defined thresholds. Consider adding alerts for:

• CPU usage > 85%
• Memory usage > 80%
• Node down for > 5 minutes

These alerts will help you proactively manage cluster reliability.

Conclusion

You have successfully set up a complete monitoring stack with Prometheus, Grafana, and Node Exporter on your Kubernetes cluster. This stack will provide you with real-time insights into your infrastructure, improve your DevOps practices, and help in detecting and resolving performance issues early.

Next Steps

• Automate your monitoring stack deployment using Helmfiles or GitOps pipelines.
• Add Alertmanager for advanced alert routing.
• Explore Loki for Kubernetes log aggregation and correlation with metrics.