Airflow logs are stored in the filesystem by default in $AIRFLOW_HOME/dags directory, this is also called remote logging. Airflow logs[…]
Read moreLatest Posts
Monitoring Kafka with Prometheus
Introduction Monitoring Kafka using Prometheus is easy. In this blog entry, we will see how we can monitor the state[…]
Read moreMonitoring Cassandra with Prometheus
Introduction Monitoring Cassandra with Prometheus can be done in various ways. Till now we have used pre-built exporters for Linux[…]
Read moreMonitoring Docker with Prometheus
Prometheus makes it very easy to monitor Docker containers. It is done in a completely non-intrusive way. In this blog[…]
Read moreMonitoring Linux with Prometheus
In our previous post, we saw how to set up a simple Prometheus server. In this post, we extend it[…]
Read moreGrafana dashboards – getting started!
Collection of metrics data is of limited use if you cannot analyse it. One of the ways to analyse data[…]
Read morePrometheus monitoring – getting started
This is the start of the mini-series on monitoring infrastructure using Prometheus. The aim is to give an insight into[…]
Read moreAirflow RBAC – Role-Based Access Control
Airflow version 1.10.0 onward introduced Role-Based Access Control(RBAC) as part of their security landscape. RBAC is the quickest way to[…]
Read moreAirflow – XCOM
Introduction Airflow XCom is used for inter-task communications. Sounds a bit complex but it is really very simple. Its implementation[…]
Read moreRedis, Docker & Raspberry PI
Happy New Year!. Time for the first post of 2020. Have been wanting to write a post on raspberry pi[…]
Read more