Getting Started with Grafana Loki

Let’s learn the fundamentals of Grafana Loki, a multi-tenant log aggregation system.

In a production environment, downtime is unacceptable. It can lead to a massive loss in business revenue and reputation. But if you have a proper logging and monitoring system in place, you will know how to debug such scenarios, which lead to downtime.

It helps in saving costs and identifying issues that might occur in the future.

Nowadays, every size of an organization is using DevOps principles and tools. Containers and Kubernetes are the most popular ones. And monitoring of such a system is done very efficiently with Prometheus. But where Prometheus lags is the logging part. It doesn’t provide a centralized logging system, and this is where Loki comes into the picture.

What is Grafana Loki?

Grafana Loki is a multi-tenant log aggregation system started by Grafana in 2018 and released under the Apache 2.0 license. This system was developed by getting inspired by Prometheus. It is majorly used with cloud providers and with tools like Prometheus and Grafana.

Loki is similar to ELK/EFK stack, but it is easier to set up and operate with better functionalities. Loki does not index the content of the log, rather it indexes timestamps and a set of labels for a log stream. It makes the index smaller, which simplifies the operations and eventually lowers the cost.

Loki Benefits

Below are the benefits of using Loki in your stack:

Loki Architecture

loki architecture - geekflare

The Loki architecture has three components – Promtail, Loki, and Grafana.

Promtail is an agent that has to be installed on each node running your applications or services. The primary responsibility of Promtail is to discover the target, attach the labels to the lo streams coming from the pods and push those logs to the Loki instances. The agent promtail tails the logs from the local file system and pushes the logs to Loki’s central server. After this, you can query back your logs using Grafana.

Loki Use Cases

Below are the popular use cases for using a logging system like Loki.

Installing Loki and Promtail

Let’s quickly find out how to install and visualize logs on Grafana. In this demonstration, I will use the generic configuration which will scrape the logs from /var/log/*log

Go to the Loki releases page, scroll down to Assets, here you will find multiple Loki and Promtail packages. Download the Loki package according to the system you are using. Do not download cli or canary Loki packages. I am downloading and for my Ubuntu system.

After downloading finishes, extract the files of Loki and Promtail, and put them in a single directory.

[email protected]:~$ mkdir loki [email protected]:~$ mv loki-linux-amd64 promtail-linux-amd64 loki [email protected]:~$ cd loki/ [email protected]:~/loki$ ls loki-linux-amd64 promtail-linux-amd64

Now, download the generic configuration file of Loki and Promtail.

[email protected]:~/loki$ wget --2021-04-30 12:00:51-- Resolving (,,, ... Connecting to (||:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1832 (1.8K) [text/plain] Saving to: ‘loki-local-config.yaml’  loki-local-config.yaml 100%[===============================================================================================>] 1.79K --.-KB/s in 0s  2021-04-30 12:00:51 (16.6 MB/s) - ‘loki-local-config.yaml’ saved [1832/1832]
[email protected]:~/loki$ wget --2021-04-30 12:05:07-- Resolving ( Connecting to (||:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘promtail-local-config.yaml’  promtail-local-config.yaml                        [ <=>                                                                                            ] 109.16K  --.-KB/s    in 0.002s    2021-04-30 12:05:08 (66.8 MB/s) - ‘promtail-local-config.yaml’ saved [111779]
[email protected]:~/loki$ ls loki-linux-amd64  loki-local-config.yaml  promtail-linux-amd64  promtail-local-config.yaml

To start Loki, run the command below with the Loki configuration file. This will start Loki and displays the Loki logs in the terminal.

[email protected]:~/loki$ ./loki-linux-amd64 -config.file=loki-local-config.yaml level=info ts=2021-04-30T16:08:37.990311491Z caller=main.go:130 msg="Starting Loki" version="(version=2.2.1, branch=HEAD, revision=babea82e)" level=info ts=2021-04-30T16:08:37.990856871Z caller=server.go:229 http=[::]:3100 grpc=[::]:9096 msg="server listening on addresses" level=info ts=2021-04-30T16:08:37.991613443Z caller=shipper_index_client.go:100 msg="starting boltdb shipper in 0 mode" level=info ts=2021-04-30T16:08:37.992568288Z caller=mapper.go:38 msg="cleaning up mapped rules directory" path=/tmp/loki/rules-temp level=error ts=2021-04-30T16:08:37.992623632Z caller=mapper.go:42 msg="unable to read rules directory" path=/tmp/loki/rules-temp err="open /tmp/loki/rules-temp: no such file or directory" level=info ts=2021-04-30T16:08:37.993520927Z caller=table_manager.go:171 msg="uploading tables" level=info ts=2021-04-30T16:08:37.993941099Z caller=module_service.go:59 msg=initialising module=store level=info ts=2021-04-30T16:08:37.994011839Z caller=module_service.go:59 msg=initialising module=memberlist-kv level=info ts=2021-04-30T16:08:37.99412775Z caller=module_service.go:59 msg=initialising module=server level=info ts=2021-04-30T16:08:37.994379648Z caller=module_service.go:59 msg=initialising module=ingester level=info ts=2021-04-30T16:08:37.994415984Z caller=ingester.go:241 msg="recovering from checkpoint"

To start Promtail, run the command below with Promtail configuration file. Promtail is required to get the logs into Loki.

[email protected]:~/loki$ ./promtail-linux-amd64 -config.file=promtail-local-config.yaml level=info ts=2021-04-30T16:35:52.775998059Z caller=server.go:229 http=[::]:9080 grpc=[::]:42557 msg="server listening on addresses" level=info ts=2021-04-30T16:35:52.776283661Z caller=main.go:112 msg="Starting Promtail" version="(version=2.2.1, branch=HEAD, revision=babea82e)" level=info ts=2021-04-30T16:35:57.776226323Z caller=filetargetmanager.go:254 msg="Adding target" key="{job=\"varlogs\"}" ts=2021-04-30T16:35:57.776902427Z caller=log.go:124 level=info msg="Seeked /var/log/fontconfig.log - &{Offset:11055 Whence:0}" level=info ts=2021-04-30T16:35:57.776983546Z caller=tailer.go:125 component=tailer msg="tail routine: started" path=/var/log/auth.log level=info ts=2021-04-30T16:35:57.777009023Z caller=tailer.go:125 component=tailer msg="tail routine: started" path=/var/log/fontconfig.log ts=2021-04-30T16:35:57.777094698Z caller=log.go:124 level=info msg="Seeked /var/log/lastlog - &{Offset:0 Whence:0}" ts=2021-04-30T16:35:57.777712594Z caller=log.go:124 level=info msg="Seeked /var/log/apport.log - &{Offset:0 Whence:0}" ts=2021-04-30T16:35:57.778200383Z caller=log.go:124 level=info msg="Seeked /var/log/alternatives.log - &{Offset:456 Whence:0}" ts=2021-04-30T16:35:57.780467419Z caller=log.go:124 level=info msg="Seeked /var/log/syslog - &{Offset:56911 Whence:0}" level=info ts=2021-04-30T16:35:57.780646032Z caller=tailer.go:125 component=tailer msg="tail routine: started" path=/var/log/kern.log level=info ts=2021-04-30T16:35:57.781289004Z caller=filetargetmanager.go:254 msg="Adding target" key="{job=\"varlogs\"}" level=info ts=2021-04-30T16:35:57.787770104Z caller=tailer.go:125 component=tailer msg="tail routine: started" path=/var/log/grafana/grafana.log ts=2021-04-30T16:35:57.782276009Z caller=log.go:124 level=info msg="Seeked /var/log/grafana/grafana.log - &{Offset:0 Whence:0}"

Visualize Log using Loki and Grafana

Grafana provides built-in support for Loki. Loki is already present in the data sources of Grafana.

Step 1: Go to Grafana Configurations and Click on “Data Sources”.


Step 2: In Data Sources, you can search the source by name or type.


Step 3: Search by the name Loki. You can see this data source is already present in Grafana. Click on Select.

add loki

Step 4: Enter the name you want to give to the data source and put http://localhost:3100 (change this to server IP if Loki is running on a different server than Grafana) in the URL because we have started Loki on port 3100.

loki data source

Click on the Test and Save button at the bottom. If you have done the Loki set up correctly, you will get the below message shown in a green box.

save and test

Step 5: Click on the Explore tab on the left side. Select Loki from the dropdown of data source selection. Now, I would like to visualize the activity of Grafana logs. To do this, you need to add the query {filename=”/var/log/grafana/grafana.log”} in the Log Browser. The green bars below are entries of events in the log file.

grafana log

You can select the time range for which you want the visualization to come on the dashboard, and you can also set the refresh interval of the query, after which the query will run again. To see more details of the logs, scroll down and click on one of the log entries, it will give all kinds of information related to the log.

detail log

Looks beautiful, isn’t it?

So, that was all about getting started. I would strongly advise you to further check out the below webinar video explaining observability with Loki.


A distributed system is composed of many applications or microservices, each one having tons of logs. You would need a cost-effective way to collect the logs, store them and then make use of them. Loki is a perfect solution for such cases. As a point of fact, you can reduce around 75% of your logging and monitoring costs by integrating Loki into your production environment.

Loki is also available through Grafana Cloud, it would be good to try it out if you don’t want to go through the installation process.

Next, get to know Grafana Tempo.