Loki

Loki: A powerful open-source logging tool for centralized log collection, storage, and analysis that is frequently combined with Grafana for comprehensive monitoring.

INTRODUCTION

Loki is a robust logging tool and open-source software used for distributed and efficient log collection, storage, and querying. It is intended to handle massive amounts of log data created by diverse applications and systems, enabling centralized log management and simple troubleshooting in complex, microservices-based architectures. Loki is frequently combined with Grafana to provide a complete platform for log display, analysis, and monitoring.

One way to check logs is by Grafana Logs Dashboards however there is another way by Grafana UI. Click on the Explore Icon of left navigation on Grafana UI then select Loki as a Datasource.

Select Loki as datasource and then select the label which you wish to watch logs for.

COMMON USES OF LOKI

• The "Line Contains" action in Loki is used to search for a specific and exact string within logs, allowing for targeted and specialized log retrieval and analysis.

  • example - Regex can be used to match certain strings inside transaction data, providing a versatile and powerful approach for searching for patterns, extracting useful information, and detecting anomalies within transactional logs or records. This aids in data analysis, validation, and pattern detection, allowing for more efficient processing and decision-making.

• Specifically, the "Line Does Not Contain" operation in Loki filters log lines based on the absence of a specific string, allowing you to eliminate log entries that do not include the requested text. This aids in refining the search and concentrating on important log data.

  • example - Earlier, too many generate challenge failures were expected errors. So, filtered them using Line does not contain operation.

• Users can view a list of previously ran searches by using the query history function. It allows you to easily access and rerun prior queries without having to retype them, making it easier to revisit and analyze earlier log data searches or inquiries. This feature improves process efficiency and makes historical log analysis easier.

• Use Loki with Grafana to perform real-time network debugging with configurable logging. Create and iterate Loki queries, monitor live logs, and refine searches to observe network behavior and fix problems quickly for improved performance and dependability.

• Utilize Loki's time range functionality in Grafana to inspect logs from previous runs. Define the time window that will be used to precisely analyze historical logs, assisting in debugging, issue resolution, and performance optimization for earlier system executions.

• To run custom script queries, use the "Download logs" feature included in the inspector. This enables personalized log analysis, providing precise and detailed information based on your individual needs.

• Loki queries can use regular expressions (regex) to do flexible and pattern-based log searches. Regex allows you to build complicated search patterns, allowing for versatile log filtering and extraction based on specific criteria, hence improving log analysis and troubleshooting skills.

  • example - This regex pattern will match the specific string "Not enough tokens to honor allocation" followed by a space, then one or more digits (represented by \d+), followed by a space, the less-than symbol <, and then one or more digits again.

    Not enough tokens to honor allocation \d+ < \d+ 

For more log queries refer Loki official doc - https://grafana.com/docs/loki/latest/query/log_queries/