Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent

Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent - InfoQ

[BT](http://www.infoq.com/int/bt/ "bt")

InfoQ Software Architects' Newsletter

A monthly overview of things you need to know as an architect or aspiring architect.

View an example

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

Close

Live Webinar and Q&A: Portable by Design: Data Mobility & Recovery Patterns for Multi-Cloud Systems (May 21, 2026)Save Your Seat

Close

Toggle Navigation

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

English edition

• English edition
• Chinese edition
• Japanese edition
• French edition

[Write for InfoQ](http://www.infoq.com/write-for-infoq/ "Write for InfoQ")

Search

RegisterSign in

Unlock the full InfoQ experience

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.

Log In

or

Don't have an InfoQ account?

Register

• **Stay updated on topics and peers that matter to you**Receive instant alerts on the latest insights and trends.
• **Quickly access free resources for continuous learning**Minibooks, videos with transcripts, and training materials.
• **Save articles and read at anytime**Bookmark articles to read whenever youre ready.

Logo - Back to homepage

NewsArticlesPresentationsPodcastsGuides

Topics

[Development](http://www.infoq.com/development/ "Development")

• [Java](http://www.infoq.com/java/ "Java")
• [Kotlin](http://www.infoq.com/kotlin/ "Kotlin")
• [.Net](http://www.infoq.com/dotnet/ ".Net")
• [C#](http://www.infoq.com/c_sharp/ "C#")
• [Swift](http://www.infoq.com/swift/ "Swift")
• [Go](http://www.infoq.com/golang/ "Go")
• [Rust](http://www.infoq.com/rust/ "Rust")
• [JavaScript](http://www.infoq.com/javascript/ "JavaScript")

Featured in Development

• #### From VR to Flat Screens: Bridging the Input and Immersion Gap

Dany Lepage discusses the architectural journey of porting a hit VR title to seven non-VR platforms. He explains how his team solved the challenges of cross-progression, diverse input paradigms, and maintaining release velocity across Steam, iOS, and PlayStation. Beyond the tech, he shares candid lessons on the "product fit" gap when translating immersive social presence to 2D screens.


All in developmentFollow Topic

[Architecture & Design](http://www.infoq.com/architecture-design/ "Architecture & Design")

• [Architecture](http://www.infoq.com/architecture/ "Architecture")
• [Enterprise Architecture](http://www.infoq.com/enterprise-architecture/ "Enterprise Architecture")
• [Scalability/Performance](http://www.infoq.com/performance-scalability/ "Scalability/Performance")
• [Design](http://www.infoq.com/design/ "Design")
• [Case Studies](http://www.infoq.com/Case_Study/ "Case Studies")
• [Microservices](http://www.infoq.com/microservices/ "Microservices")
• [Service Mesh](http://www.infoq.com/servicemesh/ "Service Mesh")
• [Patterns](http://www.infoq.com/DesignPattern/ "Patterns")
• [Security](http://www.infoq.com/Security/ "Security")

Featured in Architecture & Design

• #### How to Build an Exchange: Sub Millisecond Response Times and 24/7 Uptimes in the Cloud

Frank Yu shares Coinbase’s engineering philosophy for building resilient, fair, and fast financial exchanges. He explains the power of a single-threaded architecture combined with the Raft consensus algorithm to maintain 24/7 availability. He discusses how determinism enables zero-downtime rolling deployments and the ability to replay production logs for perfect bug reproduction.


All in architecture-designFollow Topic

[AI Infrastructure](http://www.infoq.com/ai-ml-data-eng/ "AI Infrastructure")

• [Big Data](http://www.infoq.com/bigdata/ "Big Data")
• [Machine Learning](http://www.infoq.com/machinelearning/ "Machine Learning")
• [NoSQL](http://www.infoq.com/nosql/ "NoSQL")
• [Database](http://www.infoq.com/database/ "Database")
• [Data Analytics](http://www.infoq.com/data-analytics/ "Data Analytics")
• [Streaming](http://www.infoq.com/streaming/ "Streaming")

Featured in AI, ML & Data Engineering

• #### Dynamic Moments: Weaving LLMs into Deep Personalization at DoorDash

Sudeep Das and Pradeep Muthukrishnan explain the shift from static merchandising to dynamic, moment-aware personalization at DoorDash. They share how LLMs generate natural-language "consumer profiles" and content blueprints, while traditional deep learning handles last-mile ranking. This hybrid approach allows the platform to adapt to short-lived user intent and massive catalog abundance.


All in ai-ml-data-engFollow Topic

[Culture & Methods](http://www.infoq.com/culture-methods/ "Culture & Methods")

• [Agile](http://www.infoq.com/agile/ "Agile")
• [Diversity](http://www.infoq.com/diversity/ "Diversity")
• [Leadership](http://www.infoq.com/leadership/ "Leadership")
• [Lean/Kanban](http://www.infoq.com/lean/ "Lean/Kanban")
• [Personal Growth](http://www.infoq.com/personal-growth/ "Personal Growth")
• [Scrum](http://www.infoq.com/scrum/ "Scrum")
• [Sociocracy](http://www.infoq.com/sociocracy/ "Sociocracy")
• [Software Craftmanship](http://www.infoq.com/software_craftsmanship/ "Software Craftmanship")
• [Team Collaboration](http://www.infoq.com/team-collaboration/ "Team Collaboration")
• [Testing](http://www.infoq.com/testing/ "Testing")
• [UX](http://www.infoq.com/ux/ "UX")

Featured in Culture & Methods

• #### Panel: Building a Culture that Works

The panelists share insights on evolving company culture. They discuss leveraging feedback loops, lending social capital, and the friction between legacy bureaucracy and agile engineering. The panel explains how to maintain cohesion in remote teams and use interviews to uncover the true "unmanicured" culture of a firm.


All in culture-methodsFollow Topic

DevOps

• [Infrastructure](http://www.infoq.com/infrastructure/ "Infrastructure")
• [Continuous Delivery](http://www.infoq.com/continuous_delivery/ "Continuous Delivery")
• [Automation](http://www.infoq.com/automation/ "Automation")
• [Containers](http://www.infoq.com/containers/ "Containers")
• [Cloud](http://www.infoq.com/cloud-computing/ "Cloud")
• [Observability](http://www.infoq.com/observability/ "Observability")

Featured in DevOps

• #### Beyond One-Click: Designing an Enterprise-Grade Observability Extension for Docker

Docker Extensions boost developer speed but create a "visibility gap" by isolating telemetry. To meet enterprise needs, extensions must act as bridges to centralized platforms. This article details how to use OpenTelemetry, policy-as-code, and encryption to build secure pipelines. Learn to balance developer productivity with the governance required for scalable, compliant observability.


All in devopsFollow Topic

[Events](https://events.infoq.com/ "Events")

Helpful links

• [About InfoQ](http://www.infoq.com/about-infoq "About InfoQ")
• [InfoQ Editors](http://www.infoq.com/infoq-editors "InfoQ Editors")
• [Write for InfoQ](http://www.infoq.com/write-for-infoq "Write for InfoQ")
• [About C4Media](https://c4media.com/ "About C4Media")
• [Diversity](https://c4media.com/diversity "Diversity")

Choose your language

• [En](http://www.infoq.com/news/2026/04/grafana-loki-ai-agents/# "InfoQ English")
• 中文
• 日本
• Fr


[InfoQ Homepage](http://www.infoq.com/ "InfoQ Homepage")[News](http://www.infoq.com/news "News")Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent

[DevOps](http://www.infoq.com/Devops/ "DevOps")

QCon San Francisco (Nov 16-20): Deep technical sessions. Peer conversations that change how you think.

Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent

Apr 23, 2026 3 min read

by

• Claudio Masolo

Follow Senior DevOps Engineer

#### Write for InfoQ

**Feed your curiosity.**Help 550k+ global

senior developers

each month stay ahead.Get in touch

Log in to listen to this article

Loading audio

Your browser does not support the audio element.

0:00 0:00

Normal 1.25x 1.5x

Like

• Reading list

At GrafanaCON 2026 in Barcelona, Grafana Labs announced Grafana 13 with the new Loki Kafka-backed architecture at the ingestion layer and the AI Observability in Grafana Cloud to monitor and evaluate AI systems in real time. In particular, the new CLI called GCX was announced, designed to surface Grafana Cloud data inside agentic development environments.

The traditional Loki architecture achieves high availability through replication: every incoming log line is sent to three ingesters, giving you a replication factor of three. Straightforward enough on paper. The catch is that deduplication relies on file naming, if ingesters cover the same time range, they should produce identical file names and those duplicates collapse.

!Image 12/filters:no_upscale()/news/2026/04/grafana-loki-ai-agents/en/resources/1Loki%20architecture-1776936739145.jpg)Previous and current Loki architecture Trevor Whitney, Staff Software Engineer at Grafana Labs, explained the mechanics during a briefing at GrafanaCON:

In a distributed system, the ingesters drift a bit, and any amount of drift in the time syncing of the ingesters results in those files not getting deduped by file name. Our internal metrics show that in reality, we end up storing on average 2.3x, for every log line that we ingest, we store it 2.3 times.

That 2.3x multiplier isn't an abstraction. It shows up on every line item: CPU at ingestion, memory pressure, network costs, object storage bills, and then again at query time when duplicates have to be reconciled on the fly.

The new architecture replaces the replication-at-ingestion strategy with Kafka as the durability layer. Logs land in Kafka once, ingesters consume from the queue, and the effective replication factor drops to one. Combined with a redesigned query engine that distributes work across partitions and executes in parallel, Grafana claims up to 20x less data scanned and 10x faster performance on aggregated queries.

There's a trade-off worth naming. Loki's original design principle was minimal dependencies: object storage and nothing else. The new architecture breaks that. Whitney acknowledged it directly:

Up until now, our only dependency has been object storage, and that's kind of been a goal of the project from the beginning. So yes, this does introduce a second dependency. You will now have object storage and Kafka for any distributed installation of Loki.

Single-binary deployments won't be affected; a local setup or home lab has no replication to orchestrate, so it runs fine with just file system or object storage. But anyone running Loki at scale needs to factor Kafka into their operational surface.

During the GrafanaCON, a new agent-aware CLI for integrating observability into AI-driven workflows, GCX, was launched in public preview. The premise is simple: many engineers now spend most of their day inside Claude Code, Cursor, or GitHub Copilot, and when something breaks in production, the workflow forces a context switch: out to Grafana, through dashboards, back to the editor, and then back to Grafana to verify the fix worked. GCX is designed to collapse that loop.

Ward Bekker, who led the GCX work, described the reasons of a CLI tool during a live demo:

CLIs were never out of fashion, but they're definitely more in fashion now, especially because of the agentic coding tools. A lot of folks notice that if you're using CLIs on the command line in combination with Cursor or Claude Code, it's extremely effective.

Bekker walked through a representative scenario: a synthetic monitoring check detects failures on an e-commerce order flow; Grafana Assistant runs automated root cause analysis; GCX pulls that analysis into Claude Code alongside the relevant source files; Claude Code proposes and applies a fix; GCX then queries the synthetic monitoring metrics directly to confirm recovery. No browser tab required.

Grafana Labs is not betting on one integration model. The team is shipping GCX as a CLI while also developing a remote MCP server in parallel, on the view that both have different audiences and use cases worth supporting.

These announcements sit alongside Grafana 13, which ships dynamic dashboards as generally available, adds Git-based workflow support, and expands the data source ecosystem to over 170 integrations. Grafana Labs also launched an AI Observability product in public preview for teams monitoring LLM-powered applications in production.

Grafana 13 and the Loki updates are available now. GCX is in public preview. The AI Observability solution is also in public preview in Grafana Cloud.

About the Author


#### **Claudio Masolo**

Claudio is a Senior DevOps Engineer at Nearform. In his spare time, he likes running, reading, and playing old video games.

Show more Show less

#### This content is in the DevOps topic

Follow Topic

##### Related Topics:

• DevOps### DevOps

Followers: 5047

Follow Topic

• Apache Kafka### Apache Kafka

Followers: 305

Follow Topic

• Observability### Observability

Followers: 568

Follow Topic

• Grafana### Grafana

Followers: 12

Follow Topic

• AI Coding### AI Coding

Followers: 2

Follow Topic

* #### Popular in DevOps

• ##### AWS Announces General Availability of DevOps Agent for Automated Incident Investigation

• ##### AWS Introduces S3 Files, Bringing File System Access to S3 Buckets

• ##### GitHub Copilot CLI Reaches General Availability

• ##### Anthropic Introduces Managed Agents to Simplify AI Agent Deployment

• ##### CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads

* #### Related Sponsors

• ##### 8 Deployment Pattern Structures to Transform your CI/CD

• ##### Portable by Design: Data Mobility & Recovery Patterns for Multi-Cloud Systems (Live Webinar May 21, 2026) - Save Your Seat

• ##### Design for Failure: How to Guarantee Data Access During Cloud Outages

• ##### The Essential Guide to AI Tools for Jakarta EE Developers

• ##### AI-Native Software Delivery - Download the eBook (By O'Reilly)

• #### Related Sponsor

**Online InfoQ Certified Architect Program**

Bring a real architecture challenge and work through it with senior peers in a 5-week online cohort. **Reserve Your Place.**

Related Content

• ##### How Observability and Telemetry Can Enhance the Practice of Software Engineering

Apr 23, 2026

• ##### Kubernetes Autoscaling Demands New Observability Focus beyond Vendor Tooling

Mar 31, 2026

• ##### Beyond One-Click: Designing an Enterprise-Grade Observability Extension for Docker

Apr 14, 2026 

• ##### From Alert Fatigue to Agent-Assisted Intelligent Observability

Feb 04, 2026 

• Icon##### From Confusion to Clarity: Advanced Observability Strategies for Media Workflows at Netflix

Jan 02, 2026 

• Icon##### From Dashboard Soup to Observability Lasagna: Building Better Layers

Nov 19, 2025 

• Icon##### Scaling API Independence: Mocking, Contract Testing & Observability in Large Microservices Environments

Nov 05, 2025 

• ##### Anthropic Releases Claude Mythos Preview with Cybersecurity Capabilities but Withholds Public Access

Apr 13, 2026

• ##### GitHub Copilot CLI Reaches General Availability

Apr 12, 2026

Related Sponsors

• #### Advance your architecture career with the InfoQ Certified Architect Program—practical, peer-driven certification

Join Luca Mezzalira for a 5-week online cohort on socio-technical architecture leadership. Explore how team structure, communication, and decision-making shape architecture outcomes in real engineering organisations.

• Sponsored by


Related Content

• ##### Dynamic Languages Faster and Cheaper in 13-Language Claude Code Benchmark

Apr 06, 2026

• ##### The Spring Team on Spring Framework 7 and Spring Boot 4

Apr 13, 2026 

• ##### QCon London 2026: Tools That Enable the Next 1B Developers

Mar 25, 2026

• ##### Stateful Continuation for AI Agents: Why Transport Layers Now Matter

Apr 08, 2026 

• ##### OpenTelemetry Declarative Configuration Reaches Stability Milestone

Apr 15, 2026

• ##### Airbnb Migrates High-Volume Metrics Pipeline to OpenTelemetry

Apr 14, 2026

**The InfoQ** Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

• Development

• ##### [C++26: Reflection, Memory Safety, Contracts, and a New Async Model](http://www.infoq.com/news/2026/04/cpp-26-reflection-safety-async/ "C++26: Reflection, Memory Safety, Contracts, and a New Async Model")

• ##### [From VR to Flat Screens: Bridging the Input and Immersion Gap](http://www.infoq.com/presentations/game-vr-flat-screens/ "From VR to Flat Screens: Bridging the Input and Immersion Gap")

• ##### [Cursor 3 Introduces Agent-First Interface, Moving beyond the IDE Model](http://www.infoq.com/news/2026/04/cursor-3-agent-first-interface/ "Cursor 3 Introduces Agent-First Interface, Moving beyond the IDE Model")

• Architecture & Design

• ##### [How to Build an Exchange: Sub Millisecond Response Times and 24/7 Uptimes in the Cloud](http://www.infoq.com/presentations/exchange-systems-cloud/ "How to Build an Exchange: Sub Millisecond Response Times and 24/7 Uptimes in the Cloud")

• ##### [Dropbox Collaborates with GitHub to Reduce Monorepo Size from 87GB to 20GB](http://www.infoq.com/news/2026/04/dropbox-reduces-git-optimization/ "Dropbox Collaborates with GitHub to Reduce Monorepo Size from 87GB to 20GB")

• ##### [Cloudflare Outlines MCP Architecture as Enterprises Confront Security and Governance Risks](http://www.infoq.com/news/2026/04/cloudflare-mcp/ "Cloudflare Outlines MCP Architecture as Enterprises Confront Security and Governance Risks")

• Culture & Methods

• ##### [How Observability and Telemetry Can Enhance the Practice of Software Engineering](http://www.infoq.com/news/2026/04/observability-telemetry/ "How Observability and Telemetry Can Enhance the Practice of Software Engineering")

• ##### [Panel: Building a Culture that Works](http://www.infoq.com/presentations/panel-positive-culture/ "Panel: Building a Culture that Works")

• ##### [Platform as a Product: Delivering Value While Balancing Competing Priorities](http://www.infoq.com/news/2026/04/platform-product-deliver-value/ "Platform as a Product: Delivering Value While Balancing Competing Priorities")

• AI, ML & Data Engineering

• ##### [Dynamic Moments: Weaving LLMs into Deep Personalization at DoorDash](http://www.infoq.com/presentations/llm-personalization/ "Dynamic Moments: Weaving LLMs into Deep Personalization at DoorDash")

• ##### [Subagents in Gemini CLI Enable Task Delegation and Parallel Agent Workflows](http://www.infoq.com/news/2026/04/subagents-gemini-cli/ "Subagents in Gemini CLI Enable Task Delegation and Parallel Agent Workflows")

• ##### [Google’s Aletheia Advances the State of the Art of Fully Autonomous Agentic Math Research](http://www.infoq.com/news/2026/04/deepmind-aletheia-agentic-math/ "Google’s Aletheia Advances the State of the Art of Fully Autonomous Agentic Math Research")

• DevOps

• ##### [Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent](http://www.infoq.com/news/2026/04/grafana-loki-ai-agents/ "Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability Into Coding Agent")

• ##### [GitHub Acknowledges Recent Outages, Cites Scaling Challenges and Architectural Weaknesses](http://www.infoq.com/news/2026/04/github-outages-scaling/ "GitHub Acknowledges Recent Outages, Cites Scaling Challenges and Architectural Weaknesses")

• ##### [AWS Announces General Availability of DevOps Agent for Automated Incident Investigation](http://www.infoq.com/news/2026/04/aws-devops-agent-ga/ "AWS Announces General Availability of DevOps Agent for Automated Incident Investigation")

**The InfoQ** Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

• Get a quick overview of content published on a variety of innovator and early adopter technologies
• Learn what you don’t know that you don’t know
• Stay up to date with the latest information from the topics you are interested in

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

**May 7 | June 10, 2026 | Online** Architecture decisions are hard to validate while shipping. Join a **5-week online cohort** for **senior engineers, architects, and team leads** to pressure-test real decisions, apply practical frameworks, and work through challenges with a confidential peer group. Facilitated by Luca Mezzalira, Principal Architect at AWS, this cohort helps you: * Pressure-test real decisions. * Apply frameworks to real problems. * Publish on InfoQ.com and earn your certification. **RESERVE YOUR PLACE**

[Home](http://www.infoq.com/ "Home")[Create account](http://www.infoq.com/reginit.action "Create account")Log In[QCon Conferences](http://qconferences.com/ "QCon Conferences")Events[Write for InfoQ](http://www.infoq.com/write-for-infoq/ "Write for InfoQ")[InfoQ Editors](http://www.infoq.com/infoq-editors/ "InfoQ Editors")[About InfoQ](http://www.infoq.com/about-infoq/ "About InfoQ")[About C4Media](https://c4media.com/ "About C4Media")[Media Kit](https://get.infoq.com/infoq-mediakit/ "Media Kit")[InfoQ Developer Marketing Blog](https://devmarketing.c4media.com/?utm_source=infoq "InfoQ Developer Marketing Blog")[Diversity](https://c4media.com/diversity "Diversity")

#### Events

• ##### Online InfoQ Architect Certification

May 7, 2026

• ##### QCon AI Boston

June 1-2, 2026

• ##### Online InfoQ Architect Certification

June 10, 2026

• ##### QCon San Francisco

November 16-20, 2026

#### Follow us on

Youtube 232K FollowersLinkedin 26K FollowersRSS 19K ReadersX 57.1k FollowersFacebook 21K LikesBluesky NewInstagram New

#### Stay in the know

The InfoQ PodcastEngineering Culture PodcastThe Software Architects' Newsletter

General Feedback [feedback@infoq.com](mailto:feedback@infoq.com) Advertising [sales@infoq.com](mailto:sales@infoq.com) Editorial [editors@infoq.com](mailto:editors@infoq.com) Marketing [marketing@infoq.com](mailto:marketing@infoq.com)

InfoQ.com and all content copyright © 2006-2026 C4Media Inc.

Privacy Notice, Terms And Conditions, Cookie Policy

Close

[BT](http://www.infoq.com/int/bt/ "bt")