Rewriting Observability's DNA

Welcome to Edition #49 of the newsletter!

As the AI revolution unfolds, we are seeing some remarkable transformations. AI is not just a force multiplier for observability practice, it is also rewriting the DNA of observability itself. The gravitational force of agentic AI is driving a process of reimagining, with vendors reinventing themselves and redefining their products.

We see two examples of this trend in this edition of the newsletter. Incident.io’s rollout of Investigations brings powerful agentic SRE capabilities to their solution. Dash0 push the envelope further, positioning their product as an end-to-end platform for AI-driven development and application observability.

An Apology and a Thank You

You may have noticed that recently the newsletter has been appearing at rather irregular intervals. I would like to apologise for this. I won’t trouble you with the reasons, but it has been a bit of a difficult year. I would like to express my very sincere thanks for your patience.

Feedback

We love to hear your feedback. Let us know how we are doing at:

[email protected]

BlueSky

NEWS

Deductive AI snapped up by Elastic

With the huge boom in the AI SRE sector, it was inevitable that some kind of consolidation and shakeout would follow. One of the first major vendors to be swallowed up is Deductive, who recently entered into an agreement to be acquired by Elastic.

Whilst Elastic have long boasted strong capabilities in AI tooling, the Deductive acquisition brings powerful AI-driven investigation capabilities into their core observability product.

Perses ramps up again

Perses, the open source dashboarding project, continues to steam ahead. The latest version boasts a raft of new features, plugins and ergonomic tweaks.

The standout new feature is a generic log explorer. This is not generic in the sense of being a universal parser. Instead, you write a query in the native language of your backend and the Perses Log Explorer provides a consistent set of UI controls.

The release also includes a host of new plugins – including Jaeger, GreptimeDB, Splunk and OpenSearch. This version also introduces sub-folders, a much-requested feature to assist with housekeeping.

Finally, if you are already running Perses, this is one upgrade you should definitely install, as it patches a vulnerability in the permissions system.

Incident.io roll out Investigations AI

Over the last few years a number of full-stack vendors have extended their product reach to include concerns such as incident and on-call management. In a way, it makes sense to build these workflows into a single, integrated solution – even if it seems to be eating the lunch of specialist vendors.

However, this is a two-way street. Accurate root cause analysis is pretty much the philosopher’s stone of AI observability and everybody is after a piece of the action. In their latest release, Incident.io have introduced Investigations – an agentic AI RCA tool that is activated as soon as an incident is triggered.

The agent has a holistic view of your enterprise – reasoning across telemetry, code, deployments and previous incidents. This is a very slick piece of tooling and it creates a persuasive proposition – sophisticated incident management combined with AI insights.

Dash0 Darkplane – How AI is reshaping observability

We have previously remarked that one of the impacts of AI is that observability begins in the IDE – if you are using coding agents then you are incurring cost and raising governance issues as soon as the first line of code is written. This is the new normal that Dash0 are addressing with their Darkplane suite of tools.

Coding agents are great for productivity, but that very same productivity boost has created enormous backpressures on processes such as code review, whilst also resulting in serious spikes in runtime failures. Darkplane is designed to help teams tackle these issues head on, so that you can see metrics on code review rates, costs and actual real-world usage of the features you are shipping. Some of the features are GA and some are still in the pipeline.

Elastic – we got it wrong – columnar has won

We have seen some major turnarounds at Elastic over the past few years, but publicly announcing that their product architecture is built on the wrong foundation is pretty big. This is a story that goes to the heart of one of the major dividing lines in observability at the moment – the storage wars. Vendors such as ClickHouse and Honeycomb argue that their columnar db architectures afford them high cardinality and almost unlimited scalability.

The bottom line of the announcement of Columnar Mode in Elasticsearch 9.5 is a blunt admission that columnar dbs have won the day: “When the job is to read, aggregate, and reason over large volumes of data, you organize that data by column, not by row or document.”

This is a pretty momentous article that explains the rationale for the document db architecture chosen by Elastic 16 years ago and why it is no longer the right choice. Does this mean that Elastic is now a fully columnar database? Actually, no – columnar mode is an option – not a re-write. If you are interested in the fine detail, I also recommend checking out this article by Ignacio Van Droogenbroeck.

Products

IronBee - deep agentic AI observability

AI observability tooling is evolving rapidly, but IronBee is probably ahead of much of the field. A lot of tools will take an outside-in view and provide retrospective analytics on AI agent behaviour.

IronBee takes a radically different approach, hooking into the running agentic process via MCP. This gives it the ability to reach into every aspect of the agent’s activity. It can then insert itself into the agent’s verification loop in real time, generating a highly structured context payload that the LLM can reason about without exceeding its context window or spending excessive tokens.

The product integrates with source control and CI/CD systems and supports all major coding agents.

From the Blogosphere

Telemetry-driven development in Grafana

AI coding assistants may generate large volumes of code, but that also creates other problems. That code relies on a lot of assumptions, making autonomous decisions about coding patterns and tool usage. It is very hard for humans to review every line of code and every architectural decision without essentially becoming a bottleneck that reduces the velocity of the coding cycle back to human speed.

The thrust of this article by Dafydd Thomas is that “we need a better way to maintain our confidence in our changes.” The solution lies, naturally, in observability – ensuring that your code does exactly what you were expecting. This means building a harness that uses constraints such as SLOs and RED metrics to ensure that your code will not introduce regressions.

The article shows how you can use the Grafana gcx-observability skill to instrument your codebase. This is part of a debug and development cycle that enables the agent to analyse performance and then update code to meet desired performance metrics.

The web stats that matter in 2026

DORA Metrics, Core Web Vitals, Golden Signals – there is an array of measures that engineering teams depend on for evaluating performance. However, how relevant are these signals for 2026 and are there other new signals that engineering teams should be tracking? These are questions that Dentojn Chikura tackles in an article on the LogicMonitor blog. He argues that “web performance data is abundant. The harder problem is knowing which numbers to use, what they mean for the decisions you actually make”.

This is not just an academic exercise in gaming the stats – these metrics have an impact on the bottom line. Apparently, Amazon found that every 100ms of latency cost them 1% in sales – an amazing figure if true. The article crunches the numbers and looks at the business impact of metrics across a range of business functions. If you’re building web sites, this is a useful checklist for metrics you might want to incorporate into your analytics.

OpenTelemetry

OllyGarden Agent Skills - teaching your AI agent how to do oTel right

As we have often remarked, one of the great challenges for observability teams is getting things right at the point of instrumentation – because, no matter how good your observability platform, if your telemetry is bad you will not get the best outcomes.

No vendor on the market is doing more to plug the instrumentation gap than OllyGarden. They have now released Agent Skills which, as you might expect, is a package of skills files designed to help your agent apply high-quality OpenTelemetry instrumentation to your application code.

The package has been extensively dog-fooded by the OllyGarden team and consists of 14 skills grouped around three themes:
• Instrumenting services
• Operating the telemetry pipeline
• Keeping up to date

Highly recommended!

OpenTelemetry Demo 3.0 - now with added AI!

The OpenTelemetry Demo system has always been a great resource – whether you wanted to see OpenTelemetry in action or if you just wanted a great demo of a micro-services application. It provides examples of instrumentation and OpenTelemetry Collector configuration to provide a realistic simulation of real-world practice.

Version two already boasted features such as Kafka integration, a load generator, a Valkey cache and feature flagging.

So what is new in Version 3? Well, you won’t be surprised to learn that this release brings the demo into the AI era. If you have forked the repo then be aware there are breaking changes - such as renaming all custom attributes and re-factoring Docker configuration. However, they have built it back better with new services such as a LangGraph ReAct agent, an MCP server and a ChatBot UI.

AI

How Observability transforms vibe coding into AI engineering

How should observability systems go about evaluating AI-generated code before it reaches production? This is the subject matter of an article on the Dynatrace blog by Florian Mair. As Florian argues, AI agents can often generate “production” code that omits fundamental concerns such as error handling, rate limiting and observability.

Unfortunately, the default position for most agents is to ignore observability altogether. This is a useful reminder of some of the best practices that engineers can build into their coding practices to enable the best observability outcomes.

New Relic launch Preflight - open source AI observability tooling

It’s not often that big vendors give away cool stuff with no strings attached, but maybe this is one of those happy occasions. New Relic have released Preflight – an open source AI observability tool that you can run locally and view analytics without having to have any kind of New Relic account.

There are plenty of tools out there that can do the basics - like tracking token usage and costs. A major differentiator for Preflight is that it analyses usage to detect anti-patterns that lead to inefficiencies or lower quality output. For example, it can identify patterns such as Re-reads (continuously re-fetching files that already exist inside the context window) and Stuck Loops (circular decision chains where the agent gets trapped in a reasoning loop).

The overall aim is to help developers improve their AI coding practice. You can find Preflight on the New Relic GitHub repo.

Podcasts

Kyle Forster - the state of AI SRE

Kyle Forster is the CEO and founder of RunWhen and is also one of the most original and influential thinkers in the field of AI SRE.

In the latest edition of the Operationally Intelligent podcast, Kyle talks to SquaredUp's Adam Kinniburgh and shares insights on a number of critical themes:

• the critical importance of ultra-high accuracy

• the missing 70% of observability signals

• the future of the AI SRE role (there's bad news and good news)
  oh, and then there's the small matter of preventing a global economic meltdown.

Not to be missed.

That’s all for this edition!

If you have friends or colleagues who may be interested in subscribing to the newsletter, then please share this link!

This edition’s quote is from the British educationalist Ken Robinson

About Observability 360

Hi! I’m John Hayes. As well as publishing the Observability 360 newsletter, I am also an Observability Advocate at SquaredUp.

The Observability 360 newsletter is entirely autonomous. All opinions expressed in the newsletter are my own.