jPOS Metrics Explorer: logs and AI in the loop

Most jPOS-based applications are monitored using Elasticsearch, Kibana, and Grafana, or commercial alternatives such as Datadog, Splunk, and New Relic—and I never liked that.

External monitoring solutions usually rely on a Java agent that gives a remote server access to the JVM. These applications are often PCI certified because many QSAs don't fully understand what a JVM is or how powerful a javaagent can be. Otherwise, they would probably be considered uncertifiable or, at the very least, could extend the scope of your CDE to the remote provider. That's one of the reasons I wrote SensitiveStrings: to keep in-JVM sensitive data encrypted most of the time, adding a little defense-in-depth and flying under the radar of scripts looking for sensitive card data.

Elasticsearch is an awesome tool, but it's overkill to dump all your payload into it, such as verbose jPOS logs. We use it together with Debezium to store pointers to transaction data, not the transaction data itself, which would otherwise just replicate primary storage. Kibana is excellent for monitoring indexed business data, but log data is inherently unstructured. It evolves over time, and the queries evolve with it.

Grafana is also a great product, but dashboards are typically designed once, tweaked during development, and then left untouched for years. Eventually, a new DevOps team member inherits them without really knowing how they were built or how to modify them.

Those concerns led me to integrate metrics directly into jPOS using Micrometer, producing native Prometheus and OpenTelemetry metrics so we can eliminate remote Java agents altogether. I also worked on Structured Logging so logs are precise enough that you don't have to rely on regular expressions to search for information, and instrumented jPOS with Java Flight Recorder for the situations where we need to perform deep JVM forensics.

All those pieces are finally coming together in the integrated Metrics Explorer and Log Viewer. When you're investigating a problem, you can click on a graph and jump directly to the corresponding structured log messages with a single click. Instead of being limited to predefined dashboards, you have the full power of PromQL at your fingertips.

And then comes the final piece: integrated AI.

You can simply ask, "Please check if we have GC pressure over the last six hours," click the jPOS AI icon, and immediately have that free-text request translated into PromQL. No need to know the metric names, labels, or query syntax—the AI does that for you.

The goal isn't to replace Prometheus, Grafana, Elasticsearch, or Kibana. They remain fantastic tools. The goal is to make jPOS itself understand its own runtime well enough that the most common operational and forensic tasks can be performed from a single, integrated environment, without shipping logs and JVM internals to external systems by default.

Metrics are excellent at telling you that something changed. They are less good at explaining why.

That gap usually sends an operator across several tools: a dashboard for the symptom, a query editor to narrow it down, a log system for the events around it, and perhaps documentation to reconstruct the right query. The latest jPOS Control Plane demo brings those steps together without making the result opaque.

The Metrics Explorer works with the Prometheus metrics exposed by a running jPOS application. It lets an operator move from a chart to the relevant logs, ask questions in plain language, and inspect or run the generated PromQL before relying on it.

What the demo shows

The video follows an operator through the Metrics area of the jPOS Control Plane:

• live Prometheus charts for transaction throughput, participant latency, and JVM memory
• a three-dimensional transaction-manager pipeline view, plus database and Kubernetes cluster views
• the Metrics Explorer, with PromQL completion based on metrics the deployment actually exposes
• chart zooming, panning, time ranges, and chart types suited to the query result
• a jump from a point on a chart to the Log Viewer for the five seconds on either side of that instant
• a conversational metrics question that returns a link to the exact Explorer query and time window
• Explain with AI, which reads the chart's summarized result and points out trends and follow-up checks
• natural-language PromQL generation, where the expression remains visible and editable before it is run

The important part is the connection between those features. A chart is not a dead end, an AI answer is not a black box, and a generated query is not executed somewhere the operator cannot see.

From a signal to its evidence

An alert or a chart normally begins an investigation rather than ends one. If a heap graph rises, a latency histogram changes shape, or throughput drops, the next question is usually: what was the application doing at that moment?

The Explorer makes that transition direct. Selecting a point on the chart opens the Log Viewer around the corresponding timestamp, with a ten-second window centered on the selected instant. Metrics establish the timing; logs provide the events that help explain it.

That is a small interaction with a useful consequence: the operator does not have to copy timestamps between systems or approximate the relevant window by eye. The investigation stays in context from signal to evidence.

AI that leaves a trail back to the chart

The assistant can answer a question about the metrics store in ordinary language, but its answer includes a permalink to the query and time range behind it. An operator can open that link, inspect the data, adjust the window, or take the query further.

The same principle applies in the other direction. With a chart on screen, Explain with AI provides the model a compact summary of the plotted data rather than an unbounded stream of raw samples. It can describe a trend, flag an anomaly, and suggest what to check next while the underlying chart remains in front of the operator.

For operators who know the question but not the exact PromQL, the Explorer can also turn a plain-language request into a query. It is grounded in the metric names that deployment exposes, so it is not guessing at a generic monitoring stack. The generated expression lands in the query box for review, learning, and adjustment before the operator runs it.

AI is useful here because it reduces the distance between a question and an investigation. It does not replace the chart, the query, or the logs.

One operational model

Metrics, logs, and AI are part of the same Control Plane session and permission model. The AI features are available only to an operator who can already reach the relevant metrics surface; they do not create a separate, more privileged path to production data.

This matters in payment operations, where a convenient assistant that bypasses ordinary boundaries would be a liability. The useful shape is more disciplined:

• Prometheus remains the metric source
• the Explorer exposes the actual query and its result
• the Log Viewer holds the timestamped operational evidence
• AI helps formulate and interpret questions within the user's existing permissions

The result is a workflow that is faster without becoming less verifiable. The AI can do the remembering and the first-pass reading; the operator keeps the judgement and the controls.