Audit evidence from Traceloop (OpenLLMetry) traces
The Traceloop SDK, OpenLLMetry, traces an agent with OpenTelemetry and marks tool spans as tools. Add one span processor and each of those tool calls also becomes a signed decision and outcome record on your machine.
Traceloop keeps receiving the same spans: the processor reads them inside your process and exports nothing. How it works, the verify output and sampling: Audit log from OpenTelemetry traces.
Python #
Add the package to the project that has traceloop-sdk, then one line after Traceloop.init.
pip install "signafaide[otel]"from opentelemetry import trace
from signafaide.otel import SignafaideSpanProcessor
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import tool, workflow
Traceloop.init(app_name="support-bot") # Traceloop's own line
trace.get_tracer_provider().add_span_processor(
SignafaideSpanProcessor()
) # the one line
@tool(name="refund")
def refund(order: str, amount: float) -> str:
"""Refund an order."""
return f"refunded {amount} on {order}"
@workflow(name="support")
def support():
return refund("A1", 5)
support()A LangChain agent needs no decorator: the SDK's LangChain instrumentation writes the tool spans, and the same line records them.
TypeScript #
initialize takes one more processor beside Traceloop's own.
npm install signafaideimport * as traceloop from "@traceloop/node-server-sdk";
import { SignafaideSpanProcessor } from "signafaide/otel";
traceloop.initialize({
appName: "support-bot", // Traceloop's own line
processor: new SignafaideSpanProcessor({ agent: "support-bot" }), // the one line
});
const refund = (order, amount) =>
traceloop.withTool(
{ name: "refund" },
async (order, amount) => `refunded ${amount} on ${order}`,
order,
amount,
);
await traceloop.withWorkflow({ name: "support" }, () => refund("A1", 5));The samples run with traceloop-sdk 0.63 and @traceloop/node-server-sdk 0.27.
Before the tool ran, or after #
After, in both samples and for a LangChain agent. The SDK starts a tool span and then sets its attributes, so the decision and the outcome are both written when the span ends, after the tool ran. signafaide verify says so:
telemetry: _pending/nokey/support-bot: 1 tool is recorded from telemetry spans: refund (decision written after the tool ran)
telemetry: _pending/nokey/support-bot: telemetry sampling follows the caller: a tool call inside a trace that arrives marked as not sampled is not recordedFor a decision written before the tool runs, wrap the tool with the SDK. The call is then recorded once, by the wrapper: Quickstart.
What is recorded and what is not #
- Recorded. Each tool call whose span says
traceloop.span.kindistool, orgen_ai.operation.nameisexecute_tool: the tool name and the arguments the span carries, then the result or the error. - Not recorded. A tool call with no span, and a span the sampler dropped: Sampling.
- Protecting a tool needs the SDK. The processor never stops a tool. To run a tool only once its record is confirmed, wrap it with the SDK.
- Checks stay Calibrating. With a checks file in place, the processor records the tool-call checks of each tool call it reads, always Calibrating: it notes what they find and never blocks, and it never asks the witness for a ruling.
- Evidence, not certification. Records prove integrity and timing, not that a decision was right. The co-signature and the public timestamp come after
signafaide init, which creates a free sandbox and sends only salted hashes.
Next: start free, with the SDK alone and no account. Quickstart