Overview

Overview

This page explains how to use Basalt’s Experiments API with Python. Experiments help you organize, track, and compare different approaches in your AI workflows.

Installation

Ensure you have the Basalt SDK installed:

pip install basalt-sdk

Client Initialization

Access the experiments API through the experiments property on the main Basalt client:

from basalt import Basalt

basalt = Basalt(api_key="your-api-key")

Create one Basalt instance per process and reuse it for the entire application lifetime. shutdown() permanently destroys the global OpenTelemetry TracerProvider — never call it between loop iterations or between requests. See Observability Concepts for details.

Creating Experiments

You can create experiments programmatically to register new tests or variants.

Synchronous Creation

experiment = basalt.experiments.create_sync(
    feature_slug="llm-routing",
    name="Model Selection Experiment"
)

print(f"Created experiment: {experiment.id}")

Asynchronous Creation

import asyncio

async def create_experiment():
    experiment = await basalt.experiments.create(
        feature_slug="llm-routing",
        name="Model Selection Experiment"
    )
    return experiment

asyncio.run(create_experiment())

Using Experiments with Observability

The most common use case is attaching an experiment context to your traces. This allows you to slice and dice your observability data by experiment.

from basalt.observability import start_observe, observe

@start_observe(
    feature_slug="ab-test",
    name="experiment.variant_a",
    experiment="exp-456", # The ID of the experiment
    identity={
        "organization": {"id": "123", "name": "ACME"},
        "user": {"id": "456", "name": "John Doe"}
    }
)
def run_variant_a():
    # Experiment metadata is automatically attached to the root span
    observe.set_input({"variant": "A", "model": "gpt-4o"})
    
    # ... your logic ...
    
    return "result"

Experiments in a Loop

When processing multiple items under one experiment, each start_observe() call naturally creates a separate trace — the context manager cleans up the OpenTelemetry context on exit, so the next iteration starts fresh.

experiment = basalt.experiments.create_sync(
    feature_slug="support-ticket",
    name="Ticket Analysis",
)

for ticket in tickets:
    with start_observe(
        name="analyse_ticket",
        feature_slug="support-ticket",
        experiment=experiment,
    ) as span:
        span.set_input({"ticket": ticket})
        result = process(ticket)
        observe.set_output({"result": result})

basalt.shutdown()  # Once, at the end

Do not call basalt.shutdown() inside the loop or recreate the Basalt client per iteration. shutdown() permanently kills the TracerProvider — all subsequent traces would be silently dropped.