Skip to main content

​
Caching Policy

The Basalt SDK implements a two-level caching strategy to optimize performance and ensure high availability of your prompts.

​
Caching Architecture

Basalt employs a dual-layer caching system:
  1. API Cache: Our backend maintains a distributed cache to ensure high availability and reliability, even during service disruptions.
  2. SDK Local Cache: The SDK includes a built-in local cache that stores recently retrieved prompts to minimize network requests and improve response times.

​
Default Behavior

By default, the SDK’s local cache is enabled with a 5-minute duration. This means:
  • Repeated requests for the same prompt within a 5-minute window will be served from the local cache
  • After 5 minutes, the cache entry expires and a new request to the API will be made
  • The API may still serve a cached response if the prompt didn’t change in the meantime

​
Disabling Cache

You can disable the local SDK cache for specific requests when you need the most up-to-date version of a prompt: 
// Disable caching for a specific get request
const { value } = await basalt.prompt.get('welcome-message', {
  cache: false
})
The local cache is currently available only on the get method. On the method list and describe, the API cache is entirely managing the requests.

​
Best Practices

Here are some recommendations for effectively using the caching system:
  1. Production Environments: Keep caching enabled in production to maximize performance and reduce API calls.
  2. Development Environments: Consider disabling caching during development to ensure you’re always working with the latest prompt versions: 
// In development environments, globally disable caching
const basalt = new BasaltSDK({
  apiKey: 'your-api-key',
  cacheEnabled: process.env.NODE_ENV !== 'development'
})
  1. Time-Sensitive Operations: Disable caching for operations where having the absolute latest version is critical.
  2. Burst Operations: Keep caching enabled when making many requests in a short period to avoid rate limiting.

​
Global Cache Configuration

You can configure global caching behavior when initializing the SDK: 
// Disable caching globally
const basalt = new BasaltSDK({
  apiKey: 'your-api-key',
  cacheEnabled: false
})

// Or customize cache duration (in milliseconds)
const basalt = new BasaltSDK({
  apiKey: 'your-api-key',
  cacheDuration: 60000 // 1 minute
})

​
Cache Invalidation

The SDK automatically handles cache invalidation in the following scenarios:
  1. When the cache duration expires (default: 5 minutes)
  2. When explicitly requested with cache: false
  3. When using the API to update or modify prompts (the API invalidates relevant cache entries)
For manual cache control, you can use the cache management methods: 
// Clear the entire cache
await basalt.cache.clear()

// Clear cache for a specific prompt
await basalt.cache.invalidate('prompt', 'welcome-message')

// Clear cache for a prompt category
await basalt.cache.invalidate('feature', 'onboarding')

​
Benefits of Caching

Utilizing the SDK’s caching system provides several advantages:
  • Improved Performance: Reduced latency for frequently accessed prompts
  • Reduced API Costs: Fewer API calls means lower operational costs
  • Network Resilience: Ability to serve cached content during network disruptions
  • Reduced Load: Less strain on your application and our API servers

​
When Not to Use Caching

While caching is beneficial in most scenarios, consider disabling it in these cases:
  • During active prompt development and testing
  • For prompts that must always reflect the latest updates
  • In CI/CD pipelines where reproducibility is essential