Skip to main content
One of the most unique features of Nuabase is Row-Level Caching. Unlike standard HTTP caching or LLM-response caching which typically works on the entire request, Nuabase understands that your requests are often composed of multiple independent items.

​
How it works

When you use createArrayFn to process a list of items (e.g., rows in a database, log entries, user comments), Nuabase treats each item as a separate unit of work.
  1. You send a batch of 100 items.
  2. Nuabase checks its cache for each item individually, based on the prompt + schema + item content.
  3. If 90 items have been seen before (identical content), Nuabase serves them from the cache.
  4. Only the 10 new items are sent to the LLM.
  5. The results are merged and returned to you.

​
Benefits

​
1. Massive Cost Savings

You don’t pay for LLM tokens for data you’ve already processed. If you have a process that runs every hour on “all active users”, but only a few users have changed, you only pay for the changes.

​
2. Lower Latency

Cached hits are returned instantly (milliseconds) rather than waiting for the LLM (seconds).

​
3. Idempotency

You can safely retry failed jobs or re-run pipelines without worrying about double-spending on LLM costs.

​
Example Scenario

Run 1: You send 3 leads: A, B, C.
  • Nuabase processes A, B, C.
  • Cost: 3 items.
Run 2: You send 3 leads: A, B, D (where D is new).
  • Nuabase sees A and B are in cache.
  • Nuabase processes only D.
  • Cost: 1 item.