Model Usage & Cost Tracking

Langfuse tracks the usage and cost of every LLM call in your application, so you can monitor spend across models, use cases, and over time.

For most users, cost tracking works out of the box. Langfuse ships with prices for popular models from OpenAI, Anthropic, and Google, and most integrations capture usage and cost automatically, so in most cases you don't need to configure anything.

What you can do with cost data

Once usage and cost are tracked, you can put the data to work:

• Create dashboards: monitor cost across models, tags, or users.
• Set up alerts: get notified automatically when spend crosses a threshold.
• Query with the Metrics API: retrieve aggregated usage and cost, filtered by application type, user, or tags, for analytics, billing, and rate-limiting.

How cost tracking works

Langfuse tracks the usage and costs of your LLM generations and provides breakdowns by usage types. Usage and cost can be tracked on observations of type generation and embedding.

• Usage details: number of units consumed per usage type
• Cost details: USD cost per usage type

Usage types can be arbitrary strings and differ by LLM provider. At the highest level, they can be simply input and output. As LLMs grow more sophisticated, additional usage types are necessary, such as cached_tokens, audio_tokens, image_tokens.

In the UI, Langfuse summarizes all usage types that include the string input as input usage types, similarly output as output usage types. If no total usage type is ingested, Langfuse sums up all usage type units to a total. For this to work, usage types must be mutually exclusive buckets.

Both usage details and cost details can be either

• ingested via API, SDKs or integrations
• or inferred based on the model parameter of the generation. Langfuse comes with a list of predefined popular models and their tokenizers including OpenAI, Anthropic, and Google models. You can also add your own custom model definitions or request official support for new models via GitHub. Inferred cost are calculated at the time of ingestion with the model and price information available at that point in time.

Ingested usage and cost are prioritized over inferred usage and cost:

Model definitions and prices

When cost is not ingested directly, Langfuse infers it. The model parameter of the generation is matched to a model definition, which stores a price per usage type. Langfuse then multiplies those prices by the observation's usage to calculate cost. This is especially useful for model providers or self-hosted models that do not include cost in the response.

Langfuse comes with a list of predefined popular models and their tokenizers including OpenAI, Anthropic, Google. Check out the full list (you need to sign-in).

You can manage prices in Project Settings > Models, add your own custom model definitions, or request official support for new models via GitHub.

Usage

If a tokenizer is specified for the model, Langfuse automatically calculates token amounts for ingested generations.

The following tokenizers are currently supported:

Cost

Model definitions include prices per usage type. Usage types must match exactly with the keys in the usage_details object of the generation.

Langfuse automatically calculates cost for ingested generations at the time of ingestion if (1) usage is ingested or inferred, (2) and a matching model definition includes prices.

How Langfuse maintains default model prices

Langfuse maintains its default model definitions in the open-source repository. A daily automated audit checks official provider sources for changed prices, pricing tiers, usage-key mappings, and newly released major models.

The audit follows an evidence-first workflow:

1. It verifies model IDs, prices, units, usage keys, and pricing tiers against official provider documentation.
2. It makes focused changes only when the provider evidence can be represented safely in Langfuse. Uncertain findings are reported without changing the defaults.
3. It validates the pricing data and any changed model-matching patterns.
4. When changes are needed, a pull request is opened, and the updated prices are deployed shortly after approval.

This process keeps the default definitions current without relying on a third-party pricing aggregator. Because inferred costs are calculated at ingestion time, updated defaults apply only to new generations. If you need support for a model immediately or use private pricing, add a custom model definition or ingest the cost directly.

Pricing Tiers

Some model providers charge different rates depending on context length or request attributes. For example, Anthropic's Claude Sonnet 4.5 and Google's Gemini 2.5 Pro apply higher pricing when more than 200K input tokens are used, while OpenAI Fast mode charges a premium based on the request's service_tier.

Langfuse supports pricing tiers for models, enabling accurate cost calculation across usage thresholds, model parameters, and metadata.

How tier matching works

Each model can have multiple pricing tiers, each with:

• Name: A descriptive name (e.g., "Standard", "Large Context")
• Priority: Evaluation order (0 is reserved for default tier)
• Conditions: Rules that determine when the tier applies
• Prices: Cost per usage type for this tier

When calculating cost, Langfuse evaluates tiers in priority order (excluding the default tier). The first tier whose conditions are satisfied is used. If no conditional tier matches, the default tier is applied.

Each condition selects one of these sources:

• Usage details: Match usage-detail keys with a regex, sum their values, and compare the result with gt, gte, lt, lte, eq, or neq. You can also control whether the key pattern is case-sensitive.
• Model parameters: Match a top-level model-parameter key against one or more exact values.
• Metadata: Match a top-level metadata key against one or more exact values.

For model parameters and metadata, select the source in the pricing-tier editor, enter the Top-level key, and add comma-separated exact values. String, number, and boolean values are compared as strings. Nested paths and non-primitive values are not supported. When a tier has multiple conditions, all of them must match.

For example, the "Large Context" tier for Claude Sonnet 4.5 has a condition: input > 200000, meaning it applies when the sum of all usage details matching the pattern "input" exceeds 200,000 tokens.

OpenAI Fast mode tiers match the top-level model parameter service_tier against fast or priority. For GPT-5.6 models, this condition is combined with the input-token threshold to distinguish Standard/Fast and short/long-context prices.

Custom model definitions

You can flexibly add your own model definitions (incl. pricing tiers) to Langfuse. This is especially useful for self-hosted or fine-tuned models which are not included in the list of Langfuse maintained models.

GET    /api/public/models
POST   /api/public/models
GET    /api/public/models/{id}
DELETE /api/public/models/{id}

Models are matched to generations based on:

User-defined models take priority over models maintained by Langfuse.

Further details

When using the openai tokenizer, you need to specify the following tokenization config. You can also copy the config from the list of predefined OpenAI models. See the OpenAI documentation for further details. tokensPerName and tokensPerMessage are required for chat models.

{
  "tokenizerModel": "gpt-3.5-turbo", // tiktoken model name
  "tokensPerName": -1, // OpenAI Chatmessage tokenization config
  "tokensPerMessage": 4 // OpenAI Chatmessage tokenization config
}

Cost inference for reasoning models

Cost inference by tokenizing the LLM input and output is not supported for reasoning models such as the OpenAI o1 model family. That is, if no token counts are ingested, Langfuse cannot infer cost for reasoning models.

Reasoning models take multiple steps to arrive at a response. The result from each step generates reasoning tokens that are billed as output tokens. So the cost-effective output token count is the sum of all reasoning tokens and the token count for the final completion. Since Langfuse does not have visibility into the reasoning tokens, it cannot infer the correct cost for generations that have no token usage provided.

To benefit from Langfuse cost tracking, please provide the token usage when ingesting o1 model generations. When utilizing the Langfuse OpenAI wrapper or integrations such as for Langchain, LlamaIndex or LiteLLM, token usage is collected and provided automatically for you.

For more details, see the OpenAI guide on how reasoning models work.

Ingest usage and cost manually

If available in the LLM response, ingesting usage and/or cost is the most accurate and robust way to track usage in Langfuse.

Many of the Langfuse integrations automatically capture usage details and cost details data from the LLM response. If this does not work as expected, please create an issue on GitHub.

from langfuse import observe, get_client
import anthropic

langfuse = get_client()
anthropic_client = anthropic.Anthropic()

@observe(as_type="generation")
def anthropic_completion(**kwargs):
  # optional, extract some fields from kwargs
  kwargs_clone = kwargs.copy()
  input = kwargs_clone.pop('messages', None)
  model = kwargs_clone.pop('model', None)
  langfuse.update_current_generation(
      input=input,
      model=model,
      metadata=kwargs_clone
  )

  response = anthropic_client.messages.create(**kwargs)

  langfuse.update_current_generation(
      usage_details={
          "input": response.usage.input_tokens,
          "output": response.usage.output_tokens,
          "cache_read_input_tokens": response.usage.cache_read_input_tokens
          # "total": int,  # if not set, it is derived as the sum of all usage types
        },
      # Optionally, also ingest usd cost. Alternatively, you can infer it via a model definition in Langfuse.
      cost_details={
          # Here we assume the input and output cost are 1 USD each and half the price for cached tokens.
          "input": 1,
          "cache_read_input_tokens": 0.5,
          "output": 1,
          # "total": float, # if not set, it is derived as the sum of all usage types
      }
  )

  # return result
  return response.content[0].text

@observe()
def main():
  return anthropic_completion(
      model="claude-3-opus-20240229",
      max_tokens=1024,
      messages=[
          {"role": "user", "content": "Hello, Claude"}
      ]
  )

main()

from langfuse import get_client
import anthropic

langfuse = get_client()
anthropic_client = anthropic.Anthropic()

with langfuse.start_as_current_observation(
    as_type="generation",
    name="anthropic-completion",
    model="claude-3-opus-20240229",
    input=[{"role": "user", "content": "Hello, Claude"}]
) as generation:
    response = anthropic_client.messages.create(
        model="claude-3-opus-20240229",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}]
    )

    generation.update(
        output=response.content[0].text,
        usage_details={
            "input": response.usage.input_tokens,
            "output": response.usage.output_tokens,
            "cache_read_input_tokens": response.usage.cache_read_input_tokens
            # "total": int,  # if not set, it is derived as the sum of all usage types
        },
        # Optionally, also ingest usd cost. Alternatively, you can infer it via a model definition in Langfuse.
        cost_details={
            # Here we assume the input and output cost are 1 USD each and half the price for cached tokens.
            "input": 1,
            "cache_read_input_tokens": 0.5,
            "output": 1,
            # "total": float, # if not set, it is derived as the sum of all usage types
        }
    )

import {
  startActiveObservation,
  startObservation,
  updateActiveObservation,
} from "@langfuse/tracing";

await startActiveObservation("context-manager", async (span) => {
  span.update({
    input: { query: "What is the capital of France?" },
  });

  // This generation will automatically be a child of "user-request"
  const generation = startObservation(
    "llm-call",
    {
      model: "gpt-4",
      input: [{ role: "user", content: "What is the capital of France?" }],
    },
    { asType: "generation" }
  );

  // ... LLM call logic ...

  generation.update({
    usageDetails: {
      input: 10,
      output: 5,
      cache_read_input_tokens: 2,
      some_other_token_count: 10,
      total: 27, // optional, it is derived as the sum of all usage types
    },
    costDetails: {
      // If you don't want the costs to be calculated based on model definitions, you can pass the costDetails manually.
      input: 1,
      output: 1,
      cache_read_input_tokens: 0.5,
      some_other_token_count: 1,
      total: 3.5,
    },
    output: { content: "The capital of France is Paris." },
  });

  generation.end();
});

import { observe, updateActiveObservation } from "@langfuse/tracing";

// An existing function
async function fetchData(source: string) {
  updateActiveObservation(
    {
      usageDetails: {
        input: 10,
        output: 5,
        cache_read_input_tokens: 2,
        some_other_token_count: 10,
        total: 27, // optional, it is derived as the sum of all usage types
      },
      costDetails: {
        // If you don't want the costs to be calculated based on model definitions, you can pass the costDetails manually.
        input: 1,
        output: 1,
        cache_read_input_tokens: 0.5,
        some_other_token_count: 1,
        total: 3.5,
      },
    },
    { asType: "generation" }
  );

  // ... logic to fetch data
  return { data: `some data from ${source}` };
}

// Wrap the function to trace it
const tracedFetchData = observe(fetchData, {
  name: "observe-wrapper",
  asType: "generation",
});

const result = await tracedFetchData("API");

const span = startObservation("manual-observation", {
  input: { query: "What is the capital of France?" },
});

const generation = span.startObservation(
  "llm-call",
  {
    model: "gpt-4",
    input: [{ role: "user", content: "What is the capital of France?" }],
    output: { content: "The capital of France is Paris." },
  },
  { asType: "generation" }
);

generation.update({
  usageDetails: {
    input: 10,
    output: 5,
    cache_read_input_tokens: 2,
    some_other_token_count: 10,
    total: 27, // optional, it is derived as the sum of all usage types
  },
  costDetails: {
    // If you don't want the costs to be calculated based on model definitions, you can pass the costDetails manually.
    input: 1,
    output: 1,
    cache_read_input_tokens: 0.5,
    some_other_token_count: 1,
    total: 3.5,
  },
});

generation
  .update({
    output: { content: "The capital of France is Paris." },
  })
  .end();

span.update({ output: "Successfully answered user request." }).end();

Usage types are mutually exclusive buckets

Langfuse treats every key in usage_details as a separate, non-overlapping bucket: each token must be counted in exactly one key. In particular, input must exclude tokens that are already counted in another input_* key (such as input_cached_tokens or input_cache_creation), and output must exclude tokens counted in another output_* key (such as output_reasoning_tokens). The one exception is total: it is not a bucket itself but spans all buckets and equals their sum.

Langfuse relies on this contract in three places:

• Display: the UI sums all usage types containing input to show total input usage, and all usage types containing output to show total output usage.
• Cost inference: each usage type is matched exactly against the per-usage-type prices of the model definition, and the resulting costs are summed.
• Total: if no total is ingested, Langfuse derives it as the sum of all buckets.

If buckets overlap — for example, if input still contains the tokens reported in input_cached_tokens — those tokens are displayed twice, and the inferred cost double-counts them: the cost shown in Langfuse overstates what your provider actually charged. Directly ingested cost_details are used verbatim and are not affected.

Many LLM providers report inclusive counts. For example, OpenAI's prompt_tokens (Chat Completions API) and input_tokens (Responses API) include cached tokens. Anthropic's input_tokens, in contrast, already excludes cache reads and cache writes. Inclusive counts must be converted into exclusive buckets before they are stored, by subtracting the detail counts from the top-level count.

For example, an OpenAI-style response with 17,903 prompt tokens (of which 17,817 were cache hits) and 188 completion tokens:

When Langfuse converts for you

Whether you need to do this conversion yourself depends on how the usage data reaches Langfuse:

• Langfuse integrations and SDK wrappers (e.g., the OpenAI wrapper) capture usage from the provider response and convert it for you.
• OpenTelemetry usage attributes (gen_ai.usage.* as defined by the OpenTelemetry GenAI semantic conventions, plus the llm.token_count.* namespace used by some instrumentation libraries): treated as inclusive. Langfuse normalizes them during ingestion by subtracting cache read and cache creation tokens from input.
• OpenAI usage schema passed as usage details (nested prompt_tokens_details / completion_tokens_details, see Compatibility with OpenAI): recognized and normalized during ingestion. The match is strict: if the object contains any key beyond the OpenAI usage fields, it is treated as flat keys instead.
• Langfuse-style flat keys passed as usage details via the SDKs or the API (usage_details / usageDetails, OTel attribute langfuse.observation.usage_details): stored verbatim, no normalization is applied. Values must already be exclusive.

If you write your own instrumentation that sets usage details with flat keys, check how your provider reports usage and subtract cached or detail token counts from the top-level input/output counts — exactly once — before passing them to Langfuse.

Compatibility with OpenAI

For increased compatibility with OpenAI, you can also use the OpenAI Usage schema. prompt_tokens will be mapped to input, completion_tokens will be mapped to output, and total_tokens will be mapped to total. The keys nested in prompt_tokens_details will be flattened with an input_ prefix and completion_tokens_details will be flattened with an output_ prefix. Since OpenAI reports these detail counts inclusively, Langfuse subtracts them from input and output respectively, so that the stored buckets are mutually exclusive.

from langfuse import get_client

langfuse = get_client()

with langfuse.start_as_current_observation(
    as_type="generation",
    name="openai-style-generation",
    model="gpt-4o"
) as generation:
    # Simulate LLM call
    # response = openai_client.chat.completions.create(...)

    generation.update(
        usage_details={
            # usage (OpenAI-style schema)
            "prompt_tokens": 10,
            "completion_tokens": 25,
            "total_tokens": 35,
            "prompt_tokens_details": {
                "cached_tokens": 5,
                "audio_tokens": 2,
            },
            "completion_tokens_details": {
                "reasoning_tokens": 15,
            },
        }
    )

import { startObservation } from "@langfuse/tracing";

const generation = startObservation(
  "openai-style-generation",
  {
    model: "gpt-4o",
    usageDetails: {
      // usage (OpenAI-style schema)
      prompt_tokens: 10,
      completion_tokens: 25,
      total_tokens: 35,
      prompt_tokens_details: {
        cached_tokens: 5,
        audio_tokens: 2,
      },
      completion_tokens_details: {
        reasoning_tokens: 15,
      },
    },
  },
  { asType: "generation" },
);
generation.end();

You can also ingest OpenAI-style usage via generation.update() and generation.end().

Troubleshooting

If you don't see cost on your generations, check the following:

• Usage or cost must be present. Langfuse can only infer cost if usage is ingested or a tokenizer is available for the model. If neither cost nor usage is ingested and the model has no tokenizer, no cost is calculated.
• The model must match a definition. Inferred cost requires a model definition whose match_pattern matches the generation's model value. If no model matches, add a custom model definition.
• Model definition changes only apply to new generations. If you change or add a model definition, the updated costs are applied only to generations logged afterward.
• Only generation and embedding observations track cost. Other observation types do not carry usage or cost.
• Reasoning models require ingested usage. Langfuse cannot infer cost for reasoning models such as the OpenAI o1 family without token counts. See cost inference for reasoning models.
• Gateways: If you use OpenRouter, Langfuse can directly capture the OpenRouter cost information. If you use LiteLLM, Langfuse directly captures the cost information returned in each LiteLLM response.

GitHub Discussions