Large Language Model Session Token Counter - Cognigy Documentation

Documentation Index

Fetch the complete documentation index at: /llms.txt

Use this file to discover all available pages before exploring further.

In Cognigy.AI, you can use the api.getLLMTokenUsageForSession() API call in a Code Node or an Extension Node to fetch the Large Language Model (LLM) total token usage in a conversation or chat session. This way, you can track your LLM providers costs. The LLM Session Token Counter retrieves the token usage information via LLM provider-specific API calls to return the exact number of tokens the LLM uses. If the information isn’t available by the provider, Cognigy.AI estimates the token usage.

Prerequisite

You have added a Node that uses an LLM to your Flow, for example, an AI Agent Node, LLM Prompt Node, Search Extract Output Node, or Agent Copilot Nodes.

Limitations

The LLM Session Token Counter doesn’t support embedding models from Aleph Alpha.

Fetching Session-Wide Token Usage

Add a Code Node after the Node that uses an LLM, for example, after an AI Agent Node.
Add the following code to the Code Node:

   const tokens = api.getLLMTokenUsageForSession();

This Code Node fetches the total token usage in the session and stores it in the token variable.

Returned JSON Object

The following JSON example shows an object returned by api.getLLMTokenUsageForSession():

{
    "1c945b38-5dbb-4fcf-9fdd-112bd8177f9c": {
        "IlmDisplayName": "openAI - text- embedding-3-large - 1735891122507",
        "providerType": "openAI",
        "modelType" : "text-embedding-3-large",
        "usage": {
            "inputTokens": 6,
            "outputTokens": 0
        }
    },
    "ce190e1f-eff7-4f6b-9074-9f705375a3d3": {
        "IlmDisplayName": "openAI - gpt-40 - 1735891179896",
        "providerType": "openAI",
        "modelType": "gpt-40",
        "usage": {
            "inputTokens": 441,
            "outputTokens": 106
        }
    }
}

Key	Type	Description	Example
llmReferenceId	String	The LLM reference ID.	`ce190e1f-eff7-4f6b-9074-9f705375a3d3`
llmDisplayName	String	The name of the LLM.	`openAI - gpt-40 - 1735891179896`
providerType	String	The LLM provider.	`openAI`
modelType	String	The LLM model.	`gpt-40`
usage	Object	The object containing information about the session total token usage.	-
inputTokens	Number	The number of input tokens.	441
outputTokens	Number	The number output tokens. The output tokens count for embedding models is always 0 since embedding models output embedding vectors instead of tokens.	106

For testing in the Interaction Panel, you can output the JSON object by adding api.say() at the end of your Code Node.

More Information

Last modified on April 21, 2026.