Seamless Operation with LLM Fallback and More in Cognigy.AI v4.90

Authors name: Nhu Ho November 29, 2024

Cognigy.AI v4.90 introduces LLM Fallback to help you optimize service continuity when leveraging Generative AI for your AI Agents.

Maintain Seamless Performance of LLM-Powered AI Agents

LLM connections can occasionally fail due to multiple reasons like outdated/incorrect credentials, exceeded rate or token limits, server timeout, and more.

To combat these challenges, Cognigy’s LLM Fallback ensures continuous operation by automatically activating an alternative model when the primary one encounters an issue. This proactive safeguard minimizes downtime and keeps your AI Workforce running smoothly.

Key Benefits:

• Improved service reliability and user experience
• Rapid recovery with automatic fallback switching during outages
• Increased flexibility by deploying region-specific backups or using fallback models to handle high-demand periods

How it Works:

You can set up a fallback for any chat or completion LLM by adding or selecting another model – regardless of the vendor within your LLM Resource.

The fallback model acts as a temporary replacement when the primary model encounters issues. At the same time, an email notification indicating that the main model requires attention can be triggered, allowing for timely resolution.

Other Improvements

Cognigy.AI

• Added a feature flag to activate and deactivate the OAuth2 connection type for the Azure OpenAI provider
• Granted users with the fullSupportUser role access to view Audit Events
• Renamed the Fallback Text field to Textual Description in Say Nodes
• Resolved a performance issue that slowed NLU Intent Training
• Renamed the Complete Goal Node to Complete Task and changed Goals to Tasks on the Insights Overview and Engagement dashboards

Cognigy Voice Gateway

• Added the CALL_IN_PROGRESS call event to monitor the first user input. You can enable this call event in the Generic Settings section of the Endpoint Settings
• Added the capability to anchor media when transferring via dial to help resolve NAT problems, enforce media security policies, and perform media transcoding and monitoring. You can enable this in the Voice Gateway Endpoint Settings(Call Failover > Anchor Media) or Voice Gateway Transfer Node(Advanced > Anchor Media)
• Changed the audio file URL format requirement in the Voice Gateway Transfer Node(General settings > Dial Music) and Set Session Config Node(Atmosphere Sounds > URL). MP3 URLs no longer need to end with the .mp3 extension
• Set the minimum value for Continuous ASR to 2000 milliseconds (2 seconds). The value applies to both the Generic Continuous ASR - Config Node and the Continuous ASR in the Voice Gateway Set Session Config Node

Agent Copilot

• Added a Copilot: Send Data Node to update existing HTML and IFrame tiles in the AI Copilot workspace

For further information, check out our complete Release Notes here.