Fastly, Inc. announced the launch of Fastly AI Accelerator, the company?s first AI solution designed to create a better experience for developers by helping improve performance and reduce costs across the use of similar prompts for large language models (LLM) apps. Fastly AI Accelerator is designed to reduce API calls and costs with intelligent, semantic caching. Built on Fastly?s Edge Cloud Platform and leveraging industry leading caching technology, AI Accelerator uses a specialized API gateway to dramatically improve performance for apps using popular LLMs, beginning with ChatGPT and expanding support to include additional models.

Popular AI applications can process hundreds of thousands of API calls or questions daily. Many of the questions users ask are likely very similar and may have been asked before. Without semantic caching, each call to the LLM requires going back to the provider for the information, potentially increasing costs and latency.

However, Fastly AI Accelerator?s semantic caching provides a cached response for repeated queries directly from Fastly?s high performance edge platform, instead of going back to the AI provider, helping to deliver a better experience by improving performance while reducing costs. When using Fastly AI Accelerator, developers only need to update their app to use a new API endpoint, which typically only requires changing a single line of code. Fastly AI Accelerator will then transparently implement semantic caching for OpenAI compatible APIs.

This approach goes beyond traditional caching as Fastly AI Accelerator is able to understand the context of the requests and queries, and will send a similar response if two or more requests are alike. To help developers build faster, more secure and more engaging experiences, Fastly is also now making it even easier for developers to try Fastly with an expanded free account tier that helps coders set up a new site, create a new app, or launch a new service in just a few minutes. Free tier accounts also include access to Fastly?s Content Delivery Network (CDN), generous memory and storage allotments, uncapped redirects, page rules and regular expressions.

Plus, the free Fastly tier includes security features such as TLS and always-on DDoS mitigation, observability tools, and much more.