OpenAI opens Decisions API to all customers — TechCrunch

OpenAI has opened access to the Decisions API for all customers — a model for quickly choosing between specified options and assessing confidence in such a decision. TechCrunch reports. The company announced the tool during its developer day on September 29, and it became available to all customers on Thursday.

A faster tool for developers

Models of this type are called decision models or System One models. They can quickly select an option from a set of proposed choices and provide a confidence estimate. Developers use them in individual workflows instead of large language models because such models operate faster and cost less.

OpenAI API head Nikunj Handa told the publication that the team retuned ChatGPT Luna to speed up inference and reduce the model's operating cost. According to him, in some tasks, latency was reduced from one to three seconds to 100–180 milliseconds.

Testing and capabilities

In the public version of the Decisions API, OpenAI added voice and visual capabilities and distributed computing resources across regions to reduce latency. During initial trials with Box, Datadog, Ramp, and Speak, engineers replaced OpenAI's large language models with the new API to increase throughput.

More current news is available on the UA.News Telegram channel Telegram.

Speak CTO Andrew Hsu said that the company, which creates AI tutors for language learning, reduced the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to improve the response speed of its Dots voice assistant.

Competition among decision models

The Decisions API competes, among others, with the Jev model from startup TypeSafe. In the AIM System One Index and Artificial Analysis Decision Index rankings, OpenAI's model ranked eighth and fourteenth, respectively. D1 Ultra by Liquid and Elephant-1 by Base Lab were named the leaders of these indexes.

Handa noted that Luna's advantage could be multimodality, including image classification. He also suggested that further reducing latency to about 60 milliseconds could make it possible to use the model to analyze every frame of a video.