InferaStack Gateway API

Private preview

Status: private preview. The hosted Gateway is deployed for invited applications and every request needs an InferaStack API key. In development for customers and not yet available to them.

The Gateway is designed to follow the OpenAI Chat Completions API, so existing OpenAI SDK code is designed to need only three changes: the base URL, an InferaStack API key and a model alias.

Your first request, with the OpenAI Python SDK

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferastack.com/v1",  # 1 base URL
    api_key=os.environ["INFERASTACK_API_KEY"],  # 2 API key
)

response = client.chat.completions.create(
    model="inferastack-chat-fast",              # 3 model alias
    messages=[{"role": "user", "content": "Hello from Sydney"}],
)
print(response.choices[0].message.content)

What the preview covers

Endpoint	`POST /v1/chat/completions` only. Streaming and tool calls are not in the preview.
API keys	One bearer key per organisation, each with a daily budget cap.
`inferastack-chat-fast`	High-volume, latency-sensitive tasks: drafts, classification, extraction.
`inferastack-chat-smart`	Complex reasoning and longer answers.
Routing and region	We can change the model behind an alias without changes to your code. Both aliases call Amazon Bedrock APAC cross-Region inference profiles from our Sydney (`ap-southeast-2`) gateway, so models may run in other APAC Regions: execution is not Australia-only today. Processing locations and any cross-Region routing must meet the agreed workload policy.
Costs and records	Designed to record each request's cost against its key and stop requests at the daily cap. Audit records are designed to contain metadata only. Cost reporting starts at onboarding.
Errors	Designed to use the OpenAI error format, so existing retry logic keeps working. Final error codes and HTTP statuses are published at onboarding.
InferaStack AI Grid	The same API is designed to serve InferaStack AI Grid sites and record which site ran each request. The grid is in technical validation; its design is separate from these aliases.

Request early access

We will tell you when the Gateway opens. Beyond the hosted service, deployments in your own VPC or a dedicated data centre are planned.

Request early access