Scoped and quoted before work begins

Prove one inference workload before you commit to capacity.

For application teams, integrators and enterprises considering Australian AI infrastructure. Agree what success looks like, evaluate the workload and make the next capacity decision from measured results.

Discuss a vision pilotDiscuss an LLM pilot

Start with the application

Vision/video is our priority starting point. LLM evaluation needs its own model, workload and service criteria.

Vision / Video

From camera input to a useful event

Define the event the application must detect and the operating conditions it must handle.

  • Detection quality on an agreed, labelled evaluation set
  • End-to-end event latency and camera concurrency
  • Transported bytes and cost per camera-hour or useful event
  • Measured continuity and recovery for the agreed topology
LLM Inference

From request to an accepted response

Fix the model, precision, context, output length and arrival pattern before comparing results.

  • Task quality against a customer-approved evaluation set
  • Time to first token and token delivery latency
  • Concurrency, useful throughput and cost per accepted task
  • Availability and recovery evaluated separately from vision

A defined scope and a decision you can use

1 · Agree

Workload and controls

One application, an authorised evaluation dataset, the test environment, success thresholds, responsibilities, price and timeline.

2 · Measure

Reproducible evaluation

A documented baseline and load profile, with quality, latency, throughput and resource costs measured under the agreed conditions.

3 · Decide

Acceptance and next step

A written result, limitations and an accept, revise or stop decision. Any hardware purchase or reserved service is a separate commitment.

Evaluation environments and availability require confirmation. A proxy environment can validate software behaviour; its results do not establish performance on a different GPU system.

The proposed validation environment

The InferaStack AI Grid lab is designed for Amazon EKS with NVIDIA L4 GPUs in AWS Sydney. NVIDIA DeepStream handles the video evaluation; NVIDIA Dynamo with its TensorRT-LLM engine handles the LLM evaluation. The lab precedes validation on the proposed HGX B300 site system.

Bench and installation

Measure the selected system

NVIDIA's public NeMo performance recipes guide compute benchmarking, alongside AWS GPU benchmarking methods and health checks. Record the hardware, model and workload with every result.

Video continuity

Validate recovery separately

After the pilot gate, the AWS design adds video-only cold-standby failover and overflow. Recovery time and capacity need measurement before any service commitment.

LLM boundary

No AWS backup in this design

The LLM lab test does not establish a standby service. The proposed InferaStack AI Grid resilience scope does not include LLM backup.

The first HGX B300 system is proposed and not yet ordered. Software integration details remain under design. See the full validation plan →

Agree data handling before testing

Confirm where data executes, who can access it, what content is retained and when it is deleted. An Australian control plane alone does not establish Australian-only processing. Sensitive data requires a specifically approved environment.

InferaStack AI Grid remains at proof-of-concept and technical-validation stage with no hardware-validated nodes. The Gateway is in development and is not a released customer service.

Bring one workload and its success criteria.

Tell us the application, data location requirements, expected load, budget and timing. We will confirm a practical scope.

Discuss an inference pilotExplore a dedicated deployment