Connection problem due to lack of CORS support in Triton Server, which blocks requests from frontend web applications

AI & Data Science Deep Learning (Training & Inference) TensorRT
, , ,
1

Description

Lack of native support for CORS (Cross-Origin Resource Sharing) in Triton Inference Server 23.10, which prevents direct integration with web frontend applications running on different domains. HTTP requests from web browsers (localhost:3000) to Triton Server (localhost:8000) are blocked by CORS security policies.

Environment

TensorRT Version: 23.10-py3
GPU Type: Included in container 23.10
Nvidia Driver Version: 535.154.05
CUDA Version:12.2
CUDNN Version: 8.9.5
Operating System + Version: Ubuntu 22.04.3 LTS
Python Version (if applicable): 3.10
TensorFlow Version (if applicable): Not applicable (direct use of ONNX)
PyTorch Version (if applicable): Not applicable
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tritonserver:23.10-py3

Relevant Files

Steps To Reproduce

  1. Environment Setup

# Pull the Triton container
docker pull nvcr.io/nvidia/tritonserver:23.10-py3

# Check GPU and drivers
nvidia-smi

  1. Execution of Triton Server

# Exact execution command
docker run --gpus=1 -p 8000:8000 -p 8001:8001 -p 8002:8002 \
  -v /home/user/model_repository:/models \
  nvcr.io/nvidia/tritonserver:23.10-py3 \
  tritonserver --model-repository=/models \
  --allow-http=true \
  --http-port=8000 \
  --log-verbose=1

  1. Server Verification

# Verify that Triton is working
curl -v http://localhost:8000/v2/health/ready

# Check loaded models
curl http://localhost:8000/v2/models/model

  1. Execution of the Frontend Application

# Install dependencies
npm install

# Run application
npm run dev

  1. Error Reproduction

// From the browser console (http://localhost:3030)
fetch('http://localhost:8000/v2/health/ready', {
  method: 'GET',
  headers: {
    'Content-Type': 'application/json'
  }
})
.then(response => console.log(response))
.catch(error => console.error(error));

Full Trace of Errors

Error in Browser (Console)

Access to fetch at ‘http://localhost:8000/v2/health/ready’ from origin ‘http://localhost:3030’has been blocked by CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

GET http://localhost:8000/v2/health/ready net::ERR_FAILED 200 (OK)

Triton Server Logs (Technical Success)

I0822 22:12:11.355377 1 http_server.cc:4384] HTTP request: 8 /v2/health/ready
I0822 22:12:11.355516 1 http_server.cc:4384] HTTP request: 8 /v2/health/ready
I0822 22:12:11.367992 1 http_server.cc:4384] HTTP request: 8 /v2/health/ready
# ... múltiples solicitudes recibidas correctamente

Missing Response Headers

HTTP/1.1 200 OK
Content-Length: 25
Content-Type: application/json
Date: Wed, 22 Aug 2023 22:12:11 GMT
# ❌ Faltan headers CORS:
# Access-Control-Allow-Origin: *
# Access-Control-Allow-Methods: GET, POST, OPTIONS
# Access-Control-Allow-Headers: Content-Type

CORS Configuration Attempt (Error)

# Comando fallido - parámetros no reconocidos
docker run --gpus=1 -p 8000:8000 -p 8001:8001 -p 8002:8002 \
  -v /home/user/model_repository:/models \
  nvcr.io/nvidia/tritonserver:23.10-py3 \
  tritonserver --model-repository=/models \
  --allow-http=true \
  --cors-allow-origin="http://localhost:3030" \
  --cors-allow-methods="GET,POST,OPTIONS" \
  --log-verbose=1Error resultante:

Resulting error:

tritonserver: unrecognized option '--cors-allow-origin=http://localhost:3030'
4

Hi ,

I would request you to please reach out to Triton Inference Server Github page, as this looks like a Triton issue