TensorRT-llm on Blackwell AGX thor

[AastaLLL]

Hi,

Do you use an LLM model?
If not, you can use TensorRT directly.

For LLM, you can find some benchmark data with vLLM in the topic below:

Announcing new VLLM container & 3.5X increase in Gen AI Performance in just 5 weeks of Jetson AGX Thor Launch

A key advantage of the NVIDIA software stack is its commitment to continuous improvement. Frequent software optimizations mean that existing models are consistently accelerated—delivering better performance with simple software updates. We’ve demonstrated this with Jetson platforms like Orin and Xavier, and are now showcasing these gains on Jetson AGX Thor. Just a few weeks after launch, we’ve boosted Gen AI performance on Jetson AGX Thor by up to 3.5X compared to our initial results we showcas…

Thanks.