Live LLAVA NanoDB integration

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I am using the Jetson Orin Nano to run the live LLAVA video query with the nanodb integration as follows:

jetson-containers run $(autotag nano_llm)
python3 -m nano_llm.agents.video_query --api=mlc
–model Efficient-Large-Model/VILA1.5-3b
–max-context-len 256
–max-new-tokens 32
–video-input /dev/video0
–video-output webrtc://@:8554/output
–nanodb /data/nanodb/coco/2017

However, i notice that once i start to run the code, before the gstreamer gets connected, my orin nano will hang for long before crashing and rebooting by itself.

I have changed the settings to run it in MAXN super as well as ensuring that the RAM is only being used on the inference done from the code. however this does not seem to be enough for the orin nano, and i am unable to check the tegrastats or free -h stats as the orin nano freezes before i can even reach that page.

Any advice is appreciated. Thanks

3

Hi,

Are you able to create another console to monitor the memory usage with tegrastats?
Thanks.

4

Hi @AastaLLL, I can create another console but I was unable to monitor as once the script runs with nanodb flag, the gpu freezes and there is no change to the screen and im unable to monitor any changes to the device.

Any advice on what to do next

5

There is no update from you for a period, assuming this is not an issue anymore.
Hence, we are closing this topic. If need further support, please open a new one.
Thanks ~0730

Hi,

Could you share the output of the tegrastats?
It’s expected that the RAM usage will be closer to the maximum before the freeze.

Thanks.