MLC v0.1.2 for nanollm docker image to run llama 3.2 1B

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
,
1

Hi,
I was unable to run llama 3.2 1B language model inside the nanollm:r36.4.0 image due to current mlc version is not support for the llama 3.2. Is there way to update only mlc version inside the nanollm docker image because there are no space left to download another docker image for only mlc version , and is there any way to delete other language models that installed in nanollm docker image?

3

Hi,

You can find an example below:

For llama 3.2 1B, we can run it with dustynv/mlc:r36.4.0 container.

Thanks.