MLC v0.1.2 for nanollm docker image to run llama 3.2 1B

Robotics & Edge Computing Jetson Systems Jetson Orin NX
,
1

Hi,
I was unable to run llama 3.2 1B language model inside the nanollm:r36.4.0 image due to current mlc version is not support for the llama 3.2. Is there way to update only mlc version inside the nanollm docker image because there are no space left to download another docker image for only mlc version , and is there any way to delete other language models that installed in nanollm docker image?

3

Hi,

You can find an example below:

For llama 3.2 1B, we can run it with dustynv/mlc:r36.4.0 container.

Thanks.

4

I run the compose file and here this got : " * Executing task: docker run --rm -d dustynv/mlc:r36.4.0

0d8fc62bf32f187d2e73915968db3d38547512b363af8cc06eb1bb029d023441
docker: Error response from daemon: failed to set up container networking: failed to create endpoint gallant_bouman on network bridge: Unable to enable DIRECT ACCESS FILTERING - DROP rule: (iptables failed: iptables --wait -t raw -A PREROUTING -d 172.17.0.2 ! -i docker0 -j DROP: iptables v1.8.7 (legacy): can’t initialize iptables table `raw’: Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
(exit status 3))

Run ‘docker run --help’ for more information

  • The terminal process “/usr/bin/bash ‘-c’, ‘docker run --rm -d dustynv/mlc:r36.4.0’” failed to launch (exit code: 125).
  • Terminal will be reused by tasks, press any key to close it.
    "
5

Hi,

This is a known issue.
Please downgrade the docker to version 27.5.1 to avoid the error:

sudo apt-get install -y docker-ce=5:27.5* docker-ce-cli=5:27.5* --allow-downgrades

Thanks.