Unable to install SAM2 on Orin Nano

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
,
1

I’m trying to get Meta’s SAM2 model running in my Jetson, the model is published at:

With simple, straightforward setup instructions:

pip install -e .

However I get the following errors:

Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/souf/Desktop/segment-anything-2
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      /tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/torch/_subclasses/functional_tensor.py:258: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
        cpu = _conversion_method_template(device=torch.device("cpu"))
      Traceback (most recent call last):
        File "/home/souf/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/souf/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/souf/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 458, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 70, in <module>
        File "<string>", line 51, in get_extensions
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1076, in CUDAExtension
          library_dirs += library_paths(cuda=True)
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1207, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 2416, in _join_cuda_home
          raise OSError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Platform is Jetson Orin Nano with Jetpack 6, Torch & Torchvision with Cuda support downloaded and tested.

I should also mention that the setup in a Google Colab T4 instance went without issue, so I doubt this has something to do with the packages.

EDIT: the error is complaining about CUDA_HOME not being set, however CUDA is installed and working, torch and torchvision are as well, and I manually set the CUDA_HOME variable to CUDA root path, I also set it as a global variable for all users, just in case pip was doing something with different users, but it did not help.

3

@user93959 have not been able to try SAM2 yet, but looks like an exciting model! If you get it working, I’m sure the community would appreciate sharing your results. You may need to work through some install things when trying them on Jetson aarch64+CUDA architecture (which we have many dockerfiles for in jetson-containers)

[quote=“user93959, post:1, topic:302009”]

        File "/tmp/pip-build-env-5750y179/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 2416, in _join_cuda_home
          raise OSError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

Is your CUDA Toolkit installed under /usr/local/cuda ? Have you tried setting CUDA_HOME environment variable (although that is not typically necessary). Also did you independently verify your PyTorch/torchvision installation to be working with CUDA by other means.

There is a previous container for SAM (version 1) here. That dockerfile itself is relatively simple, but it does use the other base containers which are tested during various build stages.

4

I should’ve mentioned in the post, I’ll edit right away, but cuda 12.2 is installed at the specific location, and I manually set CUDA_HOME, even as a global variable in case pip was doing something with users.

5

It looks like it may be encountering that error building a PyTorch extension. Beforehand did you install one of the PyTorch wheels built for Jetson, like from this topic:

6

I did that, downloaded the wheels from the documentation link for both PyTorch and Vision, I also made sure it works fine by running some DeepStream sample apps and trying few Python codes that runs CUDA with Pytorch.

8

@user93959 if you are still facing the issue, I would dig into the code and start patching/printing debug info to see why it throws that error during setup. Sometimes new packages may require some workarounds for aarch64.
Also it is unclear why your torch/utils/cpp_extension.py file would be under a tmp pip directory and not already installed.

9

I noticed that SAM2 requires torch>=2.3.1, torchvision>=0.18.1, so I updated torch to 2.4.0, and now I’m working on updating torchvision to 0.19 and see if that does it.

10

Sometimes the requirements.txt just says that or was frozen with it, but does not actually require it, and by virtue of changing around all the versions the CUDA-enabled builds of them get uninstalled/replaced. It sounds like you are already confirming that GPU still works in these after the setup (which is good), but you can also try editing the SAM2 requirements to loosen them. This is another thing I will typically do when trying to build new ML packages/models.

11

Hey, I am trying to do the same thing, but no success so far

12

The github installation section mentions:

The code requires python>=3.10, as well as torch>=2.3.1 and torchvision>=0.18.1

I installed torch 2.4.0 but having issues building torchvision 0.19 or 0.18.1, would you happen to have pre-built wheels I can install?

13

Hi @user93959, I have torchvision 0.18 wheel here for JP6 / CUDA 12.2, perhaps that could work for you - http://jetson.webredirect.org/jp6/cu122

Otherwise I would use jetson-containers to build your desired environment, or relax the requirements of SAM2 by loosening its dependency versions in its requirements.txt

14

Solved!

Firstly I ran the install command with --no-cache-dir and --no-build-isolation, I noticed the no-build-isolation causes to download torch and torchvision, in the end it’ll complain about the versions incompatibility but it installs anyways, after that I needed to reinstall torch and torchvision from Jetson, I used torch 2.4.0a0+07cecf4168.nv24.5 and
torchvision 0.18.0a0+6043bc2, again torchvision complained about version incompatibility with SAM2 since it requires >= 0.18.1 but it runs anyway.

I don’t know what complications can occur from running torchvision 0.18.0 but I couldn’t compile the 0.18.1, if anyone does have it please share.

Thank you @dusty_nv for your assistance!

15

Thanks @user93959, great to hear that you got it installed. I don’t expect the performance is great given that it has not been optimized outside of PyTorch, but any reference as to the kind of performance you are seeing?

16

Performance is terrible, although I do have issues with opencv when displaying the frames with masks, but it’s really slow.

Edit: My RDP connection is not helping as well, I will try using a separate monitor, maybe performance is not as terrible as I thought, but sill, Deepstream performs much better even when displayed in RDP.

17

OK yes, I can believe the default/unoptimized performance is slow as it is a large model. Judging the remote desktop refresh rate is hit or miss, but you could add some timing measurements to the code itself. I think over time that more optimized implementations of this will become available, hopefully reaching production in DeepStream or NIMs.

18

It seems advanced for me at the moment, but just out of curiosity, what does it take to optimize the model for Jetson / TensorRT?

19

Thanks for your interest in it @user93959, historically that looked like trying to export the model from PyTorch to ONNX and run it with TensorRT (or using torch2trt). However Transformer-based models are so much larger that it requires different techniques, including the use of other quantization libraries or inference APIs, like TensorRT-LLM, llama.cpp, ect.

For SAM2 approach, I would check out NanoOWL and NanoSAM projects from Jetson AI Lab. The VITs are sometimes small enough that they can be run through torch2trt or ONNX (or at least sections of them), however my understanding is that SAM2 is significantly more complex.

I’d anticipate that NVIDIA will optimize SAM2 through TAO / DeepStream, or hopefully one of us from the forums gets time to take a crack at it. I know that it’s definitely of interest, there are just a lot of models out to work on 😂

1 Like
20

Running SAM2 through DeepStream is something else! But yeah like you said too many models … hopefully we get something soon :)

Thanks for all the information @dusty_nv, I really appreciate it!!

21

Hey I’m also studying sam2 recently and want to deploy it on jetson. Can I ask what is the fps for segmentation?

22

I haven’t gotten a chance to test without additional delays from RDP connection and openCV lib, so I cannot say for sure what is the FPS, however it is really slow without optimization, hopefully that’ll happen soon.