Building Morpheus "nothing provides requested mrc 23.11"

kzietara · July 31, 2023, 12:25pm

Hello, while building Morpheus on Jetson AGX Orin container build fail with following messages:

#26 384.4 Leaving build/test directories:
#26 384.4   Work:
#26 384.4  /opt/conda/conda-bld/work
#26 384.4   Test:
#26 384.4  /opt/conda/conda-bld/test_tmp
#26 384.4 Leaving build/test environments:
#26 384.4   Test:
#26 384.4 source activate  /opt/conda/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl
#26 384.4   Build:
#26 384.4 source activate  /opt/conda/conda-bld/_build_env
#26 384.4
#26 384.4
#26 384.4     solution = solver.solve_for_action(_specs, prefix)
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/boa/core/solver.py", line 244, in solve_for_action
#26 384.4     t = self.solve(specs)
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/boa/core/solver.py", line 234, in solve
#26 384.4     raise RuntimeError("Solver could not find solution." + error_string)
#26 384.4 RuntimeError: Solver could not find solution.Mamba failed to solve:
#26 384.4  - versioneer-518
#26 384.4  - pyarrow * *_cuda
#26 384.4  - cudf 23.06.*
#26 384.4  - mrc 23.11.*
#26 384.4  - cuda-toolkit 11.8.*
#26 384.4  - libcudf 23.06.*
#26 384.4  - librdkafka 1.9.2.*
#26 384.4  - pybind11-stubgen 0.10.5.*
#26 384.4  - scikit-build 0.17.1.*
#26 384.4  - cython >=0.29,<0.30
#26 384.4  - python 3.10.*
#26 384.4  - rapidjson 1.1.*
#26 384.4  - pip
#26 384.4
#26 384.4 with channels:
#26 384.4  - rapidsai
#26 384.4  - nvidia
#26 384.4  - nvidia/label/dev
#26 384.4  - pytorch
#26 384.4  - conda-forge
#26 384.4
#26 384.4 The reported errors are:
#26 384.4 - Encountered problems while solving:
#26 384.4 -   - nothing provides requested mrc 23.11.*
#26 384.4 -
#26 384.4
#26 384.4 During handling of the above exception, another exception occurred:
#26 384.4
#26 384.4 Traceback (most recent call last):
#26 384.4   File "/opt/conda/bin/conda-mambabuild", line 10, in <module>
#26 384.4     sys.exit(main())
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main
#26 384.4     call_conda_build(action, config)
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build
#26 384.4     result = api.build(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 253, in build
#26 384.4     return build_tree(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3799, in build_tree
#26 384.4     packages_from_this = build(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2469, in build
#26 384.4     output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/render.py", line 923, in expand_outputs
#26 384.4     for output_dict, m in deepcopy(_m).get_output_metadata_set(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/metadata.py", line 2565, in get_output_metadata_set
#26 384.4     conda_packages = finalize_outputs_pass(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/metadata.py", line 909, in finalize_outputs_pass
#26 384.4     fm = finalize_metadata(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/render.py", line 636, in finalize_metadata
#26 384.4     m = parent_metadata.get_output_metadata(m.get_rendered_output(m.name()))
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/metadata.py", line 2656, in get_rendered_output
#26 384.4     for output_ in self.get_rendered_outputs_section(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/metadata.py", line 2638, in get_rendered_outputs_section
#26 384.4     self._get_contents(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/metadata.py", line 1904, in _get_contents
#26 384.4     rendered = template.render(environment=env)
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
#26 384.4     self.environment.handle_exception()
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
#26 384.4     raise rewrite_traceback_stack(source=source)
#26 384.4   File "<template>", line 45, in top-level template code
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/jinja_context.py", line 336, in pin_compatible
#26 384.4     pins, _, _ = get_env_dependencies(m, "host", m.config.variant)
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/conda_build/render.py", line 151, in get_env_dependencies
#26 384.4     actions = environ.get_install_actions(
#26 384.4   File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 150, in mamba_get_install_actions
#26 384.4     raise err
#26 384.4 conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-aarch64: {MatchSpec("mrc=23.11")}
#26 ERROR: executor failed running [/bin/bash -c /opt/conda/bin/mamba install -y -n base -c conda-forge "git-lfs" &&    source activate base &&    git lfs install &&    git config --global --add safe.directory "*" &&    cd ${MORPHEUS_ROOT_HOST} &&    MORPHEUS_PYTHON_BUILD_STUBS=OFF CONDA_BLD_PATH=/opt/conda/conda-bld ./ci/conda/recipes/run_conda_build.sh morpheus]: exit code: 1
------
 > [conda_bld_morpheus 3/3] RUN --mount=type=cache,id=workspace_cache,target=/workspace/.cache,sharing=locked     --mount=type=cache,id=conda_pkgs,target=/opt/conda/pkgs,sharing=locked     /opt/conda/bin/mamba install -y -n base -c conda-forge "git-lfs" &&    source activate base &&    git lfs install &&    git config --global --add safe.directory "*" &&    cd . &&    MORPHEUS_PYTHON_BUILD_STUBS=OFF CONDA_BLD_PATH=/opt/conda/conda-bld ./ci/conda/recipes/run_conda_build.sh morpheus:
------
executor failed running [/bin/bash -c /opt/conda/bin/mamba install -y -n base -c conda-forge "git-lfs" &&    source activate base &&    git lfs install &&    git config --global --add safe.directory "*" &&    cd ${MORPHEUS_ROOT_HOST} &&    MORPHEUS_PYTHON_BUILD_STUBS=OFF CONDA_BLD_PATH=/opt/conda/conda-bld ./ci/conda/recipes/run_conda_build.sh morpheus]: exit code: 1

Build info:

Building morpheus: with args...
   CUDA_MAJOR_VER       : 11
   CUDA_MINOR_VER       : 8
   CUDA_REV_VER         : 0
   DOCA_ARTIFACTS_HOST  :
   DOCA_REPO_HOST       :
   FROM_IMAGE           : nvidia/cuda
   LINUX_DISTRO         : ubuntu
   LINUX_VER            : 22.04
   MORPHEUS_ROOT_HOST   : .
   MORPHEUS_SUPPORT_DOCA: OFF
   PYTHON_VER           : 3.10
   RAPIDS_VER           : 23.06
   TENSORRT_VERSION     : 8.2.1.3

Can you assist? I looked at forums but found no solution.

bjarmak · August 16, 2023, 1:24am

Hey @kzietara thanks for reaching out.

We have not validated that Morpheus works on Jetson AGX Orin, as Morpheus depends on RAPIDS (a set of NVIDIA GPU-accelerated libraries) which in general does not support Jetson architectures. RAPIDS ARM support is specifically SBSA where Jetson is aarch64.

We’re very active on GitHub and invite feature requests so we can discuss and track our users’ needs.

The specific error message here is because Morpheus doesn’t have linux-aarch64 buids for MRC available on Conda - only linux-64 (see here). If you do attempt the build, you’ll first need to run the MRC Conda build in the same container, and then run the Morpheus build. Instructions for building the MRC Conda package are here.

Thanks again and please do file an issue or discussion if you attempt the build.

kzietara · September 6, 2023, 8:42am

Hello @bjarmak , thank you for your reply. If I attempt the build, I’ll make the necessary steps.