Isaac Sim Version
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Operating System
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):
GPU Information
- Model: NVIDIA GeForce RTX 3090 Ti
- Driver Version: 535.183.01
Topic Description
Detailed Description
I’m trying to generate multiple cameras (one for each cloned environment) and wrap them up into the CameraView Class on the GPU, but an error has occurred.
I feel like this isn’t the right way to create multiple cameras and get the frame of each environment. Could you give me some guidance on how to do this?
Steps to Reproduce
franka_new.zip (1.3 KB)
Error Messages
[10.761s] [ext: omni.physx.fabric-106.5.7] startup
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: Traceback (most recent call last):
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/franka_new.py", line 97, in <module>
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: main()
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/franka_new.py", line 83, in main
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: load_assets(world, cloner, target_paths)
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/franka_new.py", line 54, in load_assets
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: cameras = CameraView(prim_paths_expr="/World/envs/env.*/camera", name="camera_view")
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.sensors.camera/isaacsim/sensors/camera/camera_view.py", line 172, in __init__
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: XFormPrim.__init__(
2025-04-02 20:37:06 [11,267ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.core.prims/isaacsim/core/prims/impl/xform_prim.py", line 147, in __init__
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: self._set_xform_properties()
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.core.prims/isaacsim/core/prims/impl/xform_prim.py", line 1205, in _set_xform_properties
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: current_positions, current_orientations = self.get_world_poses()
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.sensors.camera/isaacsim/sensors/camera/camera_view.py", line 357, in get_world_poses
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: orientations = self._convert_camera_axes(orientations, U_W_TRANSFORM)
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: File "/home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.sensors.camera/isaacsim/sensors/camera/camera_view.py", line 304, in _convert_camera_axes
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: converted_matrices = torch.matmul(
2025-04-02 20:37:06 [11,268ms] [Error] [omni.kit.app._impl] [py stderr]: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_bmm)
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,289ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2025-04-02 20:37:06 [11,290ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2025-04-02 20:37:06 [11,296ms] [Warning] [omni.graph.core.plugin] Could not find category 'animation' for removal
2025-04-02 20:37:06 [11,296ms] [Warning] [carb] Client omni.syntheticdata.plugin Failed to acquire interface [omni::graph::core::INode v4.10] while unloading all plugins
Fatal Python error: Segmentation fault
Current thread 0x0000728341dca440 (most recent call first):
<no Python frame>
Extension modules: omni.mdl.pymdlsdk._pymdlsdk, psutil._psutil_linux, psutil._psutil_posix, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg._matfuncs_expm, scipy.linalg._linalg_pythran, scipy.linalg.cython_blas, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.spatial._ckdtree, scipy._lib.messagestream, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.spatial.transform._rotation, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, PIL._imaging, PIL._imagingft, osqp._osqp, scipy.io.matlab._mio_utils, scipy.io.matlab._streams, scipy.io.matlab._mio5_utils, yaml._yaml (total: 74)
Segmentation fault (core dumped)
Additional Information
What I’ve Tried
- Remove
.to(device)inCamera(). Did not solve.
position=torch.tensor([0.5, 0.0, 0.2]).to(device),
orientation=tensor_rot_utils.euler_angles_to_quats(torch.tensor([0.0, 60.0, 180.0]).to(device), degrees=True).to(device),
to
position=torch.tensor([0.5, 0.0, 0.2]),
orientation=tensor_rot_utils.euler_angles_to_quats(torch.tensor([0.0, 60.0, 180.0]), degrees=True)
- Change to CPU (numpy) solved. But I have to use GPU for deformables.
Related Issues
I also noticed that when creating multiple cameras in this way, every camera will actually capture all the information in the view, not only a specific environment. What would be a better way to allow each camera to filter out information outside its own environment besides setting the view in a narrow range?
Since CameraView isn’t working, if I tried to pack all cameras using XFormPrim, I won’t be able to call get_current_frame because they’re not camera prims anymore.
Additional Context
From the error output, in file /home/anaconda3/envs/env_isaaclab/lib/python3.10/site-packages/isaacsim/exts/isaacsim.sensors.camera/isaacsim/sensors/camera/camera_view.py, line 304, in _convert_camera_axes
Change
converted_matrices = torch.matmul(
torch.tensor(transform_matrix[:3, :3], dtype=torch.float32), orientation_matrices
)
to
converted_matrices = torch.matmul(
torch.tensor(transform_matrix[:3, :3], dtype=torch.float32).to(orientation_matrices.device), orientation_matrices
)
solve the problem.
Don’t know if this is a bug or it’s not the correct usage of CameraView.