Bug report: CloudXR SDK inside of Unity

Since the first release of CloudXR our team has been working with a setup comprised of Unity + CloudXR. To be more exact, we built a Unity wrapper around the CloudXR SDK and where able to develop our CloudXR clients directly in Unity. Technically speaking the wrapper was composed of 2 components: one for Android and one for Windows.

Once the Android wrapper was up and running, we haven’t faced any major issues with it.
Unfortunately the Windows wrapper is a completely different story. With CloudXR 1.2 the wrapper worked perfectly and we were able to set up a sophisticated testing environment inside of Unity on Windows. In CloudXR 2.0 the wrapper was broken and it doesn’t seem that CloudXR 2.1 will solve this issue.

A more detailed explanation of the issue:
Before I describe this issue, please bear in mind that all of this is currently working on Android and this issue is independent of the Unity version. We have tested it on multiple Unity releases and the issue is exactly the same.

The core of the problem is that as soon we call cxrConnect, the engine/windowsBuild simply crashes. A call to cxrCreateReceiver creates a valid handle and once we try to connect using this handle, the system crashes. Looking at Unity’s logs, it seems that the crash happens due to a “null pointer dereference” in the SDK.

We have not been able to work around this issue and rewriting our testing setup in C++ is not an option. It is perplexing that is used to work perfectly fine in CloudXr 1.1 and 1.2, but not in 2.0 and 2.1. We consider this feature critical for our project.

We are ready to offer more information if needed!

1 Like

Hi,

Thanks for the report. Could you please provide the server/client logs? Section “5.3 Log Files” of the SDK overview guide includes their locations.

Thanks,
Will

Hey,

Here are the logs.
cxr-logs.zip (3.2 KB)

There is an apparent hour difference between the client and the server, but don’t be bothered by that.

By the way: in your log file, it says that the client is trying to delete the log file into which the current logging info is written. This also looks like a bug, or?

Hey,

just tested our Unity wrapper with the new CloudXR 2.1 release and we still get the same error. The log files look more or less the same:

CloudXR Client Log 2021-05-06 12.34.27
NVIDIA CloudXR v2.1 (CL# 29897464), built on Apr 28 2021 00:33:33.

[12:34:27.738] Cleaning up older files.
[12:34:27.739] Adapter 0: Description=‘NVIDIA GeForce RTX 2080 Super with Max-Q Design’
[12:34:27.792] Direct3D setup done.
[12:34:28.144] Using cuvid yuv2rgb path
[12:34:28.274] Using cuvid yuv2rgb path

As a total a-side, please post on the forums if you decide to make that wrapper public. I realize it’s a neat piece of code to keep inhouse, and I wouldn’t blame you, but would love to have access to something like that if you ever do decide to make it open source or available thru unity market place. Super cool.

4 Likes

Hi DtEdge,

We have the same issue - made a dll wrapper for Windows, it looked like it would work but crashed in cxrConnect(), a few calls deep. Looks like a null pointer dereference, and we got the same log output. I thought it might be related to the D3DDevice (which was difficult to get a handle to in the dll, but not impossible) and exclusive mode, but I couldn’t get any more info. We started building it on v2.0, so never had a 1.2 version working.

However, our Android wrapper didn’t work either, it looked like it couldn’t negotiate a connection (although it didn’t crash). Did you work through that problem?

Hey David,

We wrote our Android wrapper some time ago and I remember there where some problems similar to what you describe, but I don’t remember how we fixed it. But I still have an advice for you: the CloudXR SDK is full of bugs, undocumented features and to be honest it does not follow best practices.
If you’re passing a pointer to their SDK, do not assume they’re making a copy of your data. It seems they’re actually just storing the pointer and using it at some later point. Therefore, make sure the data you pass to their SDK will exist for the whole duration of the session.
Another tip I can give you is to basically replicate in Unity the exact code they have in their samples, because it doesn’t seem like they’re doing any testing outside those samples. (This is implicitly related to the pointer stuff)

Good luck with the Unity wrapper. I’m glad we’re not the only ones doing it. It would be interesting to compare our solutions at some later point.

Thanks for the tips, for now we have abandoned our plugin since we couldn’t get it to work on any platform. It is a shame though, since we are Unity developers and would be able to make a much more feature-rich, cross-platform viewer if we could do it within Unity.

Perhaps I will give it another blast when a new CloudXR version drops… I suspect our solutions were similar but yes it would be fun to compare if we can get it sorted out with NDAs. :)

1 Like

Has anyone tried version 3.0 of the CloudXR SDK?
Is there still the bug encountered?

I need to develop the same type of plugin for Android and Windows and I wanted to know if there was anything new.

At the moment, when I try to connect the client, Unity crashes (in editor mode or in executable mode) after this log line : “netStreamClient loadSymbols() FAILED, aborting.”
I have tried to use my plugin dll in a C++ application and it works. (log line becomes “Streamer version: 14”)
I have tried to use my plugin dll in a C# application and it works. (log line becomes “Streamer version: 14”)
The problem therefore comes from a kind off incompatibility between the cloud xr client and Unity.

I tried version 3.1 of the CloudXR SDK and I have the same behaviour:
“netStreamClient loadSymbols() FAILED, aborting.”
And Crash!!!

Hey,

So I managed to track down why the crash happens. I can’t explain it 100% but basically if you want to run CloudXr on Windows you need to put StreamClientShared.dll in the same folder as your build executable. This does mean that you can’t actually run CloudXr directly from the Unity editor. Or maybe it is possible, but it is just too much of a hassle without more information from the CloudXr development.
In 2 words the issue seems to be related to how CloudXr loads some of this dependencies on Windows.
Our windows wrapper is not up and running yet, as we stumbled on the next error: Failed to map the DX surface to CUDA, 101, but I’m quite optimistic we’ll find a solution there.

@sfarouk has previously done some work with our dependency handling, so tagging him to review this for whether we need to adjust our loading code further.

@m.luchian You need to have all the DLLs from the SDK under Client/Lib/Windows available in the DLL search path for your process. The CUDA error you’re seeing is probably due to missing the cudart64_*.dll from that earlier directory.

1 Like

Hi guys,

Like some of you here, I am creating an Android plugin to be used by Unity. However, I get a DLLNotFoundException when I try calling the plugin from Unity. Here are the steps I took:

[Android Studio]
I have generated an AAR plugin from the hello_cloudxr_c sample Android project which uses CloudXR version 3.2. The only modifications I did to this to generate the AAR were in build.gradle (:app) where I changed the first line to apply plugin: ‘com.android.library’ and comment out applicationId “com.nvidia.ar.hellocloudxr” in the defaultConfig block.

I changed the path of -DARCORE_INCLUDE from ${project.rootDir}/…/…/libraries/include" to ${project.rootDir}/libraries/include"

I changed the path of -DGLM_INCLUDE from ${project.rootDir}/…/…/libraries/glm" to ${project.rootDir}/libraries/glm"

minSdkVersion is set to 25 and targetSDKVersion is 28.

In jni_interface.cc, I added a small function called getSomeNumber() which returns an integer.

extern "C" {
  int getSomeNumber() {
      return 2345;
  }
}

I will call this from Unity as a simple test.

After doing a clean build, app-debug.aar is created in app/build/outputs/aar. If I change the file extension to .zip, uncompress it and look in app-debug/jni/arm64-v8a, I see that only libc++_shared.so, libhello_cloudxr_native.so and liboboe.so are present. The CloudXR library isn’t there. I manually copy libCloudXRClient.so, libgrid.so, libGsAudioWebRTC.so and libPoco.so into the jniLibs folder in Android Studio. When I build again, all the required .so files are present in the AAR.

[Unity]
I have created a Unity project which has a scene containing a single button. The AAR file is copied into the Plugins folder. I have written a script called CXRConnect.cs which references the plugin and calls the getSomeNumber() function when the button is pressed.

public class CXRConnect : MonoBehaviour
{
    public Button button;
    public Text textObject;
    private const string LIBRARY_NAME = "libhello_cloudxr_native";
    
    [DllImport(LIBRARY_NAME)]
    private static extern int getSomeNumber();
        
    void Start()
    {
        Button btn = button.GetComponent<Button>();
        btn.onClick.AddListener(CallCPPLibrary);
    }
    void CallCPPLibrary()
    {
        int num = getSomeNumber();
        Debug.Log("CXR Some Num: "+num);
        textObject.text = num.ToString();
    }
}

In the Unity project settings:
Minimum API Level is set to Android 10.0 (API level 29)
Target API Level is set to Automatic (highest installed)
Scripting Backend is IL2CPP
Api Compatibility Level is .NET Standard 2.0
ARMv7 and ARM64 are ticked in Target Architectures

I am running the build on a Samsung Galaxy Note20 Ultra. When the app is running on the device and I press the button in the Unity scene, there is an error in Logcat in Android Studio which reads:

2022-07-06 10:59:42.240 3203-18112/? E/Unity: DllNotFoundException: Unable to load DLL 'hello_cloudxr_native': The specified module could not be found.
at CXRConnect.getSomeNumber () [0x00000] in <00000000000000000000000000000000>:0
at CXRConnect.CallCPPLibrary () [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Events.UnityAction.Invoke () [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Events.UnityEvent.Invoke () [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1].Invoke (T1 handler, UnityEngine.EventSystems.BaseEventData eventData) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.EventSystems.StandaloneInputModule.ProcessTouchPress (UnityEngine.EventSystems.PointerEventData p

Have any of you encountered this issue? What could be the possible cause and how should I fix it? I have checked the contents of the plugin and all the required libraries are in there.

Yes, we are having issues implementing the .AAR library into our Unity project as well.

@tegradave @wrice have you come across this issue before or have an idea of what is causing the problem?

Just hit this exact same bug in-editor working on a port of 3.2 SDK to Unity.

Visual Studio Output log:

[00:02:37.963] Missing DLL symbols!
[00:02:37.963] netStreamClient loadSymbols() FAILED, aborting.
Connection attempt failed. [6]
Exception thrown at 0x0000000000000000 in Unity.exe: 0xC0000005: Access violation executing location 0x0000000000000000.

All DLLs from 2022-05-05-CloudXR-SDK\Client\Lib\Windows were copied into Assets\Plugins\x86_64 .

Hint from @m.luchian helped. I just dumped all of the CloudXR-SDK\Client\Lib\Windows DLLs into the same directory as my Unity.exe, and now it works for me in-editor / no-more-crash.