NVIDIA System Profiler 4.0 error with Jetson TX2

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I cannot connect with the System Profiler to my Jetson TX2. I get a Communication error, but the reasons are beyond obvious.

SshExecuteCommandError (4206) {
    OriginalExceptionClass: N5boost16exception_detail10clone_implIN14QuadDSshClient19SshExecuteSudoErrorEEE
    OriginalFile: /build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Sudo.cpp
    OriginalLine: 71
    OriginalFunction: virtual int QuadDSshClient::Sudo::Execute(const string&)
    SshCommand: echo OK
    SshExecuteCommandError (4206) {
        OriginalExceptionClass: N5boost16exception_detail10clone_implIN14QuadDSshClient22SshExecuteCommandErrorEEE
        OriginalFile: /build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Command.cpp
        OriginalLine: 135
        OriginalFunction: void QuadDSshClient::Command::CloseChannelAndReadOutput(const IChannelPtr&)
        TimeoutError (125) {
            OriginalExceptionClass: N5boost16exception_detail10clone_implIN11QuadDCommon16TimeoutExceptionEEE
            OriginalFile: /build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Session.cpp
            OriginalLine: 1196
            OriginalFunction: void QuadDSshClient::Session::ChannelInfo::WaitNotification(QuadDSshClient::Session*)
            ErrorText: Operation timeout.
        }
    }
}

Could anyone provide some hints please?

Thank you

2

Could you reproduce the same issue if use older system profiler? Just want to clarify if this issue is specific to profiler or the host setting.

Could you also share the environment of you host?

3

Hi,

The host system is Debian 9 with kernel 4.9.0. I know, I know it’s not on the list of supported
host systems, so I tried my luck on a Ubuntu 16.04 distribution. I get the same error message on both the systems.

I will try with a previous version of the system profiler and let you know how it goes.

Do I have to install anything else on the host or the target systems? Do I have to give
special permissions to any components of the System Profiler? Do I need to load some special
module on the target?

Thanks

4

This is the tutorial of TSP. Though it is not for latest release.
https://docs.nvidia.com/tegra-system-profiler/index.html#developertools/mobile/tegra_system_profiler/3.7-default/04-profiling-linux.htm%3FTocPath%3D_____5

5

I’m not sure why you linked the tutorial ?! I belive it has nothing to do with that.

It "obviously"™ a problem of communication between the host and the target system as
illustrated by the stack trace. From that stack trace is seems that some part of
the profiler is trying to execute something as root - not sure if it’s on the host
or on the target system .Do I need to be root on either of the systems(that is the
host PC or the target TX2) to be able to run this tool? Does the ssh connection require
to be able to connect as root?

6

I pasted this to you because tutorial has some note:

The No authentication option is useful for devices configured for passwordless login using root username. To enable such a configuration, edit the file /etc/ssh/sshd_config on the target and specify the following option:

PermitRootLogin yes

7

I’m not using passwordless login. Is that a requirement? Do I need to be root, on either of the systems, to run the tool?

I’ve just tried with the 3.7 version of the profiler and I get the following error:

KernelModuleServiceError (1406) {
    OriginalExceptionClass: N5boost16exception_detail10clone_implIN13QuadDAnalysis24KernelModuleServiceErrorEEE
    OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Host/Analysis/LinuxDevice.cpp
    OriginalLine: 259
    OriginalFunction: void QuadDAnalysis::LinuxDevice::InitializeKernelModule()
    ServiceError (1400) {
        OriginalExceptionClass: N5boost16exception_detail10clone_implIN13QuadDAnalysis12ServiceErrorEEE
        TargetSideError (1100) {
            MethodName: Initialize
            ServiceName: Service
            ErrorText: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/KernelModuleServiceImpl.cpp(168): Throw in function void QuadDDaemon::KernelModuleServiceImpl::InitializeInternal()
                Dynamic exception type: boost::exception_detail::clone_impl<QuadDDaemon::KmsModuleInitError>
                std::exception::what: KmsModuleInitError
                [boost::errinfo_nested_exception_*] = 
                  Throw in function (unknown)
                  Dynamic exception type: boost::exception_detail::clone_impl<QuadDDaemon::QMIOCTLError>
                  std::exception::what: QMIOCTLError
                  [QuadDCommon::tag_error_text*] = Permission denied
                  [QuadDDaemon::tag_error_code*] = 19
                  [QuadDDaemon::tag_throw_file*] = /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/qm.c
                  [QuadDDaemon::tag_throw_func*] = get_module_capabilities
                  13, "Permission denied"
                
            KmsModuleInitError (2004) {
                OriginalExceptionClass: N5boost16exception_detail10clone_implIN11QuadDDaemon18KmsModuleInitErrorEEE
                OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/KernelModuleServiceImpl.cpp
                OriginalLine: 168
                OriginalFunction: void QuadDDaemon::KernelModuleServiceImpl::InitializeInternal()
                QmIOCTLError (1802) {
                    OriginalExceptionClass: N5boost16exception_detail10clone_implIN11QuadDDaemon12QMIOCTLErrorEEE
                    OriginalLine: 260
                    Errno: 13
                    ErrorText: Permission denied
                    OriginalErrorCode: 19
                    OriginalFunction: get_module_capabilities
                    OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/qm.c
                }
            }
        }
    }
}
8

It is not needed to run as root.

When did you hit that error? Enter the device ip and username/pwd through “ssh connection” button?

Please share these info to us.

Close any running instances of the System Profiler application on host machine
Rename nvlog.config.template to nvlog.config in Host-x86_64/ directory
Start the application and attempt to connect to the target board
If the connection fails, close the host application
The SystemProfiler.log file should be created in Host-x86_64/ directory. Please share this file with us

9

I hit the error first time I ran the System Profiler. I created a new SSH connection, enterd the IP and the username, clicked connect, then I have been asked the password. I eneterd the password and it was saying “Connecting to device” somewhere in the left corner of the toolbar. I hit “More info …” button and I saw the error.

Now I closed everything and I rebooted the TX2. I get an error but I do not get the same error anymore.

Log file below:

20621        CrashReporter Server 8 client 7 for process

20621        CrashReporter Starting pipeServer.
20626        AppLib_Script Loading CorePlugin
20626        AppLib_Script Loading QuadDPlugin
20626 quadd_common_communication 1DeviceManager@AsyncProcessor[0xbf2b50] is creating.
20632 quadd_common_communication Thread[20632] started servicing 1DeviceManager@AsyncProcessor.
20631 quadd_common_communication Thread[20631] started servicing 1DeviceManager@AsyncProcessor.
20626 quadd_common_communication 1DeviceManager@AsyncProcessor[0xbf2b50] created.
20626 quadd_device_manager DeviceManager[0xc05700] created.
20626 quadd_device_manager DeviceManager[0xc05700] is starting.
20633 quadd_common_communication Thread[20633] started servicing 1DeviceManager@AsyncProcessor.
20635 quadd_common_communication Thread[20635] started servicing 1DeviceManager@AsyncProcessor.
20634 quadd_common_communication Thread[20634] started servicing 1DeviceManager@AsyncProcessor.
20626 quadd_device_monitor DeviceMonitor[0xc05880] is creating.
20626     quadd_device_adb AdbForwardingPool[0xbe2f50]: was created: starting port=15555 size=5

20626 quadd_device_monitor DeviceMonitor[0xc05880] created.
20626 quadd_device_monitor DeviceMonitor[0xc05880] is starting.
20626 quadd_device_monitor DeviceMonitor[0xc05880] started.
20626 quadd_device_manager DeviceManager[0xc05700] started.
20626 quadd_common_communication 2PerfModule@AsyncProcessor[0xb17ac0] is creating.
20632 quadd_device_monitor DeviceMonitor[0xc05880] is connecting to ADB.
20636 quadd_common_communication Thread[20636] started servicing 2PerfModule@AsyncProcessor.
20637 quadd_common_communication Thread[20637] started servicing 2PerfModule@AsyncProcessor.
20638 quadd_common_communication Thread[20638] started servicing 2PerfModule@AsyncProcessor.
20639 quadd_common_communication Thread[20639] started servicing 2PerfModule@AsyncProcessor.
20626 quadd_common_communication 2PerfModule@AsyncProcessor[0xb17ac0] created.
20640 quadd_common_communication Thread[20640] started servicing 2PerfModule@AsyncProcessor.
20635 quadd_device_monitor DeviceMonitor[0xc05880] is sending track-devices request to ADB.
20632 quadd_device_monitor DeviceMonitor[0xc05880] connected to ADB.
20626 quadd_common_communication 3TraceModule@AsyncProcessor[0xbf7f20] is creating.
20641 quadd_common_communication Thread[20641] started servicing 3TraceModule@AsyncProcessor.
20642 quadd_common_communication Thread[20642] started servicing 3TraceModule@AsyncProcessor.
20635     quadd_device_adb Sending ADB query: 18 "host:track-devices"
20643 quadd_common_communication Thread[20643] started servicing 3TraceModule@AsyncProcessor.
20626 quadd_common_communication 3TraceModule@AsyncProcessor[0xbf7f20] created.
20644 quadd_common_communication Thread[20644] started servicing 3TraceModule@AsyncProcessor.
20635 quadd_device_monitor DeviceMonitor[0xc05880] sent track-devices request to ADB.
20634 quadd_device_monitor DeviceMonitor[0xc05880] is reading track-devices response from ADB.
20634 quadd_device_monitor DeviceMonitor[0xc05880]: track-devices response: 
20634 quadd_device_monitor DeviceMonitor[0xc05880] read track-devices response from ADB.
20634 quadd_device_monitor DeviceMonitor[0xc05880] is reading track-devices response from ADB.
20626 quadd_device_manager DeviceManager[0xc05700] is adding an observer[0xbf9760].
20632 quadd_device_manager DeviceManager[0xc05700] added the observer[0xbf9760].
20626            Interface Version: 3.7.224-e982b7b Linux
20626               AppLib Duplicate menu accelerator detected in command &Save Layout
20626              HostApp Merged manifest has no application version
20626              HostApp Merged manifest has no application id
20626              HostApp Merged manifest has no application docs
20626     Project Explorer Restored projects: 1, going to read from file immediately: 1.
20626               AppLib State 'QuadDPlugin.ShowProjectExplorer' has changed to 'true'
20626   GlobalStateMachine Entered MultiTargetMainState
20626   GlobalStateMachine Entered MainState
20626               AppLib State 'QuadDPlugin.NewDeviceButtonVisible' has changed to 'true'
20626                   Qt [ProjectExplorer] Saved 1 projects.
20649            Interface Project saved to file: /home/mac/.tegraprofiler/Projects/Project 1/Project 1.qdproj
20626     quadd_device_ssh SshDevice[0x13a5430]: constructed
20626 quadd_device_ssh_validator SshDeviceValidator[0x13a3230]: constructed, device=SSH-0x0xd70760
20635 quadd_device_validator New device validation status: Status: DeviceValidationStarted

20635 quadd_device_ssh_validator SshDeviceValidator[0x13a3230]: device=SSH-0x0xd70760 step=CheckAndConnect
20635 quadd_device_validator New device validation status: Status: ConnectingDevice

20635 quadd_common_communication 4SshClient@AsyncProcessor[0x7f3d3c0076f0] is creating.
20635 quadd_common_communication 4SshClient@AsyncProcessor[0x7f3d3c0076f0] created.
20651 quadd_common_communication Thread[20651] started servicing 4SshClient@AsyncProcessor.
20635 quadd_common_vsshclient Session[0x7f3d3c0075f0]: : ssh client session has been constructed
20626            Interface Status: DeviceValidationStarted (40), SSH-0x0xd70760.
20626            Interface Status: ConnectingDevice (30), SSH-0x0xd70760.
20652            Interface Project saved to file: /home/mac/.tegraprofiler/Projects/Project 1/Project 1.qdproj
20635 quadd_common_vsshclient Session[0x7f3d3c0075f0]: : ssh client session has been destroyed
20635 quadd_common_communication 4SshClient@AsyncProcessor[0x7f3d3c0076f0] is stopping.
20651 quadd_common_communication Thread[20651] stopped servicing 4SshClient@AsyncProcessor.
20635 quadd_common_communication 4SshClient@AsyncProcessor[0x7f3d3c0076f0] stopped.
20635 quadd_device_validator New device validation status: Status: DeviceConnectionError
Error {
  Type: AnalysisInternalError
  Props {
    Items {
      Type: OriginalExceptionClass
      Value: "N5boost16exception_detail10clone_implINS0_19error_info_injectorINS_6system12system_errorEEEEE"
    }
    Items {
      Type: OriginalFile
      Value: "/home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Common/Communication/Client/TcpClient.cpp"
    }
    Items {
      Type: OriginalLine
      Value
20635 quadd_device_validator New device validation status: Status: DeviceValidationFinished

20626            Interface Status: DeviceConnectionError (9), SSH-0x0xd70760.
20626            Interface Status: DeviceValidationFinished (41), SSH-0x0xd70760.
20626     quadd_device_ssh SshDevice[0x13a5430]: destroyed
10

cetatzeanum,

It looks like a known issue as

https://devtalk.nvidia.com/default/topic/1035560

I’ll help check it internally and report anything once we find.

11

Hello,
For the original issue (reported in the first post). Is the local user on Jetson TX2 board allowed to use sudo?

When System Profiler connects to the target board using username/password, it expects, that the user is allowed to execute sudo (using its login password) on target board.

12

Yes the user is in the sudoers list. I can execute commands using sudo.
The image on the TX2 is the one in the JetPack with the user nvidia. I have
not modified anything related to the users.

I reboot the board, restarted the profiler and now I am constantly getting this error:

KernelModuleServiceError (1406) {
    OriginalExceptionClass: N5boost16exception_detail10clone_implIN13QuadDAnalysis24KernelModuleServiceErrorEEE
    OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Host/Analysis/LinuxDevice.cpp
    OriginalLine: 259
    OriginalFunction: void QuadDAnalysis::LinuxDevice::InitializeKernelModule()
    ServiceError (1400) {
        OriginalExceptionClass: N5boost16exception_detail10clone_implIN13QuadDAnalysis12ServiceErrorEEE
        TargetSideError (1100) {
            MethodName: Initialize
            ServiceName: Service
            ErrorText: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/KernelModuleServiceImpl.cpp(168): Throw in function void QuadDDaemon::KernelModuleServiceImpl::InitializeInternal()
                Dynamic exception type: boost::exception_detail::clone_impl<QuadDDaemon::KmsModuleInitError>
                std::exception::what: KmsModuleInitError
                [boost::errinfo_nested_exception_*] = 
                  Throw in function (unknown)
                  Dynamic exception type: boost::exception_detail::clone_impl<QuadDDaemon::QMIOCTLError>
                  std::exception::what: QMIOCTLError
                  [QuadDCommon::tag_error_text*] = Permission denied
                  [QuadDDaemon::tag_error_code*] = 19
                  [QuadDDaemon::tag_throw_file*] = /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/qm.c
                  [QuadDDaemon::tag_throw_func*] = get_module_capabilities
                  13, "Permission denied"
                
            KmsModuleInitError (2004) {
                OriginalExceptionClass: N5boost16exception_detail10clone_implIN11QuadDDaemon18KmsModuleInitErrorEEE
                OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/KernelModuleServiceImpl.cpp
                OriginalLine: 168
                OriginalFunction: void QuadDDaemon::KernelModuleServiceImpl::InitializeInternal()
                QmIOCTLError (1802) {
                    OriginalExceptionClass: N5boost16exception_detail10clone_implIN11QuadDDaemon12QMIOCTLErrorEEE
                    OriginalLine: 260
                    Errno: 13
                    ErrorText: Permission denied
                    OriginalErrorCode: 19
                    OriginalFunction: get_module_capabilities
                    OriginalFile: /home/devtools/teamCityBuildAgent/work/20a3cfcd1c25021d/QuadD/Target/quadd_d/quadd_d/jni/qm.c
                }
            }
        }
    }
}

That might come from the fact that on the target I have the version 4.0 of the JetPack installed, whereas the profiler is the 3.7 version.

13

Thank you for the log file. We are looking at the issue. Could you try the profiler from the 4.0 version?

14

System Profiler v 4.0 log:

I15:23:10:480|CrashReporter|21781|:87[]: Server 8 client 7 for process

I15:23:10:523|HostApp|21785|Main.cpp:71[setupDpiScaling]: The interface scaling factor set to 1;1.
I15:23:10:641|AppLib_Script|21785|Manifest.js:3[<global>]: Loading CorePlugin
I15:23:10:642|AppLib_Script|21785|Manifest.js:5[<global>]: Loading QuadDPlugin
I15:23:10:643|quadd_common_core|21785|AsyncProcessorHolder.h:23[AsyncProcessorHolder]: AsyncProcessorHolder[0x229bb70]: 3 AsyncProcessors
I15:23:10:643|quadd_common_core|21785|AsyncProcessor.cpp:18[AsyncProcessor]: 1GlobalProcessor@AsyncProcessor[0x22794a0] is creating.
I15:23:10:643|quadd_common_core|21791|AsyncProcessor.cpp:34[operator()]: Thread[21791] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:643|quadd_common_core|21792|AsyncProcessor.cpp:34[operator()]: Thread[21792] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:643|quadd_common_core|21793|AsyncProcessor.cpp:34[operator()]: Thread[21793] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:643|quadd_common_core|21794|AsyncProcessor.cpp:34[operator()]: Thread[21794] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:643|quadd_common_core|21795|AsyncProcessor.cpp:34[operator()]: Thread[21795] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:643|quadd_common_core|21796|AsyncProcessor.cpp:34[operator()]: Thread[21796] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:644|quadd_common_core|21785|AsyncProcessor.cpp:63[AsyncProcessor]: 1GlobalProcessor@AsyncProcessor[0x22794a0] created.
I15:23:10:644|quadd_common_core|21785|AsyncProcessor.cpp:18[AsyncProcessor]: 2DeviceManager@AsyncProcessor[0x2270860] is creating.
I15:23:10:644|quadd_common_core|21798|AsyncProcessor.cpp:34[operator()]: Thread[21798] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:644|quadd_common_core|21797|AsyncProcessor.cpp:34[operator()]: Thread[21797] started servicing 1GlobalProcessor@AsyncProcessor.
I15:23:10:644|quadd_common_core|21800|AsyncProcessor.cpp:34[operator()]: Thread[21800] started servicing 2DeviceManager@AsyncProcessor.
I15:23:10:644|quadd_common_core|21785|AsyncProcessor.cpp:63[AsyncProcessor]: 2DeviceManager@AsyncProcessor[0x2270860] created.
I15:23:10:644|quadd_common_core|21803|AsyncProcessor.cpp:34[operator()]: Thread[21803] started servicing 2DeviceManager@AsyncProcessor.
I15:23:10:644|quadd_common_core|21801|AsyncProcessor.cpp:34[operator()]: Thread[21801] started servicing 2DeviceManager@AsyncProcessor.
I15:23:10:644|quadd_common_core|21785|AsyncProcessor.cpp:18[AsyncProcessor]: 3SshClient@AsyncProcessor[0x2279880] is creating.
I15:23:10:644|quadd_common_core|21802|AsyncProcessor.cpp:34[operator()]: Thread[21802] started servicing 2DeviceManager@AsyncProcessor.
I15:23:10:644|quadd_common_core|21804|AsyncProcessor.cpp:34[operator()]: Thread[21804] started servicing 3SshClient@AsyncProcessor.
I15:23:10:644|quadd_common_core|21785|AsyncProcessor.cpp:63[AsyncProcessor]: 3SshClient@AsyncProcessor[0x2279880] created.
I15:23:10:644|quadd_common_core|21799|AsyncProcessor.cpp:34[operator()]: Thread[21799] started servicing 2DeviceManager@AsyncProcessor.
I15:23:10:644|quadd_device_manager|21785|DeviceManager.cpp:148[DeviceManager]: DeviceManager[0x228d5a0] created.
I15:23:10:644|quadd_device_manager|21785|DeviceManager.cpp:153[Start]: DeviceManager[0x228d5a0] is starting.
I15:23:10:644|quadd_device_monitor|21785|DeviceMonitor.cpp:80[DeviceMonitor]: DeviceMonitor[0x228d7e0] is creating.
I15:23:10:645|quadd_device_monitor|21785|DeviceMonitor.cpp:91[DeviceMonitor]: DeviceMonitor[0x228d7e0]: WorkaroundAdbLastDeviceBug=no.
I15:23:10:645|quadd_device_monitor|21785|DeviceMonitor.cpp:93[DeviceMonitor]: DeviceMonitor[0x228d7e0] created.
I15:23:10:645|quadd_device_monitor|21785|DeviceMonitor.cpp:115[Start]: DeviceMonitor[0x228d7e0] is starting.
I15:23:10:645|quadd_device_monitor|21785|DeviceMonitor.cpp:146[Start]: DeviceMonitor[0x228d7e0] started.
I15:23:10:645|quadd_device_manager|21785|DeviceManager.cpp:157[Start]: DeviceManager[0x228d5a0] started.
I15:23:10:645|quadd_device_monitor|21803|DeviceMonitor.cpp:261[ConnectToAdb]: DeviceMonitor[0x228d7e0, 0] is connecting to ADB.
I15:23:10:645|quadd_device_manager|21785|DeviceManager.cpp:718[AddObserver]: DeviceManager[0x228d5a0] is adding an observer[0x227f790].
I15:23:10:645|quadd_device_manager|21801|DeviceManager.cpp:776[operator()]: DeviceManager[0x228d5a0] added the observer[0x227f790].
I15:23:10:645|quadd_device_monitor|21803|DeviceMonitor.cpp:282[ConnectToAdb]: DeviceMonitor[0x228d7e0, 0] connected to ADB.
I15:23:10:645|quadd_device_monitor|21803|DeviceMonitor.cpp:338[SendTrackDevicesQuery]: DeviceMonitor[0x228d7e0, 0] is sending track-devices request to ADB.
I15:23:10:645|quadd_device_adb|21803|AdbClient.cpp:280[SendQuery]: Sending ADB query: 18 "host:track-devices"
I15:23:10:645|quadd_device_monitor|21803|DeviceMonitor.cpp:351[SendTrackDevicesQuery]: DeviceMonitor[0x228d7e0, 0] sent track-devices request to ADB.
I15:23:10:645|quadd_device_monitor|21802|DeviceMonitor.cpp:394[ParseTrackDevicesResponse]: DeviceMonitor[0x228d7e0, 0] is reading track-devices response from ADB.
I15:23:10:645|quadd_device_monitor|21802|DeviceMonitor.cpp:399[ParseTrackDevicesResponse]: DeviceMonitor[0x228d7e0, 0]: track-devices response: 
I15:23:10:645|quadd_device_monitor|21802|DeviceMonitor.cpp:433[UpdateDevicesList]: DeviceMonitor[0x228d7e0, 0] read track-devices response from ADB.
I15:23:10:646|TelemetryService|21785|TelemetryService.cpp:80[TelemetryService]: Reporting of telemetry is enabled. 
I15:23:10:646|TelemetryService|21785|TelemetryService.cpp:83[TelemetryService]: Tracking ID: "UA-59254929-18".
I15:23:10:646|TelemetryService|21785|TelemetryService.cpp:86[TelemetryService]: Host application: "SystemProfiler".
I15:23:10:646|TelemetryService|21785|TelemetryService.cpp:89[TelemetryService]: Host application version: 4.0.387-eecbec0
I15:23:10:646|TelemetryService|21785|TelemetryService.cpp:92[TelemetryService]: Default role: "external".
I15:23:10:646|Interface|21785|QuadDService.cpp:78[QuadDService]: Version: 4.0.387-eecbec0 Linux
E15:23:10:648|AppLib|21785|:318[]: Duplicate menu accelerator detected in command &Save Layout
F15:23:10:745|HostApp|21785|Main.cpp:170[main]: Merged manifest has no application version
E15:23:10:745|HostApp|21785|Main.cpp:171[main]: Merged manifest has no application id
E15:23:10:745|HostApp|21785|Main.cpp:172[main]: Merged manifest has no application docs
I15:23:10:789|Project Explorer|21785|ProjectExplorer.cpp:102[restoreOpenedProjects]: Restored projects: 1, going to read from file immediately: 1.
I15:23:10:814|GlobalStateMachine|21785|StateMachine.cpp:201[onEntry]: Entered MultiTargetMainState
I15:23:10:814|GlobalStateMachine|21785|StateMachine.cpp:107[onEntry]: Entered MainState
W15:23:10:850|Qt|21785|:78[]: QLayout: Attempting to add QLayout "" to QuadDUI::FileSelectionWidget "", which already has a layout
W15:23:10:850|Qt|21785|:78[]: QLayout: Attempting to add QLayout "" to QuadDUI::FileSelectionWidget "", which already has a layout
W15:23:10:910|Qt|21785|:78[]: QLayout: Attempting to add QLayout "" to QuadDUI::FileSelectionWidget "", which already has a layout
W15:23:10:910|Qt|21785|:78[]: QLayout: Attempting to add QLayout "" to QuadDUI::FileSelectionWidget "", which already has a layout
I15:23:11:868|Interface|21809|Project.cpp:918[saveProjectImpl]: Project saved to file: /home/mac/.systemprofiler/Projects/Project 1/Project 1.qdproj
I15:23:21:982|quadd_device_ssh|21785|SshDevice.cpp:441[SshDevice]: SshDevice[0x26bb6c0]: constructed
I15:23:21:982|quadd_device_ssh_validator|21785|SshDeviceValidatorSelector.cpp:14[SshDeviceValidatorSelector]: SshDeviceValidatorSelector[0x26b9dd0]: constructed, device=SSH-0x0x23d5330
I15:23:21:985|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: DeviceValidationStarted

I15:23:21:985|quadd_device_ssh_validator|21799|SshDeviceValidatorSelector.cpp:55[CheckAndValidate]: CheckAndValidate[0x26b9dd0]: device=SSH-0x0x23d5330 step=CheckAndConnect
I15:23:21:985|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: ConnectingDevice

I15:23:21:985|quadd_common_vsshclient|21799|Session.cpp:95[Session]: Session[0x7febac004940]: : ssh client session has been constructed
I15:23:21:987|quadd_common_vsshclient|21799|Session.cpp:107[InitializeSshSession]: Session[0x7febac004940]: : initializing ssh session
I15:23:21:987|quadd_common_vsshclient|21799|Session.cpp:110[InitializeSshSession]: Session[0x7febac004940]: : handshaking
I15:23:21:989|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: DeviceValidationStarted (40), SSH-0x0x23d5330.
I15:23:21:989|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: ConnectingDevice (30), SSH-0x0x23d5330.
I15:23:22:093|quadd_common_vsshclient|21799|Session.cpp:115[InitializeSshSession]: Session[0x7febac004940]: : authenticating
I15:23:22:136|quadd_common_vsshclient|21799|Session.cpp:118[InitializeSshSession]: Session[0x7febac004940]: : switching to non-blocking mode
I15:23:22:137|quadd_device_ssh|21799|SshDevice.cpp:597[QueryShell]: Querying shell command: "[ -x /usr/lib/ldqnx-64.so.2 ] && echo $?"
I15:23:22:137|quadd_common_vsshclient|21799|Command.cpp:97[ExecuteInternal]: Command[0x7febe77fd620]: Executing command=[ -x /usr/lib/ldqnx-64.so.2 ] && echo $?
I15:23:22:137|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:137|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fd4a0
I15:23:22:137|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:137|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fd4a0
I15:23:22:386|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fd4a0
I15:23:22:428|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fd4a0
I15:23:22:428|quadd_common_vsshclient|21804|SshChannelHandle.cpp:15[SshChannelHandle]: SshChannelHandle[0x7febb8000c60]: ssh channel initialized, session handle=0x7febac000aa0, channel handle=0x7febb8000960
I15:23:22:428|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:428|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:428|quadd_common_sshclient|21804|Session.cpp:307[ExecuteHandler]: Session[0x7febac004940]: : executing command, ChannelInfo=0x7febac01f878, command=[ -x /usr/lib/ldqnx-64.so.2 ] && echo $?
I15:23:22:447|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:493[SendEofHandler]: Session[0x7febac004940]: : sending EOF, ChannelInfo=0x7febac01f878
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac01f878
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:448|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac01f878
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac01f878
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:562[CloseHandler]: Session[0x7febac004940]: : closing channel, ChannelInfo=0x7febac01f878
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:406[ReadErrorHandler]: Session[0x7febac004940]: : reading error output , ChannelInfo=0x7febac01f878
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:363[ReadHandler]: Session[0x7febac004940]: : reading from the channel, ChannelInfo=0x7febac01f878
I15:23:22:463|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:958[HasMoreDataHandler]: Session[0x7febac004940]: : checking has more data, ChannelInfo=0x7febac01f878
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:921[WaitClosedHandler]: Session[0x7febac004940]: : waiting closed, ChannelInfo=0x7febac01f878
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:609[operator()]: Session[0x7febac004940]: : getting exit code, ChannelInfo=0x7febac01f878
I15:23:22:464|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:464|quadd_common_vsshclient|21799|SshChannelHandle.cpp:26[~SshChannelHandle]: SshChannelHandle[0x7febb8000c60]: ssh channel destroyed
E15:23:22:464|quadd_device_ssh|21799|SshDevice.cpp:606[QueryShell]: Shell query failed. Exit code: 1. Stdout: ""
I15:23:22:464|quadd_device_ssh_validator|21799|SshDeviceValidator.cpp:23[SshDeviceValidator]: SshDeviceValidator[0x7febac01f850]: constructed, device=SSH-0x0x23d5330
I15:23:22:464|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: DeviceValidationStarted

I15:23:22:464|quadd_device_ssh_validator|21799|SshDeviceValidator.cpp:98[CheckConnection]: SshDeviceValidator[0x7febac01f850]: device=SSH-0x0x23d5330 step=CheckConnection
I15:23:22:464|quadd_device_posix_validator|21799|PosixDeviceValidator.cpp:15[CheckAndUpdateProperties]: PosixDeviceValidator[0x7febac01f850]: device=SSH-0x0x23d5330 step=CheckAndUpdateProperties
I15:23:22:464|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: DeviceValidationStarted (40), SSH-0x0x23d5330.
I15:23:22:464|quadd_device_ssh|21799|SshDevice.cpp:630[QueryShellAsSu]: Querying elevated shell command: "echo OK"
I15:23:22:465|quadd_common_vsshclient|21799|Sudo.cpp:36[Execute]: Sudo[0x7febac01ed40]: Executing command=echo OK
I15:23:22:465|quadd_common_vsshclient|21799|Command.cpp:97[ExecuteInternal]: Command[0x7febe77fc620]: Executing command=which sudo
I15:23:22:465|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:466|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:466|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:466|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:502|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:502|quadd_common_vsshclient|21804|SshChannelHandle.cpp:15[SshChannelHandle]: SshChannelHandle[0x7febb8000900]: ssh channel initialized, session handle=0x7febac000aa0, channel handle=0x7febb8000960
I15:23:22:502|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:503|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:503|quadd_common_sshclient|21804|Session.cpp:307[ExecuteHandler]: Session[0x7febac004940]: : executing command, ChannelInfo=0x7febac020ed8, command=which sudo
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:493[SendEofHandler]: Session[0x7febac004940]: : sending EOF, ChannelInfo=0x7febac020ed8
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:520|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:22:537|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:22:537|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:22:537|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:537|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:562[CloseHandler]: Session[0x7febac004940]: : closing channel, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:406[ReadErrorHandler]: Session[0x7febac004940]: : reading error output , ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:363[ReadHandler]: Session[0x7febac004940]: : reading from the channel, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:363[ReadHandler]: Session[0x7febac004940]: : reading from the channel, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:958[HasMoreDataHandler]: Session[0x7febac004940]: : checking has more data, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:921[WaitClosedHandler]: Session[0x7febac004940]: : waiting closed, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:609[operator()]: Session[0x7febac004940]: : getting exit code, ChannelInfo=0x7febac020ed8
I15:23:22:538|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:538|quadd_common_vsshclient|21799|SshChannelHandle.cpp:26[~SshChannelHandle]: SshChannelHandle[0x7febb8000900]: ssh channel destroyed
I15:23:22:539|quadd_common_vsshclient|21799|Command.cpp:97[ExecuteInternal]: Command[0x7febe77fc620]: Executing command=sudo -kS true
I15:23:22:539|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:539|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:539|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:539|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:577|quadd_common_vsshclient|21804|Session.cpp:35[OpenChannelHandler]: Session[0x7febac004940]: : opening new channel, ChannelInfo=0x7febe77fc4a0
I15:23:22:577|quadd_common_vsshclient|21804|SshChannelHandle.cpp:15[SshChannelHandle]: SshChannelHandle[0x7febb8000c60]: ssh channel initialized, session handle=0x7febac000aa0, channel handle=0x7febb8000960
I15:23:22:577|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:577|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:577|quadd_common_sshclient|21804|Session.cpp:307[ExecuteHandler]: Session[0x7febac004940]: : executing command, ChannelInfo=0x7febac020ed8, command=sudo -kS true
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:493[SendEofHandler]: Session[0x7febac004940]: : sending EOF, ChannelInfo=0x7febac020ed8
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:257[ExecuteRequests]: Session[0x7febac004940]: : There are no more outstanding requests.
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:211[operator()]: Session[0x7febac004940]: : Request queue size=1
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:264[ExecuteRequests]: Session[0x7febac004940]: : New request has been added.
I15:23:22:581|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:24:583|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:24:583|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:26:585|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:26:585|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:28:587|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:28:587|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:30:589|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:30:589|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:32:591|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:32:591|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:34:593|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:34:593|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:36:595|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:36:595|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:38:598|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:38:598|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:40:599|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:40:599|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:42:601|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:42:601|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:44:603|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:44:603|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:46:605|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:46:605|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:48:606|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:48:606|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
I15:23:50:607|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:50:607|quadd_common_vsshclient|21804|Session.cpp:516[WaitEofHandler]: Session[0x7febac004940]: : waiting EOF, ChannelInfo=0x7febac020ed8
E15:23:52:581|quadd_common_sshclient|21799|Session.cpp:1190[WaitNotification]: Wait() is probably hanged.
E15:23:52:581|quadd_common_sshclient|21799|Command.cpp:134[CloseChannelAndReadOutput]: Command[0x7febe77fc620]: Running command failed. Reason=/build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Session.cpp(1196): Throw in function void QuadDSshClient::Session::ChannelInfo::WaitNotification(QuadDSshClient::Session*)
Dynamic exception type: boost::exception_detail::clone_impl<QuadDCommon::TimeoutException>
std::exception::what: TimeoutException
[QuadDCommon::tag_error_text*] = Operation timeout.
.
E15:23:52:582|quadd_common_sshclient|21799|ReadWriteChannel.cpp:21[~ReadWriteChannel]: ReadWriteChannel[0x7febac020e70]: /build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Session.cpp(292): Throw in function QuadDSshClient::Session::ChannelInfo& QuadDSshClient::Session::FindChannelInfoAndMakeBusy(QuadDSshClient::Session::ChannelIdType)
Dynamic exception type: boost::exception_detail::clone_impl<QuadDCommon::InvalidArgumentException>
std::exception::what: InvalidArgumentException
[QuadDCommon::tag_error_text*] = Channel is busy.

W15:23:52:582|quadd_common_sshclient|21799|SshChannelHandle.cpp:23[~SshChannelHandle]: SshChannelHandle[0x7febb8000c60]: channel hasn't been closed properly
I15:23:52:582|quadd_common_vsshclient|21799|SshChannelHandle.cpp:26[~SshChannelHandle]: SshChannelHandle[0x7febb8000c60]: ssh channel destroyed
E15:23:52:582|quadd_common_sshclient|21799|SshChannelHandle.cpp:223[FreeChannelHandle]: FreeChannelHandle: libssh2_channel_free(0x7febb8000960) failed and suppressed error=-37
E15:23:52:582|quadd_common_sshclient|21799|Sudo.cpp:70[Execute]: Sudo[0x7febac01ed40]: Running command=echo OK failed. Reason=/build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Command.cpp(135): Throw in function void QuadDSshClient::Command::CloseChannelAndReadOutput(const IChannelPtr&)
Dynamic exception type: boost::exception_detail::clone_impl<QuadDSshClient::SshExecuteCommandError>
std::exception::what: SshExecuteCommandError
[boost::errinfo_nested_exception_*] = 
  /build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Session.cpp(1196): Throw in function void QuadDSshClient::Session::ChannelInfo::WaitNotification(QuadDSshClient::Session*)
  Dynamic exception type: boost::exception_detail::clone_impl<QuadDCommon::TimeoutException>
  std::exception::what: TimeoutException
  [QuadDCommon::tag_error_text*] = Operation timeout.

.
I15:23:52:583|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: DeviceConnectionError
Error {
  Type: SshExecuteCommandError
  Props {
    Items {
      Type: OriginalExceptionClass
      Value: "N5boost16exception_detail10clone_implIN14QuadDSshClient19SshExecuteSudoErrorEEE"
    }
    Items {
      Type: OriginalFile
      Value: "/build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Sudo.cpp"
    }
    Items {
      Type: OriginalLine
      Value: "71"
    }
    Items {
      Type: OriginalFunction
      Value: "virtual int QuadDSshClient::Sudo::Execute(const string&)"
    }
    Items {
      Type: SshCommand
      Value: "echo OK"
    }
  }
  SubError {
    Type: SshExecuteCommandError
    Props {
      Items {
        Type: OriginalExceptionClass
        Value: "N5boost16exception_detail10clone_implIN14QuadDSshClient22SshExecuteCommandErrorEEE"
      }
      Items {
        Type: OriginalFile
        Value: "/build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Command.cpp"
      }
      Items {
        Type: OriginalLine
        Value: "135"
      }
      Items {
        Type: OriginalFunction
        Value: "void QuadDSshClient::Command::CloseChannelAndReadOutput(const IChannelPtr&)"
      }
    }
    SubError {
      Type: TimeoutError
      Props {
        Items {
          Type: OriginalExceptionClass
          Value: "N5boost16exception_detail10clone_implIN11QuadDCommon16TimeoutExceptionEEE"
        }
        Items {
          Type: OriginalFile
          Value: "/build/agent/work/20a3cfcd1c25021d/QuadD/Common/SshClient/Session.cpp"
        }
        Items {
          Type: OriginalLine
          Value: "1196"
        }
        Items {
          Type: OriginalFunction
          Value: "void QuadDSshClient::Session::ChannelInfo::WaitNotification(QuadDSshClient::Session*)"
        }
        Items {
          Type: ErrorText
          Value: "Operation timeout."
        }
      }
    }
  }
}

I15:23:52:584|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: DeviceValidationFinished

I15:23:52:584|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: DeviceConnectionError (9), SSH-0x0x23d5330.
I15:23:52:584|quadd_device_validator|21799|IDeviceValidatorImpl.cpp:122[MakeStatus]: New device validation status: Status: DeviceValidationFinished

I15:23:52:586|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: DeviceValidationFinished (41), SSH-0x0x23d5330.
I15:23:52:586|quadd_device_ssh|21785|SshDevice.cpp:446[~SshDevice]: SshDevice[0x26bb6c0]: destroyed
I15:23:52:587|Interface|21785|DeviceList.cpp:738[onDeviceValidated]: Status: DeviceValidationFinished (41), SSH-0x0x23d5330.
I15:23:52:607|quadd_common_sshclient|21804|Session.cpp:188[WaitSessionSocket]: Session[0x7febac004940]: : select() returned due to timeout.
I15:23:58:877|GlobalStateMachine|21785|StateMachine.cpp:225[onExit]: Exited MultiTargetMainState
I15:23:58:877|GlobalStateMachine|21785|StateMachine.cpp:127[onExit]: Exited MainState
E15:23:58:943|AppLib|21785|:1478[]: Failed to get the host window service.
E15:23:58:943|AppLib|21785|:1211[]: Failed to get setting key for dock container.
E15:23:58:946|AppLib|21785|:1478[]: Failed to get the host window service.
E15:23:58:946|AppLib|21785|:1211[]: Failed to get setting key for dock container.
I15:23:58:964|quadd_device_manager|21785|DeviceManager.cpp:732[RemoveObserver]: DeviceManager[0x228d5a0] is removing an observer[0x227f790].
I15:23:58:964|quadd_device_manager|21785|DeviceManager.cpp:739[RemoveObserver]: DeviceManager[0x228d5a0] removed the observer[0x227f790].
I15:23:58:964|quadd_device_manager|21785|DeviceManager.cpp:162[Terminate]: DeviceManager[0x228d5a0] is stopping.
I15:23:58:964|quadd_device_monitor|21785|DeviceMonitor.cpp:98[Terminate]: DeviceMonitor[0x228d7e0] is terminating.
I15:23:58:964|quadd_device_manager|21785|DeviceManager.cpp:201[Terminate]: DeviceManager[0x228d5a0] stopped.
I15:23:58:964|quadd_common_core|21785|AsyncProcessorHolder.h:56[Terminate]: AsyncProcessorHolder[0x229bb70]: Stopping
I15:23:58:964|quadd_common_core|21785|AsyncProcessor.cpp:75[Stop]: 1GlobalProcessor@AsyncProcessor[0x22794a0] is stopping.
I15:23:58:964|quadd_common_core|21785|AsyncProcessor.cpp:89[Stop]: 1GlobalProcessor@AsyncProcessor[0x22794a0] is waiting for work threads finish.
I15:23:58:964|quadd_common_core|21796|AsyncProcessor.cpp:58[operator()]: Thread[21796] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:964|quadd_common_core|21795|AsyncProcessor.cpp:58[operator()]: Thread[21795] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:964|quadd_common_core|21791|AsyncProcessor.cpp:58[operator()]: Thread[21791] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:964|quadd_common_core|21794|AsyncProcessor.cpp:58[operator()]: Thread[21794] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:964|quadd_common_core|21793|AsyncProcessor.cpp:58[operator()]: Thread[21793] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:964|quadd_common_core|21797|AsyncProcessor.cpp:58[operator()]: Thread[21797] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:965|quadd_common_core|21792|AsyncProcessor.cpp:58[operator()]: Thread[21792] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:965|quadd_common_core|21798|AsyncProcessor.cpp:58[operator()]: Thread[21798] stopped servicing 1GlobalProcessor@AsyncProcessor.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:97[Stop]: 1GlobalProcessor@AsyncProcessor[0x22794a0] stopped.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:75[Stop]: 2DeviceManager@AsyncProcessor[0x2270860] is stopping.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:89[Stop]: 2DeviceManager@AsyncProcessor[0x2270860] is waiting for work threads finish.
I15:23:58:965|quadd_common_core|21799|AsyncProcessor.cpp:58[operator()]: Thread[21799] stopped servicing 2DeviceManager@AsyncProcessor.
I15:23:58:965|quadd_common_core|21801|AsyncProcessor.cpp:58[operator()]: Thread[21801] stopped servicing 2DeviceManager@AsyncProcessor.
I15:23:58:965|quadd_common_core|21803|AsyncProcessor.cpp:58[operator()]: Thread[21803] stopped servicing 2DeviceManager@AsyncProcessor.
I15:23:58:965|quadd_common_core|21800|AsyncProcessor.cpp:58[operator()]: Thread[21800] stopped servicing 2DeviceManager@AsyncProcessor.
I15:23:58:965|quadd_common_core|21802|AsyncProcessor.cpp:58[operator()]: Thread[21802] stopped servicing 2DeviceManager@AsyncProcessor.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:97[Stop]: 2DeviceManager@AsyncProcessor[0x2270860] stopped.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:75[Stop]: 3SshClient@AsyncProcessor[0x2279880] is stopping.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:89[Stop]: 3SshClient@AsyncProcessor[0x2279880] is waiting for work threads finish.
I15:23:58:965|quadd_common_core|21804|AsyncProcessor.cpp:58[operator()]: Thread[21804] stopped servicing 3SshClient@AsyncProcessor.
I15:23:58:965|quadd_common_core|21785|AsyncProcessor.cpp:97[Stop]: 3SshClient@AsyncProcessor[0x2279880] stopped.
I15:23:58:965|quadd_common_core|21785|AsyncProcessorHolder.h:72[Terminate]: AsyncProcessorHolder[0x229bb70]: Destroying

I also (tried to) attached the version 3.7 latest log file.
TegraSystemProfiler.log (401 KB)
profiler_log.gz (15 KB)

15

Any news on this one? The silence is not encouraging!
Do you need any other info from my part?

16

Hi cetatzeanum,

I was not able to reproduce this issue locally.
Could you please add more logs.

Please put nvlog.config file in the target home directory, example of nvlog.config:

$> /tmp/target_nvlog.log

  • 50iw 50ef 0IW 0EF global
  • quadd_vdaemon_c_dwarf
  • quadd_vdaemon_c_ehunwind

ForceFlush
Format [${file:0}:${sfunc:0}:${line:0}]: $text

and restart the device.

  1. Reproduce the problem.

  2. Attach the following target logs/info:
    ll /proc/quadd/
    ll /dev/quadd*
    cat /proc/quadd/*
    dmesg
    /tmp/target_nvlog.log

and host System Profiler log file.

Thanks,
Igor.

17
  1. I already did that. Did you see the previous logs?
  2. I can reproduce it at every run.
  3. Files are attached. I do not have any /tmp/target_nvlog.log

All the logs are from the System Profiler v4.

Thanks
Marius
SystemProfiler.log (37.7 KB)
nvidia_tx2_commands.log (4.53 KB)
nvidia_tx2_dmesg.log (70.6 KB)

18

There are two nvlog.config files - one for the host, another for the target (they affect two log files: on the host and on the target separately).

>>> I do not have any /tmp/target_nvlog.log
This log file should be mentioned in the target nvlog.config (please see example above).

Target nvlog.config file should be put in your target home directory (for example: /home/nvidia/nvlog.config)
Your device must be restarted for nvlog.config changes to take effect.

>>> nvidia_tx2_dmesg.log
Please send dmesg output (and target_nvlog.log) after failed profiling session (kernel part of profiler outputs some useful information to dmesg during profiling).

19

Hello,
We are still working on these issues:

  • https://devtalk.nvidia.com/default/topic/1035855/jetson-tx2/nvidia-system-profiler-4-0-error-with-jetson-tx2/post/5262707/#5262707
  • https://devtalk.nvidia.com/default/topic/1035855/jetson-tx2/nvidia-system-profiler-4-0-error-with-jetson-tx2/post/5262718/#5262718

but we are still unable to reproduce these on our side. I will update this thread as soon as we get something.

20

I haven’t looked at the ssh logs, but some hosts now disallow older protocols, and at least older nsight has the protocols embedded such that it can use only those older protocols. If this version of nsight is still using the older ssh protocols (I have not checked what has changed), then mixing of older and newer ssh will cause a failure. This could be completely unrelated, but thought I’d mention it:
[url]https://devtalk.nvidia.com/default/topic/988099/nsight-eclipse-edition/is-ssh-private-key-remote-access-broken-in-cuda-8-0-nsight-/[/url]