Random Camera Loss After Docker Restart in VST v1.2.52/v1.2.58 with 8 Cameras (H265)

[rafikimielec]

I’m currently working with VST v1.2.52 and v1.2.58, and I’m encountering an issue managing multiple cameras.

Here’s an overview of the setup and the problems I’ve encountered:

Configuration Overview:

I have a setup of up to 8 cameras (h265, 30fps) running with the default configuration. The system initially works fine when setting up the cameras, but after restarting Docker Compose, the cameras are randomly lost. Some cameras do not reconnect after a restart. The problem occurs on the AGX device; no matter how many cameras up to 8 I set, the problem exists.

The main issue appears in the logs as follows:

vst    | 0
vst    | [2024-9-13T15:6:46.573Z:281472955953184:sensor_management.cpp:1095: setStreamDefaultSettings]   
vst    | [2024-9-13T15:6:46.574Z:281472955953184:sensor_monitoring.cpp:223: notifyEvent]   Notification Manager instance is not created
vst    | [2024-9-13T15:6:46.574Z:281472955953184:sensor_management.cpp:492: getAndAddProxyUrl] Device is error : 102

Additionally, I see the following when the system attempts to reset the camera settings:

[getAndAddProxyUrl] [2024-9-13T15:12:47.568Z:281472834654240:sensor_management_utils.cpp:1439: setStreamDefaultSettings] Error setting camera default to encode settings for 0

Questions:

1. Random Camera Loss After Restart:

• Is any known issue or additional configuration needed to prevent cameras from being lost after restarting Docker Compose?
• Do I need to adjust specific steps or settings to ensure the cameras consistently reconnect, even after restart?

Any advice or guidance on troubleshooting this issue or optimizing VST for production would be greatly appreciated.

[kesong]

VST is foundation-service on JPS release:Jetson Platform Services — Metropolis on Jetson documentation 0.1.0 documentation. Are you restart VST services by: sudo systemctl restart jetson-vst? How you find the camera is loss?

[rafikimielec]

Thank you for your response.

We only use the VST service due to its recording, API, and GUI capabilities. Our setup consists of a single Docker container running VST with Docker Compose using the following configuration:

vst:
  image: nvcr.io/<>/release/vst:v1.2.52_aarch64
  user: "0:0"
  network_mode: "host"
  runtime: nvidia
  logging:
    driver: "json-file"
    options:
      max-size: "8192m"
      max-file: "3"
  container_name: vst
  entrypoint: sh -c '/root/vst_release/launch_vst --debug-level 3 2>&1 | tee -a /root/vst_release/webroot/log/vst.log'
  volumes:
    - ./config/vst/vst_config.json:/root/vst_release/configs/vst_config.json
    - ./config/vst/vst_storage.json:/root/vst_release/configs/vst_storage.json
    - /data/vst-volume:/root/store
    - /data/logging-volume:/root/vst_release/webroot/log
    - /usr/lib/aarch64-linux-gnu/nvidia/:/usr/lib/aarch64-linux-gnu/tegra/

The issue occurs inside the GUI after restarting Docker Compose—some cameras are lost, and I see the error messages mentioned in my previous post. However, everything works fine if I remove the error cameras and add them back manually.

The primary concern is the stability after restarting.

I’ll also include a screenshot of the GUI after the restart and verify the camera stream using the ffmpeg command:
ffmpeg -i <camera-rtsp> -f null /dev/null

Lastly, I will include another screenshot showing the cameras working correctly after removing and re-adding them.

[rafikimielec]

@kesong

sudo systemctl restart jetson-vst

After testing with the default VST service settings, we confirmed that the cameras do not restart correctly, throwing errors. Upon reviewing the configuration, we found that autodiscovery in the ONVIF section was turned off:

"onvif": {
   "device_discovery_timeout_secs": -1, # here
    "onvif_request_timeout_secs": 20,
    "device_discovery_freq_secs": -1, # here
    "device_discovery_interfaces": ["eth1"],
    "max_devices_supported": 8,
    "default_bitrate_kbps": 8000,
    "default_framerate": 30,
    "default_resolution": "1920x1080",
    "default_gov_length": 60
}

When we enabled autodiscovery again, the service worked as expected. However, the issue now is that when autodiscovery is on, it also detects cameras that we do not want to add.

We tried disabling the feature in adoptor.json by turning off all related settings, but it didn’t stop the autodiscovery.

Question: How can we disable the autodiscovery feature entirely for any VST camera?

[rafikimielec]

I noticed in the logs that VST has an auto-discovery feature for cameras, as indicated by the message:

Is camera auto discovery enabled ?: 1

However, I can’t find any clear documentation on how to enable or disable this feature. Could you point me to the correct configuration setting or command to toggle the auto-discovery on and off?

I really appreciate any help you can provide.

[kesong]

Can you refer here to disable onvif auto discovery: How to disable "sensor auto scan" on VST - #4 by kesong?

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks