Unexpected exception during Live Stream Summarization in VSS UI

Accelerated Computing Intelligent Video Analytics Visual AI Agent
1

Setup:

Steps to Reproduce:

  1. Open the VSS UI and navigate to Live Stream Summarization.

  2. Modify the prompts to reflect expected events in the video stream.

  3. Create an alert for the event you want to detect.

  4. Click Start Streaming & Summarization.

  5. Observe the via-server logs and monitor for created alerts.

Actual Outcome:
After approximately 20 minutes of stream processing, the following exception is sometimes raised:

2025-10-21 08:52:23,887 ERROR Error in acreate_graph: 'list' object has no attribute 'get'
Traceback (most recent call last):
  File ".../ingestion/base.py", line 343, in acreate_graph
    await self._extract_entities_with_fallback(
  ...
  File ".../llm.py", line 957, in aprocess_response
    not rel.get("head")
AttributeError: 'list' object has no attribute 'get'

No new alerts are created for a period following this error.

Expected Outcome:

  • No unexpected exceptions should be logged.

  • The application should continuously create new events during stream processing.

Additional Notes:

  • The issue does not occur immediately but is reproducible with stream durations of ~20 minutes.

  • The same exception has also been observed during regular video processing, not just live streams.

3

This problem seems to be caused by the dependency package langchain-experimental of context-aware-rag. Can you share the video file and prompts and parameters of via_client_cli.py that reproduce the problem?

The type error on line 957 caused this problem

4

Please find attached the logs reproducing the issue. This time, I created an alert, although I’m not fully convinced it’s necessary.

For context, I was using Releases · bluenviron/mediamtx · GitHub as the RTSP service and FFmpeg to stream a looped video using the following command:

Shell

ffmpeg -loglevel debug -stream_loop -1 -re -i abandon_01_extended.mp4 \

-c copy -fflags +genpts -max_delay 5000000 \

-rtsp_transport tcp -buffer_size 512k \

-f rtsp rtsp://localhost:8554/mystream

Here are the prompts I used:

  • Caption:
    “Write a concise dense caption describing this video. CRITICAL PRIORITY – identify weapon handling, violence, assault, medical emergencies. SECONDARY – item concealment, barcode tampering, fake scanning, unscanned items, abandoned items or packages. Include scanning activity context (person moving items left to right).”

  • Caption Summarization:
    “You will be given captions from sequential clips of a video. Aggregate captions in the format start_time:end_time:caption based on whether captions are related to one another or create a continuous scene. Focus on identifying and highlighting any irregular, suspicious, or dangerous behaviors.”

  • Aggregation:
    “Based on the available information, generate a checkout activity report organized chronologically and in logical sections. Cluster the output into: Theft Indicators, Scanning Avoidance, Barcode Manipulation, Suspicious Behavior, Abandoned Transaction, and Emergency Situation. This should be a concise yet descriptive summary of all important scanning events and anomalies. Format the output in Markdown for readability. Timestamps are in seconds, so please format them as SS.SSS.”

Let me know if you need any additional details or clarification.

logs_02.tar.gz (935.2 KB)

5

@junshengy any thoughts on the provided logs

6

We’ve encountered a similar issue during internal testing, but it’s difficult to reproduce. We’re already trying to resolve it internally, and we’ll update you here if we find any new findings or solutions.