VideoStreamRecorderOp issue with gxf_index empty

Continuing the discussion from Holoscan problem using VideoRecorderOp for saving segmentation output:

Dear all,
I am facing a similar problem to this topic but unfortunately the answer by the initial user is not very informative.
Basically I am grabbing frames from an Epiphan video capture system and want to record the video stream. I use a FormatConverter for dropping the alpha channel and convert to Tensor with the use of FormatConverterOp.
I then use the VideoStreamRecorderOp to save the serialized frames to disk.
I keep having the message

2024-03-11 15:40:59.900 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c

The file myvideo.gxf_index (recorder basename is “myvideo”) is always empty (0 bytes) while the file myvideo.gxf_entities is coherent (for instance if I receive 81 times the message above, this file is of size 503.9Mb which corresponds to 1920x1080x3 (input resolution, 3 channels uint8) x 81 (# frames) = 503.88Mb).
This way i cannot use the provided scripts in holoscan-sdk (holoscan-sdk/scripts at 90763fae219f27b36b758809fc052a416ee7a377 · nvidia-holoscan/holoscan-sdk · GitHub) to reconstruct a video file from gxf.
Note: If I use a HolovizOp, the frames are well displayed on my screen.

I do not understand what i do wrong and how to fix. Could you please help me here ?

Hereafter are provided the script I use (i installed holoscan 0.6.0 on an Orin DevKit) as well as the YAML config file:

import os

from holoscan.core import Application
from holoscan.conditions import CountCondition
from holoscan.operators import (HolovizOp, 
                                V4L2VideoCaptureOp, 
                                VideoStreamRecorderOp, 
                                FormatConverterOp)
from holoscan.resources import (BlockMemoryPool, 
                                UnboundedAllocator, 
                                CudaStreamPool, 
                                MemoryStorageType)


# Now define a simple application using the operators defined above
class App(Application):
    """Example of an application that uses the operators defined above.

    This application has the following operators:

    - V4L2VideoCaptureOp
    - FormatConverterOp
    - VideoStreamRecorderOp

    """

    def compose(self):
        source_args = self.kwargs("source")

        cuda_stream_pool = CudaStreamPool(
            self,
            name="cuda_stream",
            dev_id=0,
            stream_flags=0,
            stream_priority=0,
            reserved_size=1,
            max_size=5,
        )

        if "width" in source_args and "height" in source_args:
            # width and height given, use BlockMemoryPool (better latency)
            width = source_args["width"]
            height = source_args["height"]
            n_channels = 4
            block_size = width * height * n_channels
            # allocator = BlockMemoryPool(
            #     self, name="pool", storage_type=0, block_size=block_size, num_blocks=1
            # )
            allocator = UnboundedAllocator(self, name="pool")

            source = V4L2VideoCaptureOp(
                self,
                name="source",
                allocator=allocator,
                **source_args,
            )

            # Set Holoviz width and height from source resolution
            visualizer_args = self.kwargs("visualizer")
            visualizer_args["width"] = width
            visualizer_args["height"] = height
            visualizer = HolovizOp(
                self,
                name="visualizer",
                allocator=allocator,
                **visualizer_args,
            )

            # Set FormatConverter width and height from source resolution
            converter_args = self.kwargs("converter")
            converter = FormatConverterOp(
                self,
                name="converter",
                pool=allocator,
                cuda_stream_pool=cuda_stream_pool, # not much difference w/ and w/o this line : TODO to be investigated
                **converter_args,
            )

        else:
            # width and height not given, use UnboundedAllocator (worse latency)
            source = V4L2VideoCaptureOp(
                self,
                name="source",
                allocator=UnboundedAllocator(self, name="pool"),
                **self.kwargs("source"),
            )
            visualizer = HolovizOp(
                self,
                name="visualizer",
                **self.kwargs("visualizer"),
            )
            converter = FormatConverterOp(
               self,
               name="converter",
               **self.kwargs("converter"),
            )

        # Set recorder
        recorder = VideoStreamRecorderOp(
            self,
            name="recorder", 
            **self.kwargs("recorder")
        )

        self.add_flow(source, converter, {("signal", "source_video")})
        # self.add_flow(converter, visualizer, {("tensor", "receivers")})
        self.add_flow(converter, recorder, {("tensor", "input")})


def main(config_file):
    app = App()
    # if the --config command line argument was provided, it will override this config_file
    app.config(config_file)
    app.run()
    print("Application has finished running.")


if __name__ == "__main__":
    config_file = os.path.join(os.path.dirname(__file__), "recorder.yaml")
    main(config_file=config_file)
source:  # V4L2VideoCaptureOp
  device: "/dev/video0"
  width: 1920
  height: 1080

visualizer:  # Holoviz

converter: # FormatConverterOp
  out_tensor_name: "converted_tensor"
  in_dtype: "rgba8888"
  out_dtype: "rgb888"

recorder: # VideoStreamRecorderOp
  directory: "/home/igs/Projects/holoscan-test/myholoscan/test-hs-record/recordings"
  basename: "myvideo"

and hereafter are the logs i get in terminal (NOTE: i interrupt the application by stopping the process with Ctrl+C command):

[info] [gxf_executor.cpp:210] Creating context
[info] [gxf_executor.cpp:1595] Loading extensions from configs...
[info] [gxf_executor.cpp:1741] Activating Graph...
[info] [resource_manager.cpp:79] ResourceManager cannot find Resource of type: nvidia::gxf::GPUDevice for entity [eid: 00002, name: __entity_2]
[info] [resource_manager.cpp:106] ResourceManager cannot find Resource of type: nvidia::gxf::GPUDevice for component [cid: 00003, name: cuda_stream]
[info] [resource.hpp:44] Resource [type: nvidia::gxf::GPUDevice] from component [cid: 3] cannot find its value from ResourceManager
[info] [gxf_executor.cpp:1771] Running Graph...
[info] [gxf_executor.cpp:1773] Waiting for completion...
[info] [gxf_executor.cpp:1774] Graph execution waiting. Fragment: 
[info] [greedy_scheduler.cpp:190] Scheduling 3 entities
Opening in BLOCKING MODE 
2024-03-11 15:40:56.009 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
2024-03-11 15:40:56.037 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
2024-03-11 15:40:56.063 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
2024-03-11 15:40:56.088 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
...
2024-03-11 15:40:59.848 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
2024-03-11 15:40:59.873 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
2024-03-11 15:40:59.900 WARN  /workspace/holoscan-sdk/gxf_extensions/stream_playback/video_stream_serializer.cpp@188: No serializer found for component '' with type ID 0x7982aeac37f141beade86f00b4b5d47c
^C[info] [greedy_scheduler.cpp:390] Stopping scheduler.
[ubuntu:2427644:0:2427658] Caught signal 11 (Segmentation fault: invalid permissions for mapped object at address 0xffff83d94150)
==== backtrace (tid:2427658) ====
 0  /home/igs/Projects/holoscan-test/envhs060/lib/python3.8/site-packages/holoscan/graphs/..//lib/libucs.so.0(ucs_handle_error+0x2d4) [0xffff8c3bbce4]
 1  /home/igs/Projects/holoscan-test/envhs060/lib/python3.8/site-packages/holoscan/graphs/..//lib/libucs.so.0(+0x2ae74) [0xffff8c3bbe74]
 2  /home/igs/Projects/holoscan-test/envhs060/lib/python3.8/site-packages/holoscan/graphs/..//lib/libucs.so.0(+0x2b21c) [0xffff8c3bc21c]
 3  linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff950b07c0]
 4  [0xffff83d94150]
=================================
Segmentation fault (core dumped)

To follow up on this topic, I tried the same with Holoscan 1.0.3 under JP6.0DP and got the same issue (*.gxf_entities populated with frames, apparently, but *.gxf_index is still an empty file).

One hypothesis I had was maybe killing the program with a Ctl+C command prevented to have the file closed properly (?) and tried to use a CountCondition to just record 20 frames and stop but the following declaration

        recorder = VideoStreamRecorderOp(
            self,
            CountCondition(self, 20),
            name="recorder", 
            **self.kwargs("recorder")
        )

with just adding a CountCondition(self, 20) leads to the following error

recorder.py
[info] [gxf_executor.cpp:211] Creating context
Traceback (most recent call last):
File “/recorder/test-hs-record/recorder.py", line 121, in
main(config_file=config_file)
File "
/recorder/test-hs-record/recorder.py”, line 115, in main
app.run()
File “****/recorder/test-hs-record/recorder.py”, line 99, in compose
recorder = VideoStreamRecorderOp(
TypeError: init(): incompatible constructor arguments. The following argument types are supported:
1. holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp(fragment: holoscan.core._core.Fragment, directory: str, basename: str, flush_on_tick: bool = False, name: str = ‘recorder’)
Invoked with: <holoscan.Application: name:>, id: -1
name: noname_count_condition
fragment: “”
args:

  • name: count
    type: int64_t
    value: 20
    spec:
    fragment: “”
    params:
  • name: count
    type: int64_t
    description: The total number of times this term will permit execution.
    flag: kNone; kwargs: name=‘recorder’, directory=‘******/recorder/test-hs-record/recordings’, basename=‘myvideo’
    [info] [gxf_executor.cpp:230] Destroying context

Same problem of defining a CountCondition on V4L2VideoCaptureOp.
So I have no way to define a “proper” execution stop to test the impact on the *.gxf_index file so far.

Thanks for your kind help.
Best

My last try is by replacing the V4L2VideoCaptureOpby a VideoStreamReplayerOp as a video source (based on the example video_replayer from the holoscan-sdk).
If I configure the replayer with repeat: false and run my program, the *.gxf_index is well populated now ! So it was indeed the way to stop the application that seems to be the root cause of the issue. This seems weird and my lead to unexpected behavior in case of application crash for instance. I will report an issue on the github.

An other point: I can run the utility script convert_gxf_entities_to_video.py (holoscan-sdk/scripts) and reconstruct my .mp4 file. I checked the number and format of frames in both the source and the recorded stream and found for both the same which is coherent.

Guessed frame rate: 30.254941575078767 fps
Frame array shape: 540x960x3 (height x width x channels)

To recreate the video, the command line in the README of holoscan-sdk/script should be adapted to get the corresponding input resolution correct (here 960x540) otherwise the reconstructed video is not correct (in my case 29 frames of 2x8 mosaiced images of the racerx).

Hope this can help others facing the same issue as mine.
Best !
JP

Hi there, thank you for raising this issue to our attention. For others who may run into the same issue in the future, the solution has been posted in VideoStreamRecorderOp fails create *.gxf_index file on the flight · Issue #22 · nvidia-holoscan/holoscan-sdk · GitHub.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.