RTSP Pipeline with Audio and Video on GStreamer

Developer Tools Nsight Systems Profiling Embedded Targets
, ,
1

Hi everyone,

I’m trying to build an RTSP pipeline using GStreamer to handle both video and audio streams from an IP camera. When the camera only sends a video stream, the following pipeline works perfectly:

rtspsrc location=%s latency=0 !  
rtph264depay ! h264parse !  
video/x-h264,stream-format=byte-stream,alignment=au ! queue ! appsink sync=TRUE emit-signals=TRUE name=appsink-video

However, when the camera provides both video and audio streams, the pipeline no longer works. I’ve been experimenting to create a working pipeline for dual streams, but I keep hitting issues.

First Attempt

rtspsrc location=%s name=source ! application/x-rtp,payload=96,media=video ! rtph264depay ! h264parse ! appsink sync=TRUE emit-signals=TRUE name=appsink-video  
source. ! application/x-rtp,payload=97,media=audio,clock-rate=8000,encoding-name=MPEG4-GENERIC,mode=AAC-hbr ! rtpmp4gdepay ! aacparse ! audio/mpeg,mpegversion=4,stream-format=adts ! queue ! appsink sync=TRUE emit-signals=TRUE name=appsink-audio

Here, I get the following error during parsing:

DEBUG aacparse gstaacparse.c:306:gst_aac_parse_sink_setcaps:<aacparse0> setcaps: audio/mpeg, mpegversion=(int)4, stream-format=(string)raw  
ERROR aacparse gstaacparse.c:354:gst_aac_parse_sink_setcaps:<aacparse0> Need codec_data for raw AAC

Second Attempt

rtspsrc location=%s latency=0 !  
capsfilter caps=application/x-rtp,media=video,payload=96,encoding-name=H264 !  
rtph264depay ! h264parse !  
video/x-h264,stream-format=byte-stream,alignment=au ! queue !  
appsink sync=TRUE emit-signals=TRUE name=appsink-video  

rtspsrc location=%s latency=0 !  
capsfilter caps=application/x-rtp,media=audio,payload=97,encoding-name=MPEG4-GENERIC,clock-rate=8000,mode=AAC-hbr !  
rtpG726depay ! parsebin !  
audio/mpeg,mpegversion=4,stream-format=adts ! queue !  
appsink sync=TRUE emit-signals=TRUE name=appsink-audio

Unfortunately, this approach also fails.

Additional Context

  • I’m working in an embedded environment built with Buildroot, so my GStreamer setup is quite minimal, and I have limited plugins.
  • I don’t need any transcoding; the goal is to simply act as a “tunnel” for the streams.

Does anyone have experience with similar issues or know of a solution? Any suggestions or insights would be greatly appreciated!

Thanks in advance!