Adding overlay to the Tegra Camera API (Argus) "gstVideoEncode" sample

Hi,

I’ve been playing with the gstVideoEncode sample that creates a stream using the Argus API and pipes it into a GStreamer pipeline via NvEGLStreamSrc. Now I’m trying to add an overlay (timeoverlay/clockoverlay/textoverlay), how it blocks the pipeline without an error message.

Sample Pipeline:
nveglstreamsrc -> queue -> omxh264enc -> qtmux -> filesink

My Pipeline:
nveglstreamsrc -> queue -> nvvidconv -> clockoverlay -> omxh264enc -> splitmuxsink

This does not generate any errors just doesn’t seem to pipe frames into the file at all. I have successfully done H264 encoding using NvCameraSrc before just fine.

Working Full GStreamer pipeline:
nvcamerasrc -> nvvidconv -> clockoverlay -> omxh264enc -> splitmuxsink

Why does Argus and NvEGLStreamSrc not work? Please help! Thanks!

Hi randyzhg, please share a sample to reproduce the issue. Thanks.

Ok now it’s reporting Internal data flow errors. But I still think this should be do able. What am I doing wrong? Thanks!

#include <Argus/Argus.h>
#include <gst/gst.h>
#include <stdlib.h>
#include <unistd.h>
#include "Error.h"
#include "Options.h"
#include "PreviewConsumer.h"
#include "GLContext.h"
#include <dirent.h>
#include <string>

namespace ArgusMesa
{
	// Globals
	
	static ArgusSamples::EGLDisplayHolder g_display;
	static const Argus::Size2D<uint32_t> PREVIEW_STREAM_SIZE(3840, 2160);
	
	class GstFramework
	{
		protected:
			GstState gst_state;
			GstElement *m_pipeline;
		
		public:
			GMainLoop *loop;
			
			GstFramework()
				: gst_state(GST_STATE_NULL)
				, m_pipeline(NULL)
				, loop(g_main_loop_new(NULL,FALSE)) {}
				
			~GstFramework()
			{
				shutdown();
			}
			
			/**
			 * Initialize the GStreamer video encoder pipeline.
			 * @param[in] eglStream --- The EGLStream to consume frames from.
			 */
			bool initialize(EGLStreamKHR eglStream)
			{
				// Initialize GStreamer.
				gst_init(NULL, NULL);
				loop = g_main_loop_new(NULL, FALSE);
		

				GstBus *bus;
				guint bus_watch_id;
				GstElement *src, *srcfilter, *nvconv;
				GstCaps *caps;


				/* Create GStreamer Elements */
				m_pipeline = gst_pipeline_new(NULL);
				src = gst_element_factory_make("nveglstreamsrc", NULL);
				srcfilter = gst_element_factory_make("capsfilter", NULL);
				nvconv = gst_element_factory_make("nvvidconv", NULL);
				GstElement *overlay = gst_element_factory_make("clockoverlay", NULL);
				GstElement *enc = gst_element_factory_make("omxh264enc",NULL);
				GstElement *filesink = gst_element_factory_make("splitmuxsink", NULL);
	
				if (!m_pipeline || !src || !srcfilter || !nvconv || !overlay || !enc || !filesink) {
					g_printerr("One element count not be created.\n");
					return false;
				}

				/* Set up GStreamer Bus Message Handler */
				bus = gst_pipeline_get_bus(GST_PIPELINE(m_pipeline));
				bus_watch_id = gst_bus_add_watch(bus, bus_callback, loop);
				gst_object_unref(bus);


				/* Configure GStreamer Element properties */
				g_object_set(G_OBJECT(src), "display", g_display.get(), NULL);
				g_object_set(G_OBJECT(src), "eglstream", eglStream, NULL);
	
				caps = gst_caps_from_string("video/x-raw(memory:NVMM), width=2592, height=1944, framerate=30/1, format=I420");
				g_object_set(srcfilter, "caps", caps, NULL);
				gst_caps_unref(caps);

				g_object_set(filesink, "location", "%05d.264", NULL);
	
				/* Add all elements to the pipeline */
				gst_bin_add_many(GST_BIN(m_pipeline),
								src, srcfilter, nvconv, overlay, enc, filesink, NULL);
	
				/* Link elements */
				gst_element_link_many(src, srcfilter, nvconv, overlay, enc, filesink, NULL);

				printf("GST Intialization done\n");

				return true;
			}
			
			/**
			 * Watches for messages on the pipeline bus.
			 * @param[in] bus --- The GStreamer bus that is being watched.
			 * @param[in] msg --- The message that was received on the bus.
			 * @param[in] data --- Any user data to be passed into the callback.
			 */
			static gboolean bus_callback(GstBus *bus, GstMessage *msg, gpointer data)
			{
				GMainLoop *lp = (GMainLoop *) data;

				switch (GST_MESSAGE_TYPE(msg)) {
					case GST_MESSAGE_EOS: {
						g_main_loop_quit (lp);
						break;
					}
					
					case GST_MESSAGE_ERROR: {
						gchar  *debug;
						GError *error;

						gst_message_parse_error(msg, &error, &debug);
						g_free (debug);

						g_printerr("Error: %s\n", error->message);
						g_error_free(error);

						g_main_loop_quit(lp);
						break;
					}
			
					default:
						break;
				}
				return TRUE;
			}

			/**
			 * Stops the pipeline and frees resources.
			 */
			void shutdown()
			{
				if (gst_state == GST_STATE_PLAYING)
					stopRecording();
				
				if (m_pipeline)
					gst_object_unref(m_pipeline);
					
				m_pipeline = NULL;
			}
			
			/**
			 * Start GStreamer pipeline.
			 */
			bool startRecording()
			{
				if (!m_pipeline)
					ORIGINATE_ERROR("GStreamer pipeline not initialized");
				
				if (gst_state != GST_STATE_NULL)
					ORIGINATE_ERROR("GStreamer pipeline already running");
			
				if (gst_element_set_state(m_pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE)
					ORIGINATE_ERROR("Failed to start pipeline");
				
				gst_state = GST_STATE_PLAYING;
				return true;
			}
	
			/**
			 * Stop GStreamer pipeline.
			 */
			bool stopRecording()
			{
				if (!m_pipeline)
					ORIGINATE_ERROR("GStreamer pipeline not initialized");
				
				if (gst_state != GST_STATE_PLAYING)
					ORIGINATE_ERROR("GStreamer pipeline not running");
			
				if (gst_element_set_state(m_pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)
					ORIGINATE_ERROR("Failed to stop pipeline");
				
				gst_state = GST_STATE_NULL;
				return true;
			}
	}; // class GstFramework
	
	struct ExecuteOptions
	{
		uint32_t cameraIndex;
		uint32_t captureSeconds;
	};
	
	/**
	 * Executes the GStreamer pipeline with options.
	 * @param[in] options --- ExecuteOptions struct with 
	 */
	static bool execute(const ExecuteOptions& options)
	{
		
		using namespace Argus;
		PROPAGATE_ERROR(g_display.initialize(EGL_DEFAULT_DISPLAY));
		
		/* Create CameraProvider */
		UniqueObj<CameraProvider> cameraProvider(CameraProvider::create());
		ICameraProvider *iCameraProvider = interface_cast<ICameraProvider>(cameraProvider);
		
		printf("Argus Version: %s\n", iCameraProvider->getVersion().c_str());
		if (!iCameraProvider)
			ORIGINATE_ERROR("Failed to open CameraProvider");

		/* Get/use the first available CameraDevice */
		std::vector<CameraDevice*> cameraDevices;
		if (iCameraProvider->getCameraDevices(&cameraDevices) != STATUS_OK)
			ORIGINATE_ERROR("Failed to get CameraDevices");
		if (cameraDevices.size() == 0)
			ORIGINATE_ERROR("No CameraDevices available");
		if (cameraDevices.size() <= options.cameraIndex)
			ORIGINATE_ERROR("Camera %d not available; there are %d cameras",
		options.cameraIndex, (unsigned)cameraDevices.size());

		CameraDevice *cameraDevice = cameraDevices[options.cameraIndex];
		ICameraProperties *iCameraProperties = interface_cast<ICameraProperties>(cameraDevice);
		if (!iCameraProperties)
			ORIGINATE_ERROR("Failed to get ICameraProperties interface");

		/* Create CaptureSession */
		UniqueObj<CaptureSession> captureSession(iCameraProvider->createCaptureSession(cameraDevice));
		ICaptureSession *iSession = interface_cast<ICaptureSession>(captureSession);
		if (!iSession)
			ORIGINATE_ERROR("Failed to create CaptureSession");
		
		// Get the sensor mode to determine the video output stream resolution.
		std::vector<Argus::SensorMode*> sensorModes;
		iCameraProperties->getBasicSensorModes(&sensorModes);
		if (sensorModes.size() == 0)
			ORIGINATE_ERROR("Failed to get sensor modes");
		ISensorMode *iSensorMode = interface_cast<ISensorMode>(sensorModes[0]);
		printf("Capture Resolution: %dx%d\n", iSensorMode->getResolution().width(), iSensorMode->getResolution().height());
		if (!iSensorMode)
			ORIGINATE_ERROR("Failed to get sensor mode interface");
		
		/* Set common output stream settings */
		UniqueObj<OutputStreamSettings> streamSettings(iSession->createOutputStreamSettings());
		IOutputStreamSettings *iStreamSettings = interface_cast<IOutputStreamSettings>(streamSettings);
		if (!iStreamSettings)
			ORIGINATE_ERROR("Failed to create OutputStreamSettings");
		iStreamSettings->setPixelFormat(PIXEL_FMT_YCbCr_420_888);
		iStreamSettings->setEGLDisplay(g_display.get());

		/* Create video encoder stream */
		iStreamSettings->setResolution(PREVIEW_STREAM_SIZE);
		UniqueObj<OutputStream> videoStream(iSession->createOutputStream(streamSettings.get()));
		IStream *iVideoStream = interface_cast<IStream>(videoStream);
		if (!iVideoStream)
			ORIGINATE_ERROR("Failed to create video stream");

		/* Create capture Request and enable the streams in the Request */
		UniqueObj<Request> request(iSession->createRequest(CAPTURE_INTENT_VIDEO_RECORD));
		IRequest *iRequest = interface_cast<IRequest>(request);
		if (!iRequest)
			ORIGINATE_ERROR("Failed to create Request");
		if (iRequest->enableOutputStream(videoStream.get()) != STATUS_OK)
			ORIGINATE_ERROR("Failed to enable video stream in Request");

		/* Initialize the GStreamer video encoder consumer */
		GstFramework gstPipeline;
		if (!gstPipeline.initialize(iVideoStream->getEGLStream()))
			ORIGINATE_ERROR("Failed to initialize gstPipeline EGLStream consumer");
		if (!gstPipeline.startRecording())
			ORIGINATE_ERROR("Failed to start video recording");

		/* Perform repeat capture requests for requested number of seconds */
		if (iSession->repeat(request.get()) != STATUS_OK)
			ORIGINATE_ERROR("Failed to start repeat capture requests");
		else
			g_main_loop_run(gstPipeline.loop);
		
		/* If the GMainLoop stops running, the code below will execute */
		iSession->stopRepeat();

		/* Wait until all frames have completed before stopping recording. */
		/// @todo: Not doing this may cause a deadlock.
		iSession->waitForIdle();

		/* Stop the pipeline */
		if (!gstPipeline.stopRecording())
			ORIGINATE_ERROR("Failed to stop pipeline");
		gstPipeline.shutdown();
		videoStream.reset();

		return true;
	}

}; // namespace ArgusMesa


int main(int argc, char **argv)
{
    printf("Executing: %s\n", basename(argv[0]));

    ArgusSamples::Value<uint32_t> cameraIndex(0);
    ArgusSamples::Value<uint32_t> captureTime(10);

    ArgusMesa::ExecuteOptions executeOptions;
    executeOptions.cameraIndex = cameraIndex.get();
    executeOptions.captureSeconds = captureTime.get();

    if (!ArgusMesa::execute(executeOptions))
        return EXIT_FAILURE;

    return EXIT_SUCCESS;
}

hi randyzhg, please share Makefile also. Thanks.

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.5

# Default target executed when no arguments are given to make.
default_target: all

.PHONY : default_target

# Allow only one "make -f Makefile2" at a time, but pass parallelism.
.NOTPARALLEL:


#=============================================================================
# Special targets provided by cmake.

# Disable implicit rules so canonical targets will work.
.SUFFIXES:


# Remove some rules from gmake that .SUFFIXES does not remove.
SUFFIXES =

.SUFFIXES: .hpux_make_needs_suffix_list


# Suppress display of executed commands.
$(VERBOSE).SILENT:


# A target that is always out of date.
cmake_force:

.PHONY : cmake_force

#=============================================================================
# Set environment variables for the build.

# The shell in which to execute make rules.
SHELL = /bin/sh

# The CMake executable.
CMAKE_COMMAND = /usr/bin/cmake

# The command to remove a file.
RM = /usr/bin/cmake -E remove -f

# Escaping for special characters.
EQUALS = =

# The top-level source directory on which CMake was run.
CMAKE_SOURCE_DIR = /home/nvidia/workspace/deeplearning/mesa_argus

# The top-level build directory on which CMake was run.
CMAKE_BINARY_DIR = /home/nvidia/workspace/deeplearning/mesa_argus/build

#=============================================================================
# Targets provided globally by CMake.

# Special rule for the target edit_cache
edit_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "No interactive CMake dialog available..."
	/usr/bin/cmake -E echo No\ interactive\ CMake\ dialog\ available.
.PHONY : edit_cache

# Special rule for the target edit_cache
edit_cache/fast: edit_cache

.PHONY : edit_cache/fast

# Special rule for the target rebuild_cache
rebuild_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Running CMake to regenerate build system..."
	/usr/bin/cmake -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR)
.PHONY : rebuild_cache

# Special rule for the target rebuild_cache
rebuild_cache/fast: rebuild_cache

.PHONY : rebuild_cache/fast

# Special rule for the target package
package: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Run CPack packaging tool..."
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && /usr/bin/cpack --config ./CPackConfig.cmake
.PHONY : package

# Special rule for the target package
package/fast: package

.PHONY : package/fast

# Special rule for the target package_source
package_source:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Run CPack packaging tool for source..."
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && /usr/bin/cpack --config ./CPackSourceConfig.cmake /home/nvidia/workspace/deeplearning/mesa_argus/build/CPackSourceConfig.cmake
.PHONY : package_source

# Special rule for the target package_source
package_source/fast: package_source

.PHONY : package_source/fast

# Special rule for the target install/strip
install/strip: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Installing the project stripped..."
	/usr/bin/cmake -DCMAKE_INSTALL_DO_STRIP=1 -P cmake_install.cmake
.PHONY : install/strip

# Special rule for the target install/strip
install/strip/fast: install/strip

.PHONY : install/strip/fast

# Special rule for the target install/local
install/local: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Installing only the local directory..."
	/usr/bin/cmake -DCMAKE_INSTALL_LOCAL_ONLY=1 -P cmake_install.cmake
.PHONY : install/local

# Special rule for the target install/local
install/local/fast: install/local

.PHONY : install/local/fast

# Special rule for the target list_install_components
list_install_components:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Available install components are: \"Unspecified\""
.PHONY : list_install_components

# Special rule for the target list_install_components
list_install_components/fast: list_install_components

.PHONY : list_install_components/fast

# Special rule for the target install
install: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Install the project..."
	/usr/bin/cmake -P cmake_install.cmake
.PHONY : install

# Special rule for the target install
install/fast: preinstall/fast
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Install the project..."
	/usr/bin/cmake -P cmake_install.cmake
.PHONY : install/fast

# The main all target
all: cmake_check_build_system
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -E cmake_progress_start /home/nvidia/workspace/deeplearning/mesa_argus/build/CMakeFiles /home/nvidia/workspace/deeplearning/mesa_argus/build/workspace/multimediaPipeline/CMakeFiles/progress.marks
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/all
	$(CMAKE_COMMAND) -E cmake_progress_start /home/nvidia/workspace/deeplearning/mesa_argus/build/CMakeFiles 0
.PHONY : all

# The main clean target
clean:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/clean
.PHONY : clean

# The main clean target
clean/fast: clean

.PHONY : clean/fast

# Prepare targets for installation.
preinstall: all
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/preinstall
.PHONY : preinstall

# Prepare targets for installation.
preinstall/fast:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/preinstall
.PHONY : preinstall/fast

# clear depends
depend:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 1
.PHONY : depend

# Convenience name for target.
workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule
.PHONY : workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule

# Convenience name for target.
multimediaPipeline: workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule

.PHONY : multimediaPipeline

# fast build rule for target.
multimediaPipeline/fast:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build
.PHONY : multimediaPipeline/fast

m_pipeline.o: m_pipeline.cpp.o

.PHONY : m_pipeline.o

# target to build an object file
m_pipeline.cpp.o:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.o
.PHONY : m_pipeline.cpp.o

m_pipeline.i: m_pipeline.cpp.i

.PHONY : m_pipeline.i

# target to preprocess a source file
m_pipeline.cpp.i:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.i
.PHONY : m_pipeline.cpp.i

m_pipeline.s: m_pipeline.cpp.s

.PHONY : m_pipeline.s

# target to generate assembly for a file
m_pipeline.cpp.s:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.s
.PHONY : m_pipeline.cpp.s

# Help Target
help:
	@echo "The following are some of the valid targets for this Makefile:"
	@echo "... all (the default if no target is provided)"
	@echo "... clean"
	@echo "... depend"
	@echo "... edit_cache"
	@echo "... rebuild_cache"
	@echo "... package"
	@echo "... package_source"
	@echo "... multimediaPipeline"
	@echo "... install/strip"
	@echo "... install/local"
	@echo "... list_install_components"
	@echo "... install"
	@echo "... m_pipeline.o"
	@echo "... m_pipeline.i"
	@echo "... m_pipeline.s"
.PHONY : help



#=============================================================================
# Special targets to cleanup operation of make.

# Special rule to run CMake to check the build system integrity.
# No rule that depends on this can have commands that come from listfiles
# because they might be regenerated.
cmake_check_build_system:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 0
.PHONY : cmake_check_build_system
# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.5

# Default target executed when no arguments are given to make.
default_target: all

.PHONY : default_target

# Allow only one "make -f Makefile2" at a time, but pass parallelism.
.NOTPARALLEL:


#=============================================================================
# Special targets provided by cmake.

# Disable implicit rules so canonical targets will work.
.SUFFIXES:


# Remove some rules from gmake that .SUFFIXES does not remove.
SUFFIXES =

.SUFFIXES: .hpux_make_needs_suffix_list


# Suppress display of executed commands.
$(VERBOSE).SILENT:


# A target that is always out of date.
cmake_force:

.PHONY : cmake_force

#=============================================================================
# Set environment variables for the build.

# The shell in which to execute make rules.
SHELL = /bin/sh

# The CMake executable.
CMAKE_COMMAND = /usr/bin/cmake

# The command to remove a file.
RM = /usr/bin/cmake -E remove -f

# Escaping for special characters.
EQUALS = =

# The top-level source directory on which CMake was run.
CMAKE_SOURCE_DIR = /home/nvidia/workspace/deeplearning/mesa_argus

# The top-level build directory on which CMake was run.
CMAKE_BINARY_DIR = /home/nvidia/workspace/deeplearning/mesa_argus/build

#=============================================================================
# Targets provided globally by CMake.

# Special rule for the target edit_cache
edit_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "No interactive CMake dialog available..."
	/usr/bin/cmake -E echo No\ interactive\ CMake\ dialog\ available.
.PHONY : edit_cache

# Special rule for the target edit_cache
edit_cache/fast: edit_cache

.PHONY : edit_cache/fast

# Special rule for the target rebuild_cache
rebuild_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Running CMake to regenerate build system..."
	/usr/bin/cmake -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR)
.PHONY : rebuild_cache

# Special rule for the target rebuild_cache
rebuild_cache/fast: rebuild_cache

.PHONY : rebuild_cache/fast

# Special rule for the target package
package: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Run CPack packaging tool..."
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && /usr/bin/cpack --config ./CPackConfig.cmake
.PHONY : package

# Special rule for the target package
package/fast: package

.PHONY : package/fast

# Special rule for the target package_source
package_source:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Run CPack packaging tool for source..."
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && /usr/bin/cpack --config ./CPackSourceConfig.cmake /home/nvidia/workspace/deeplearning/mesa_argus/build/CPackSourceConfig.cmake
.PHONY : package_source

# Special rule for the target package_source
package_source/fast: package_source

.PHONY : package_source/fast

# Special rule for the target install/strip
install/strip: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Installing the project stripped..."
	/usr/bin/cmake -DCMAKE_INSTALL_DO_STRIP=1 -P cmake_install.cmake
.PHONY : install/strip

# Special rule for the target install/strip
install/strip/fast: install/strip

.PHONY : install/strip/fast

# Special rule for the target install/local
install/local: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Installing only the local directory..."
	/usr/bin/cmake -DCMAKE_INSTALL_LOCAL_ONLY=1 -P cmake_install.cmake
.PHONY : install/local

# Special rule for the target install/local
install/local/fast: install/local

.PHONY : install/local/fast

# Special rule for the target list_install_components
list_install_components:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Available install components are: \"Unspecified\""
.PHONY : list_install_components

# Special rule for the target list_install_components
list_install_components/fast: list_install_components

.PHONY : list_install_components/fast

# Special rule for the target install
install: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Install the project..."
	/usr/bin/cmake -P cmake_install.cmake
.PHONY : install

# Special rule for the target install
install/fast: preinstall/fast
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Install the project..."
	/usr/bin/cmake -P cmake_install.cmake
.PHONY : install/fast

# The main all target
all: cmake_check_build_system
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -E cmake_progress_start /home/nvidia/workspace/deeplearning/mesa_argus/build/CMakeFiles /home/nvidia/workspace/deeplearning/mesa_argus/build/workspace/multimediaPipeline/CMakeFiles/progress.marks
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/all
	$(CMAKE_COMMAND) -E cmake_progress_start /home/nvidia/workspace/deeplearning/mesa_argus/build/CMakeFiles 0
.PHONY : all

# The main clean target
clean:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/clean
.PHONY : clean

# The main clean target
clean/fast: clean

.PHONY : clean/fast

# Prepare targets for installation.
preinstall: all
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/preinstall
.PHONY : preinstall

# Prepare targets for installation.
preinstall/fast:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/preinstall
.PHONY : preinstall/fast

# clear depends
depend:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 1
.PHONY : depend

# Convenience name for target.
workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f CMakeFiles/Makefile2 workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule
.PHONY : workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule

# Convenience name for target.
multimediaPipeline: workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/rule

.PHONY : multimediaPipeline

# fast build rule for target.
multimediaPipeline/fast:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build
.PHONY : multimediaPipeline/fast

m_pipeline.o: m_pipeline.cpp.o

.PHONY : m_pipeline.o

# target to build an object file
m_pipeline.cpp.o:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.o
.PHONY : m_pipeline.cpp.o

m_pipeline.i: m_pipeline.cpp.i

.PHONY : m_pipeline.i

# target to preprocess a source file
m_pipeline.cpp.i:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.i
.PHONY : m_pipeline.cpp.i

m_pipeline.s: m_pipeline.cpp.s

.PHONY : m_pipeline.s

# target to generate assembly for a file
m_pipeline.cpp.s:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(MAKE) -f workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/build.make workspace/multimediaPipeline/CMakeFiles/multimediaPipeline.dir/m_pipeline.cpp.s
.PHONY : m_pipeline.cpp.s

# Help Target
help:
	@echo "The following are some of the valid targets for this Makefile:"
	@echo "... all (the default if no target is provided)"
	@echo "... clean"
	@echo "... depend"
	@echo "... edit_cache"
	@echo "... rebuild_cache"
	@echo "... package"
	@echo "... package_source"
	@echo "... multimediaPipeline"
	@echo "... install/strip"
	@echo "... install/local"
	@echo "... list_install_components"
	@echo "... install"
	@echo "... m_pipeline.o"
	@echo "... m_pipeline.i"
	@echo "... m_pipeline.s"
.PHONY : help



#=============================================================================
# Special targets to cleanup operation of make.

# Special rule to run CMake to check the build system integrity.
# No rule that depends on this can have commands that come from listfiles
# because they might be regenerated.
cmake_check_build_system:
	cd /home/nvidia/workspace/deeplearning/mesa_argus/build && $(CMAKE_COMMAND) -H$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 0
.PHONY : cmake_check_build_system

Hi randyzhg,
We are still not able to build it. There is hard coding in your cmake:

cd /home/nvidia/workspace/deeplearning/mesa_argus/build

Anyway, there is mismatch of width and height in your code:

static const Argus::Size2D<uint32_t> PREVIEW_STREAM_SIZE(3840, 2160);
caps = gst_caps_from_string("video/x-raw(memory:NVMM), width=2592, height=1944, framerate=30/1, format=I420");

It may be rootcause.

I changed the resolution size, it doesn’t make it work.
What does the compiler complain about when you try to build?

I don’t understand how you are not able to build this.

Try this:
Select-all, copy and paste and overwrite the main.cpp in /home/nvidia/tegra_multimesa_api/argus/samples/gstVideoEncode/main.cpp
And then just build it as if you were building your own sample code. I just did this on my TX2 and it built the executable just fine…

Please specify output of nvvidconv and add ‘tee’ between nvvidconv and clockoverlay:
$ gst-launch-1.0 nvcamerasrc ! nvvidconv ! ‘video/x-raw,format=I420’ ! tee ! clockoverlay ! nvoverlaysink

I know gst-launch-1.0 works and I know nvcamerasrc works. I’m not able to get it to work when I use Argus with nveglstreamsrc. When I add this to my libargus code it still does not work I get an “Error: Internal data flow error.”

Attach a sample based on gstVideoEncode and verfied on r28.1/TX2 + onboard ov5693.
It demonstrates Argus -> nveglstreamsrc ! queue ! nvvidconv ! clockoverlay ! omxh264enc ! qtmux ! filesink
main.cpp (18 KB)

I don’t get it this very similar to what I did, but this works… Anyway thanks soooo much for your help this week!

Ok I have another question. Why can’t I add an AppSink to the pipeline. It seems that the AppSink and FileSink can’t co-exist. I have tried using an Appsink by itself and it works fine.

You can use tee
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-tee.html

I know how to use tee. But the behaviour I’m observing is that the presence of both sinks causes the pipeline to stall. Even if I just add the appsink to the pipeline without linking it it seems to block something.

For more examples of using tee element, you may go to http://gstreamer-devel.966125.n4.nabble.com/