Hi,
Please try
gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mov "
BGR is not supported by nvvidconv. Please use videoconvert to convert it to RGBA and then send to nvvidconv.