Trouble with Live Portrait Quick Start - "unable to set to state PLAYING"

luke83 · September 23, 2023, 12:00am

Greetings. I am following the Live Portrait Quick Start Guide. I successfully ran the following commands in Windows Subsystem for Linux 2:

$ bash ./maxine_lp_init.sh
$ bash ./maxine_lp_start.sh
$ bash ./maxine_lp_client_start.sh

The problem happens when I try to run the following in the client’s interactive shell:

  GLOG_v=4 video_controller_client --grpc_uri=127.0.0.1:50060\
    --video_config=/host/lp_test_client_config.json \
    --video_input=/host/sample_inputs/video_inputs/live_portrait_sample_video.mp4 \
    --aux_input=/host/sample_inputs/aux_inputs/live_portrait_sample_portrait.png \
    --video_output=/host/outputs/maxine_video_out.mp4

I get this error:

WARNING: Logging before InitGoogleLogging() is written to STDERR
I20230922 23:42:07.597575    10 video_controller_client.cc:434] gPRC connection uri : 127.0.0.1:50060
I20230922 23:42:07.598493    10 client_carrier.cpp:39] InjectSpan
I20230922 23:42:08.281419    10 video_controller_streaming_client.cc:309] Creating sender pipe video_send_aux: filesrc  name=filesrc_aux ! pngdec name=imagedec_aux  ! nvvideoconvert name=nvvconv_aux  ! video/x-raw(memory:NVMM),format=NV12,colorimetry=bt709 ! nvv4l2h264enc profile=Main  bitrate=5242800   name=nvenc_aux ! queue ! h264parse name=h264parse_aux  config-interval=-1 ! fakesink sync=true name=fakesink_grpc_aux async=false
E20230922 23:42:08.355175    10 nv_gst_pipeline.cc:144] Pipeline:video_send_aux unable to set to state PLAYING
E20230922 23:42:08.355237    10 video_controller_streaming_client.cc:412] Pipeline launch failed

(video_controller_client:10): GLib-CRITICAL **: 23:42:08.355: g_main_loop_unref: assertion 'loop != NULL' failed
I20230922 23:42:08.355574    10 nv_gst_pipeline.cc:558] Pipeline:video_send_aux deleted.
E20230922 23:42:08.355620    10 video_controller_client.cc:512] Failed to create video-send pipeline
I20230922 23:42:08.355698    10 video_controller_client.cc:290] Stopping client...
terminate called without an active exception
Aborted

My Ubuntu 22.04 can “see” my RTX 2080 Ti card when I type nvidia-smi on the commandline. Is the problem caused by running these scripts on Ubuntu under Windows Subsystem for Linux 2?

luke83 · October 2, 2023, 10:34pm

Regarding this specific line in the error message above:

(video_controller_client:10): GLib-CRITICAL **: 23:42:08.355: g_main_loop_unref: assertion 'loop != NULL' failed

…could the current Live-Portrait containers contain debugging code by mistake?

luke83 · October 5, 2023, 9:12pm

The tutorial itself advises that it can only be run on sm_86 and sm_75 architectures, which my original machine did not have.

An AWS EC2 g4dn.2xlarge instance running a Tesla T4 GPU solved the GLib-CRITICAL error problem. The free NVIDIA Deep Learning Base AMI on the AWS marketplace helped immensely in starting a Linux instance that could run this Live Portrait tutorial.

I’m running into a new error now, however, when I run this command in the client’s interactive shell:

  GLOG_v=4 video_controller_client --grpc_uri=127.0.0.1:50060\
    --video_config=/host/lp_test_client_config.json \
    --video_input=/host/sample_inputs/video_inputs/live_portrait_sample_video.mp4 \
    --aux_input=/host/sample_inputs/aux_inputs/live_portrait_sample_portrait.png \
    --video_output=/host/outputs/maxine_video_out.mp4

New error:

WARNING: Logging before InitGoogleLogging() is written to STDERR
I20231005 18:12:22.866077    10 video_controller_client.cc:434] gPRC connection uri : 127.0.0.1:50060
I20231005 18:12:22.866919    10 client_carrier.cpp:39] InjectSpan
I20231005 18:12:23.192798    10 video_controller_streaming_client.cc:309] Creating sender pipe video_send_aux: filesrc  name=filesrc_aux ! pngdec name=imagedec_aux  ! nvvideoconvert name=nvvconv_aux  ! video/x-raw(memory:NVMM),format=NV12,colorimetry=bt709 ! nvv4l2h264enc profile=Main  bitrate=5242800   name=nvenc_aux ! queue ! h264parse name=h264parse_aux  config-interval=-1 ! fakesink sync=true name=fakesink_grpc_aux async=false 
I20231005 18:12:23.283469    10 video_controller_streaming_client.cc:415] Dumping graph video_send_aux
E20231005 18:12:23.289826    22 nv_gst_pipeline.cc:502] MessageHandlerPipeline:video_send_aux ERROR from element nvenc_aux: Could not get/set settings from/on resource. error domain: gst-resource-error-quark error_code: 13
E20231005 18:12:23.289842    22 nv_gst_pipeline.cc:503] MessageHandler Error details: gstv4l2object.c(3511): gst_v4l2_object_set_format_full (): /GstPipeline:video_send_aux/GstBin:GstBin:video_send_aux-unique-id/nvv4l2h264enc:nvenc_aux:
Device is in streaming mode
E20231005 18:12:23.289852    22 video_controller_streaming_client.cc:140] ErrorCallback Sender- -1-Pipeline:video_send_aux ERROR from element nvenc_aux: Could not get/set settings from/on resource. error domain: gst-resource-error-quark error_code: 13
I20231005 18:12:23.289867    22 nv_gst_pipeline.cc:185] Pipeline:video_send_aux returned, stopping playback.
I20231005 18:12:23.289942    10 video_controller_client.cc:518] Set Grpc msg for Portrait image of size 0
I20231005 18:12:23.289963    10 video_controller_client.cc:248] Setting Live Portrait Config
I20231005 18:12:23.488305    14 video_controller_client.cc:253]  Server closed Cancelled on the server side
I20231005 18:12:23.488323    14 video_controller_client.cc:290] Stopping client...
I20231005 18:12:23.488366    14 video_controller_client.cc:301] Stopping receiver pipe...
I20231005 18:12:23.489117    28 nv_gst_pipeline.cc:475] Pipeline:video_receive: End of stream
I20231005 18:12:23.489132    28 nv_gst_pipeline.cc:477] MessageHandlerPipeline:video_receive: Quitting pipeline on EOS.
I20231005 18:12:23.489146    28 nv_gst_pipeline.cc:185] Pipeline:video_receive returned, stopping playback.
I20231005 18:12:26.488482    10 video_controller_client.cc:588] Sender pipelines have Finished
I20231005 18:12:26.488507    10 video_controller_client.h:42] Calling Finish for client.
I20231005 18:12:26.488539    10 video_controller_client.cc:594] Sending EOS to receiver pipeline
I20231005 18:12:26.489089    10 video_controller_client.cc:608] Exiting client..
I20231005 18:12:26.490036    10 nv_gst_pipeline.cc:558] Pipeline:video_receive deleted.
I20231005 18:12:26.492063    10 nv_gst_pipeline.cc:558] Pipeline:video_send_aux deleted.

Excerpt from the above:

E20231005 18:12:23.289826    22 nv_gst_pipeline.cc:502] MessageHandlerPipeline:video_send_aux ERROR from element nvenc_aux: Could not get/set settings from/on resource. error domain: gst-resource-error-quark error_code: 13
E20231005 18:12:23.289842    22 nv_gst_pipeline.cc:503] MessageHandler Error details: gstv4l2object.c(3511): gst_v4l2_object_set_format_full (): /GstPipeline:video_send_aux/GstBin:GstBin:video_send_aux-unique-id/nvv4l2h264enc:nvenc_aux:
Device is in streaming mode
E20231005 18:12:23.289852    22 video_controller_streaming_client.cc:140] ErrorCallback Sender- -1-Pipeline:video_send_aux ERROR from element nvenc_aux: Could not get/set settings from/on resource. error domain: gst-resource-error-quark error_code: 13

I will keep working on this, but I would appreciate any insight into the above error.