# Output Tensor in FaceDetectIR **URL:** \ **Category:** DeepStream SDK\ **Tags:** python, camera\ **Created:** [January 20, 2021, 1:50pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311 "2021-01-20T13:50:16Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [January 20, 2021, 1:50pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/1 "2021-01-20T13:50:17Z")
**• Hardware Platform (Jetson Nano)** **• DeepStream Version 5.0.1** **• JetPack Version 4.4** **• Issue Type( questions)** Hi, i’d like to crop detected face after inferencing. I’m using [SSD Parser](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/deepstream-ssd-parser) as an example and my problem is that in `pgie_src_pad_buffer_probe` when I call `l_user = frame_meta.frame_user_meta_list` ` l_user` is `None` but tracker and nvosd are working fine they detect face perfectly my config file is > [property] > gpu-id=0 > net-scale-factor=0.0039215697906911373 > tlt-model-key=tlt\_encode > tlt-encoded-model=/opt/nvidia/deepstream/deepstream-5.0/samples/models/tlt\_pretrained\_models/facedetectir/resnet18\_facedetectir\_pruned.etlt > labelfile-path=labels\_facedetectir.txt > int8-calib-file=/opt/nvidia/deepstream/deepstream-5.0/samples/models/tlt\_pretrained\_models/facedetectir/facedetectir\_int8.txt > model-engine-file=/opt/nvidia/deepstream/deepstream-5.0/samples/models/tlt\_pretrained\_models/facedetectir/resnet18\_facedetectir\_pruned.etlt\_b1\_gpu0\_fp16.engine > input-dims=3;240;384;0 > uff-input-blob-name=input\_1 > batch-size=1 > process-mode=1 > model-color-format=0 > network-mode=2 > num-detected-classes=1 > interval=0 > gie-unique-id=1 > output-blob-names=output\_bbox/BiasAdd;output\_cov/Sigmoid > [class-attrs-all] > pre-cluster-threshold=0.2 > group-threshold=1 > eps=0.2 ---
**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\ **Post date:** [January 21, 2021, 3:05am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/3 "2021-01-21T03:05:45Z")
Hi, If the osd works correctly, you can check the object meta from here: > Thanks. ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [January 27, 2021, 12:10pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/5 "2021-01-27T12:10:09Z")
Can you specify how to print all object meta? ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [January 28, 2021, 9:49am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/6 "2021-01-28T09:49:11Z")
I reinstalled everything and moved project from /opt/nvidia/deepstream/deepstream-5.0/… to home directory and copied …/deepstream\_python\_apps/apps/common with engine, labels and configs. I made this because had problems with permissions. Now it’s working, I have my roi In ObjectMeta. ---
**Author:** ![Andrey1984](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andrey1984/32/14130_2.png) [@Andrey1984](https://forums.developer.nvidia.com/u/Andrey1984)\ **Post date:** [January 29, 2021, 8:51am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/7 "2021-01-29T08:51:06Z")
Hi guys, how is it going? Could you extend on how to try the facedetectIR with deepstream, please? Is it just a matter of running the command with proper config? which exactly config? Does it require to generate/ download the faceIR model somehow? Thanks ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [January 29, 2021, 9:31am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/8 "2021-01-29T09:31:57Z")
Hi, first you can download pruned version [FacedetectIR](https://ngc.nvidia.com/catalog/models/nvidia:tlt_facedetectir/files?version=pruned_v1.0) following README at `/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models` than you are checking that everything is ok launching `deepstream-app -c deepstream_app_source1_facedetectir.txt `. You’ll have to carefully write paths (don’t write full path like `/opt/nvidia..`this can brake due to bug) to etlt file, labeles etc. After first launch you’ll get your tlt engine `resnet18_facedetectir_pruned.etlt_b1_gpu0_fp16.engine` check that your path to engine is correct with new engine (it should be fp16 not int8). Than I installed python API for deepstream and used deepstream-image-multistreamer example. And corrected program for config, which worked with deepstream-app. Main problem was with permissions I moved my project with dependencies to `/home`, because i had problems with permissions which prevented me from gaining my output data) Here is my gstreamer [pipeline](https://forums.developer.nvidia.com/t/detection-on-mjpeg-stereocamera-failure/160501/6) for FacedetectIR ---
**Author:** ![Andrey1984](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andrey1984/32/14130_2.png) [@Andrey1984](https://forums.developer.nvidia.com/u/Andrey1984)\ **Post date:** [January 30, 2021, 7:59am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/9 "2021-01-30T07:59:49Z")
@pilotfdd Thank you very much! It works up to the point of execution of the step below ```auto deepstream-app -c deepstream_app_source1_facedetectir.txt ``` However, given the two downloaded files are ```auto resnet18_facedetectir_pruned.etlt facedetectir_int8.txt ``` that are the result of executing of the readme command below: ```auto mkdir -p ../../models/tlt_pretrained_models/facedetectir && \ wget https://api.ngc.nvidia.com/v2/models/nvidia/tlt_facedetectir/versions/pruned_v1.0/files/resnet18_facedetectir_pruned.etlt \ -O ../../models/tlt_pretrained_models/facedetectir/resnet18_facedetectir_pruned.etlt && \ wget https://api.ngc.nvidia.com/v2/models/nvidia/tlt_facedetectir/versions/pruned_v1.0/files/facedetectir_int8.txt \ -O ../../models/tlt_pretrained_models/facedetectir/facedetectir_int8.txt ``` why should the engine file named ```auto resnet18_facedetectir_pruned.etlt_b1_gpu0_fp16.engine ``` be created? I can see there are only these files: ```auto /opt/nvidia/deepstream/deepstream-5.0/samples/models/tlt_pretrained_models/facedetectir/resnet18_facedetectir_pruned.etlt /opt/nvidia/deepstream/deepstream-5.0/samples/models/tlt_pretrained_models/facedetectir/resnet18_facedetectir_pruned.etlt_b1_gpu0_int8.engine ``` trying the other pipeline with ```auto /opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models$ cp ../../streams/sample_1080p_h264.mp4 test.mp4 gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/config_infer_primary_facedetectir.txt batch-size=1 unique-id=1 ! nvtracker ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! nvoverlaysink ``` it works But it doesn’t seem to detect faces on played videos, does it? the original example also comes redacted already so that the detection doesn’t seem to happen, right? It seems to boundebox faces if running ```auto deepstream-app -c /opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/deepstream_app_source1_peoplenet.txt ``` but doesn’t seem to show detection or draw boxes if running ```auto deepstream-app -c /opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/deepstream_app_source1_facedetectir.txt ``` Another complication is that it doesn’t seem to support input from vl4 usb camera? ```auto deepstream-app -c /opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/deepstream_app_source1_peoplenet.txt (deepstream-app:26591): GStreamer-CRITICAL **: 00:28:24.050: passed '0' as denominator for `GstFraction' (deepstream-app:26591): GStreamer-CRITICAL **: 00:28:24.050: passed '0' as denominator for `GstFraction' ** ERROR: : Failed to link 'src_elem' (image/jpeg; video/mpeg, mpegversion=(int)4, systemstream=(boolean)false; video/mpeg, mpegversion=(int)2; video/mpegts, systemstream=(boolean)true; video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[1, 32768], height=(int)[1, 32768], framerate=(fraction)[0/1, 2147483647/1]; video/x-dv, systemstream=(boolean)true; video/x-h263, variant=(string)itu; video/x-h264, stream-format=(string){ byte-stream, avc }, alignment=(string)au; video/x-pwc1, width=(int)[1, 32768], height=(int)[1, 32768], framerate=(fraction)[0/1, 2147483647/1]; video/x-pwc2, width=(int)[1, 32768], height=(int)[1, 32768], framerate=(fraction)[0/1, 2147483647/1]; video/x-raw, format=(string){ RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }, width=(int)[1, 32768], height=(int)[1, 32768], framerate=(fraction)[0/1, 2147483647/1]; video/x-sonix, width=(int)[1, 32768], height=(int)[1, 32768], framerate=(fraction)[0/1, 2147483647/1]; video/x-vp8; video/x-vp9; video/x-wmv, wmvversion=(int)3, format=(string)WVC1) and 'src_cap_filter1' (video/x-raw, width=(int)0, height=(int)0) ** ERROR: : create_camera_source_bin failed ** ERROR: : create_pipeline failed ** ERROR: : Failed to create pipeline Quitting App run failed ``` @pilotfdd However, if you could share the code to save /cut faces from video it might be helpful to try / for reference. Thank you very much Any ideas on how can we use the FaceDetectIR/ PeopleNet with test4/test5 apps that have mqqt option so that data gets to AWS from the detection via the MQQT ? Also any idea how to get it to work with usb camera? Probably it doesn’t work with camera/ default sample file doe to wrong resolution? Does FaceDecectIR require input files to be necessarily 384 X 240 ? so far I tried with 1080p inputs that did not work. ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [February 2, 2021, 8:38am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/10 "2021-02-02T08:38:59Z")
> [@Andrey1984](#): > > why should the engine file named > > ```auto > resnet18_facedetectir_pruned.etlt_b1_gpu0_fp16.engine > > ``` Maybe you are using other device? Which can work with int8 precision, than you don’t have to care about this. > [@Andrey1984](#): > > But it doesn’t seem to detect faces on played videos, does it? > the original example also comes redacted already so that the detection doesn’t seem to happen, right? If i understood correctly, yes. There is no detection, because there is no faces on that video, you can upload your video in mp4 format with faces and everything should work nicely. > [@Andrey1984](#): > > Another complication is that it doesn’t seem to support input from vl4 usb camera? Check your configuration for camera resolution carefully with `v4l2-ctl -d /dev/video1 --list-formats-ext` seems like problem with this. I had problems with webcams, because I’m using MJPEG format. But as I remember, when I’ve made fake cam with v4l2loopback, deepstream-app worked correctly with tracking. > [@Andrey1984](#): > > @pilotfdd However, if you could share the code to save /cut faces from video it might be helpful to try / for reference. ``` def crop_face(image,obj_meta,confidence): rect_params=obj_meta.rect_params top=int(rect_params.top) left=int(rect_params.left) width=int(rect_params.width) height=int(rect_params.height) image=image[top:height+top,left:width+left] return image ``` > [@Andrey1984](#): > > Any ideas on how can we use the FaceDetectIR/ PeopleNet with test4/test5 apps that have mqqt option so that data gets to AWS from the detection via the MQQT ? Didn’t looked in that direction much. > [@Andrey1984](#): > > Does FaceDecectIR require input files to be necessarily 384 X 240 ? No the resolutin doesn’t affect it, but aspect ratio seems to, because roi different for different aspect ratios. And i don’t know how to deal with that, yet. ---
**Author:** ![Andrey1984](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andrey1984/32/14130_2.png) [@Andrey1984](https://forums.developer.nvidia.com/u/Andrey1984)\ **Post date:** [February 2, 2021, 11:51am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/12 "2021-02-02T11:51:41Z")
@pilotfdd I have managed both peoplenet/facedetectIR to take as input rtsp stream that is generated withe with zed camera ```auto gst-zed-rtsp-launch -a 192.168.x.x zedsrc ! videoconvert ! 'video/x-raw, format=(string)I420' ! omxh265enc ! rtph265pay name=pay0 pt=96 name=pay0 ZED RTSP Server ----------------- * Stream ready at rtsp://192.168.x.x:8554/zed-stream ``` running the peoplenet will indicate detected face/person, the same works with facedetectIR after enabligh the sink but where to put the cut code for execution? it needs to go to some specific file? config file? else? separate .c/ cpp file? that needs to be built? also to load opencv libraries, also read from deepstream outputs? e.g. with gstreamer rtsp opencv wrapper , then execute the code as a separate application? or it could be implemented without running the deepstream app but with the gstreamer pipeline mentined previously, but the crop/save face will still require to be build in a separate file? Could you extend where/how to apply the facecut code, please? ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [February 3, 2021, 9:14am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/13 "2021-02-03T09:14:14Z")
As I know there a several ways to do that. First and best to write a gstreamer plugin in C/C++, Python, Vala etc. Like in this [article](https://medium.com/@karthickai/using-gstreamer-plugin-to-extract-detected-object-from-nvidia-deepstream-104879cc7154). Second write a program which will launch your pipeline in C/C++, Python. You can take as example this [program](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/5cb4cb8be92e079acd07d911d265946580ea81cd/apps/deepstream-imagedata-multistream/deepstream_imagedata-multistream.py#L70) like i did, rewrite pipeline, change config file, and instead of draw\_bounding\_boxes use my crop\_face function. ---
**Author:** ![Andrey1984](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andrey1984/32/14130_2.png) [@Andrey1984](https://forums.developer.nvidia.com/u/Andrey1984)\ **Post date:** [February 11, 2021, 2:24am UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/14 "2021-02-11T02:24:15Z")
@pilotfdd could you share exact steps, please? so it won’t be the deepstream-meta-test or deepstream-app -c “facedetectIR config file” but rather deepstream\_imagedata-multistream.py to execute? lines will need to be modified only in this file or in multiple? the pipeline you are reffering to is the gstreamer pipeline probably that runs with deepstream without executing binaries probably? this [code](https://stackoverflow.com/a/62142678) does it without deepstream [saving cropped faces] ```auto import cv2 import os classifier = cv2.CascadeClassifier(cv2.data.haarcascades+"haarcascade_frontalface_default.xml") dirFace = 'cropped_face' # Create if there is no cropped face directory if not os.path.exists(dirFace): os.mkdir(dirFace) print("Directory " , dirFace , " Created ") else: print("Directory " , dirFace , " has found.") webcam = cv2.VideoCapture(0) # Camera 0 according to USB port # video = cv2.VideoCapture(r"use full windows path") # video path while (True): (f, im) = webcam.read() # f returns only True, False according to video access # (f, im) = video.read() # video if f != True: break # im=cv2.flip(im,1,0) #if you would like to give mirror effect # detectfaces faces = classifier.detectMultiScale( im, # stream scaleFactor=1.10, # change these parameters to improve your video processing performance minNeighbors=20, minSize=(30, 30) # min image detection size ) # Draw rectangles around each face for (x, y, w, h) in faces: cv2.rectangle(im, (x, y), (x + w, y + h),(0,0,255),thickness=2) # saving faces according to detected coordinates sub_face = im[y:y+h, x:x+w] FaceFileName = "cropped_face/face_" + str(y+x) + ".jpg" # folder path and random name image cv2.imwrite(FaceFileName, sub_face) # Video Window cv2.imshow('Video Stream',im) key = cv2.waitKey(1) & 0xFF # q for exit if key == ord('q'): break webcam.release() ``` However, it would be great to try saving cropped faces with TensorRT use also ---
**Author:** ![pilotfdd](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@pilotfdd](https://forums.developer.nvidia.com/u/pilotfdd)\ **Post date:** [February 18, 2021, 12:49pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/15 "2021-02-18T12:49:46Z")
I don’t think I should share all the code but the steps are: Take **deepstream\_imagedata-multistream.py** program Change pipeline creation as you need (In my case I used MJPEG cams) Change line 331 as `pgie.set_property('config-file-path', "config_infer_primary_facedetectir.txt")` I’ve met problems with permissions in /opn/nvidia/deepstream/… so i moved all which files - facedetectir\_int8.txt - labels.txt - resnet18\_facedetectir\_pruned.etlt - resnet18\_facedetectir\_pruned.etlt.1 - resnet18\_facedetectir\_pruned.etlt\_b1\_gpu0\_fp16.engine in another directory at home in directory with python script And i had to copy `common` directory with this files - bus\_call.py - FPS.py - ` __init__.py` - is\_aarch\_64.py - ` __pycache__ ` - utils.py in same directory. Changed sys in `deepstream_imagedata-multistream.py` according to this changes. And all destinations in `config_infer_primary_facedetectir.txt` If program is running then you will rewrite function where you are getting frame buffer in numpy format. `def tiler_sink_pad_buffer_probe(pad,info,u_data):` ---
**Author:** ![Andrey1984](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andrey1984/32/14130_2.png) [@Andrey1984](https://forums.developer.nvidia.com/u/Andrey1984)\ **Post date:** [February 18, 2021, 1:26pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/16 "2021-02-18T13:26:31Z")
@pilotfdd Thank you for sharing! I will try to follow your steps. However, for me it turned out less complicated to use plain opencv rather than figure out complications of deepstream implementation ```auto xhost + docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY --privileged --ipc=host -v /tmp/.X11-unix/:/tmp/.X11-unix/ -v /tmp/argus_socket:/tmp/argus_socket -v /dev/video2:/dev/video2 --cap-add SYS_PTRACE iad.ocir.io/idso6d7wodhe/jetson_nx/opencv541 /bin/bash -c ' cp -r /usr/local/opencv-4.5.1-dev/lib/python3.6/dist-packages/cv2 /usr/lib/python3.6/dist-packages/cv2 && export OPENCV_VERSION=opencv-4.5.1-dev && export LD_LIBRARY_PATH=/usr/local/$OPENCV_VERSION/lib && wget https://raw.githubusercontent.com/AndreV84/Jetson/master/usbcam.py && ls && pwd && python3 usbcam.py' ``` this one just saves cropped face images from the camera [ given usb camera has index /dev/video2. e.g. it can be used to save to temporary container [destroyed after launch] or could save to mount to a folder [ requires adding a folder e.g. `-v /path/path` also `cd path &&` ---
**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\ **Post date:** [October 12, 2021, 4:15pm UTC](https://forums.developer.nvidia.com/t/output-tensor-in-facedetectir/166311/17 "2021-10-12T16:15:09Z")
Closed on October 12, 2021, 4:15pm UTC