TX1 ISP Queries

Autonomous Machines Jetson & Embedded Systems Jetson TX1
21

For the raw capture current only support v4l2-ctl bypass mode only.
like below command to capture the raw data.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=ov1080.raw
22

Hi,

Is it possible to use the TX1 ISP in the following way?

Memory -> ISP -> Memory

Right now the libargus APIs as well as nvcamerasrc plugin uses the ISP in the following way.

CSI -> VI -> ISP -> Memory

From what I saw in the jetpack multimedia API webinar, the ISP has support for input from either VI or Memory. But there is no example demonstrating this. Can somebody from Nvidia confirm that this option is feasible in the hardware level. And if yes, will it be supported?

External Media

23

That pipeline is possible however the current release Argus are not support it. Need time to check if we have plan to implement it and will update it to you.

24

Thank you for the quick response ShaneCCC.

25

Hi dilipkumar
We did have plan to support your use case. But the schedule are not settle now.

26

Thanks for the update ShaneCCC. We can hope to see some examples to use the ISP in the manner I described, in a future release of L4T?

27

I have the same question/request. Would really like to use the ISP from memory/buffers allocated by the application. That memory could be fed from CSI cameras using V4L2 or other methods as well as GigE/USB cameras (which would allow some prototyping with different image sensors).

Also it would allow testing of the demosaic and other functions without having any camera present at all. A sample application that takes a raw Bayer image from a file, demosaics it, with/without first applying Bayer domain functions, shows use of Bayer conversion functions and shows the RGB/YUV functions as well, finally saving it to disk would be much appreciated.

Now that JetPack 3 is released, is there a roadmap or schedule for allowing the pipeline described earlier in this thread?

28

Hi tpjg
The GigE/USB camera already YUV data why do you still need ISP?

29

For easy prototyping, for example try some image sensors that are not (yet) available in a camera with a CSI interface. Test them with the TX1/TX2. Most (industrial) cameras can output the raw Bayer data as well as YUV or RGB.

30

Hi Shane,

Just wanted to add my two pennies. Not everyone requires an ISP just so that they can convert RAW to YUV. Other functionality provided by the ISP could be of interest. The factors mentioned by tpjg are very much commendable. Other factors which help include optimal power consumption, due to non requirement of additional hardware. The image quality provided by TX1 ISP might also be better than other properitory ISP chipsets. We will never know for sure, unless we have an opportunity to test it in an easy manner.

The option to use the memory to ISP pipeline would be very much useful for developers such as myself for easy testing and image quality comparisons.

31

Hi
There’s virtual video driver are going implement now. If there’s no accident. The plan to public will happen at next release.

32

Thanks for the update ShaneCCC. Looking forward to the next L4T/Jetpack release for TX1.

33

Hi, Shane

Sorry for digging up this old thread. We are still interested in seeing the virtual video driver support and I cannot find it in the 28.1 release. Do you have some reference document for it?

thanks
Xiaoyong

34

@xiaoyongtijee
I would like to apologize for this feature is delay due to developer resource is much tighter to complete this feature.

35

Thanks a lot for the update Shane. We’ll focus one the existing mipi/csi path for now.
It will make our hardware/software design more flexible if we have the virtual video driver feature. Do you know if we will still get it soon?

thanks
Xiaoyonog

36

Hi,
Sorry for bringing up an old issue. Can someone from Nvidia provide any update regarding this? Is MEMORY → ISP image pipeline supported in the latest L4T release for TX1 or atleast for TX2 ?

37

Hi,

I like to bump this thread. Is MEMORY → ISP supported now or are there plans to support it? Is there a low level ISP API available so I could implement this pipeline even Argus does not support it?

38

@arne
No any API can support this case in current release.
The plan is on going but I can’t tell when will ready.

39

Hi all
I am using Tx2 with Jetpack 3.2
My goal is capture the raw data from the radar sensor and store it in memory
when i using following v4l2-ctl command i am able to capture the raw data

v4l2-ctl --set-fmt-video=width=8192,height=384,pixelformat=“RG8” --stream-mmap --stream-count=100 -d /dev/video0 --stream-to=file.raw

But I am not sure whether i am getting the raw data directly from VI or not

The problem is i dont want ISP here, I want to get the raw data directly from CSI->VI path
I am not aware of Internal pipelining of capturing raw data,i Just want to know whether the above command is really bypassing ISP or not, If the the above command still uses ISP, Guide me how to disable/stop the functionalities of ISP completely (or) Is there any other v4l2 command to stop the functionalities of ISP

Thank you

40

if you add --set-ctrl bypass_mode=0 it should bypass ISP