Display dynamic image(gif)

Omniverse General Discussion
1

Hi, I’m using kit to develope some assets management extension. One of my purpose is find a way to preview an animation asset( .usd file). While the simplest way to do that is upload a related .gif image, but I don’t know how to display that .gif image in Kit, or ImGUI, I’m using the Python API omni.ui, and not familiar to C/C++.
Can someone please give me some hints, thanks a lot

2

Hello @CeaserWayneCSW! Welcome to the Community! I will reach out to our development team for more help in answering your question.

I did a search… I do apologize for my programming ignorance… but I did find this information in our documentation: omni.ui.image. I just wanted to post it here in case it helps you move forward.

Also, if you haven’t seen it already, here is a link to our Kit Programming Manual

3

@CeaserWayneCSW There is no gif support for omni.ui currently. If you want to support that, you’d have build your own widget. What I can think of is:

  1. You need to gif read library for python for read all frames.
  2. You can register update event omni.kit.app.get_app().get_update_event_stream().create_subscription_to_pop to update frames and control the timing by yourself.
4

Thanks, I’ll give it a try. Will reponse here if I can make it work^^

5

Hi @rozhang , here’s my understand of the solution:

  1. use PIL( or something else) to read all frames from a .gif.
  2. use create_subscription_to_pop() to registry a function to do the displaying work.

But got a problem. I’m trying some code like below:

from PIL import Image, ImageSequence

path = r'path/to/gifimage.gif'
window = ui.Window('gif image', width=450, height=450)
img = Image.open(path)

def on_update(e: carb.events.IEvent):
	for frame in ImageSequence.Iterator(img):
		window.frame.clear()
		with window.frame:
			ui.Image(frame)


subscription_holder = (
	omni.kit.app.get_app().get_update_event_stream().create_subscription_to_pop(on_update, name="My Subscription Name")
)

the ui.Image() only except an URL, but during the iteration, I can only provide the image data (for frame in ImageSequence.Iterator(img)), and I don’t know if there is an another way like “StringField().model” to provide these image data.

And yet, I have tried an unexceptable way just for test:

"""
for each frame in .gif, create a .jpg file for it
store in folder path  {gif_frames_jpeg}
"""
window = ui.Window('gif image', width=450, height=450)

async def on_update():
	with window.frame:
		for i in range(len(frames):
			path = r'path\to\gif_frames_jpeg\{}'.format(f'gif_frame_{i}.jpg')
			ui.Image(path)
			await asyncio.sleep(.04)

asyncio.ensure_future(on_update())

I can get an “animation” in this way, but it cause severe strobe between pictures(frames). So I’m wondering even if I solve the URL problem above, and I can pass the image data directly to ui.Image(), will it still cause this strobe situation?

At last, one more stupid question…How can I stop this update event? I have never use this carb library before…

6

@CeaserWayneCSW There is a byte provider omni.ui.ByteImageProvider, and you don’t need to clear window each frame.

self._window = ui.Window('gif image', width=256, height=256)
with self._window.frame:
    self._image_provider = ui.ByteImageProvider()
    self._image = ui.ImageWithProvider(self._image_provider, width=xxx, height=xxx, fill_policy=omni.ui.IwpFillPolicy.IWP_STRETCH)

def on_update(e: carb.events.IEvent):
    self._image_provider.set_bytes_data(your_frame_bytes_array, (width, height))

By the way, you only need to set the handle to None, then the subscription to update event will be released.

7

@rozhang Hi, is there any workable example or extension using this ByteImageProvider?
Need to know what exact type of <bytes_array> should passed to ByteImageProvider.set_bytes_data(), I’ve tried PIL.ImageSequnce/[subitem], Numpy.array.tolist()/tobytes(). None of these is working and sometime cause Kit hang or crash.

8 
import omni.ui as ui
from PIL import ImageSequence, Image


with Image.open("h:/1.png") as frame:
    window = ui.Window('gif image', width=256, height=256)
    with window.frame:
        image_provider = ui.ByteImageProvider()
        image_provider.set_bytes_data(frame.getdata(), frame.size)
        image = ui.ImageWithProvider(image_provider, width=256, height=256, fill_policy=omni.ui.IwpFillPolicy.IWP_STRETCH)

1

@CeaserWayneCSW Here is an example to display this PNG. So by default, the default format of the bytesarray is assumed to be either a set of MxNx4 ints/floats (RGBA) or a set of MxNx(sequence of 4 ints/floats) - i.e. in the second case, that would be array of 4-tuples.

There is also another function, set_raw_bytes_data in the ByteImageProvider , which consumes raw C++ memory pointers, and if you have a plugin which outputs C++ mem pointer for each GIF frame, then it may work fast.

1 Like
9

@rozhang
Thank you! Got it work now. It’s because each frame extracted from the gif animation is in ‘P’ mode, not ‘RGBA’ mode, so the BytesImageProvider.set_bytes_data() won’t take the parameter and even crash sometime.
I’m doing this using the asyncio.sleep to control the interval of each two frames, by using the ByteImageProvider, no strobe like before.
Here’s the snippet:

async def show_dynamic_image(path):
    global is_playing, task
    if is_playing:
        return
    try:
        is_playing = not is_playing
        image = Image.open(path)
        arr = ImageSequence.Iterator(image)
        while True:
            frame = next(arr)
            if frame.mode != 'RGBA':
                frame = frame.convert('RGBA')
            img_pvd.set_bytes_data(frame.getdata(), frame.size)
            await asyncio.sleep(0.01)
    except StopIteration as e:
        is_playing = not is_playing
        img_pvd.set_bytes_data(first_frame.getdata(), first_frame.size)
        task = None
1 Like
10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.