PIP not able to download the AzureOpenAI for LangChain LLMS

Omniverse Developer
1

Hi,

I am trying to integrate the langchain llms hosted on Azure in a custom extension for USD Composer. But somehow the AzureOpenAI in langchain is not getting downloaded with the PIP.

I tried all the possible ways to download it but somehow its not downloading the full/correct version of LangChain

[python.pipapi]
requirements = [“openai”, “langchain[llms]”]
#requirements = [“openai”, “langchain[all]”]
#requirements = [“openai”, “langchain”]
#requirements = [“openai”, “langchain==0.0.228”]
use_online_index = true

2023-07-10 07:52:43 [Error] [omni.ext.python] ImportError: cannot import name 'AzureOpenAI' from 'langchain.llms' (C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\Create.Next\2022.3\pip3-envs\default\langchain\llms\__init__.py)

2

Hi @Pranav.Buradkar. Let me check with the dev team on this.

3

Hi @Pranav.Buradkar. While we’re waiting to hear back from the dev team, could you try on the latest version of USD Composer 2023? That version has Python 3.10 instead of 3.7. Isaac Sim would be upgrading to that in the near future too.

4

Sure, I will test that.

5

What I would recommend is installing pip packages you need outside of the app process, using the python we include or system python of the same version. When doing pip install you can pass --target=[path] to install into a specific folder. This is what omni.kit.pipapi extension is doing if you look inside.

Then you can add that path either explicitly to sys.path or just make it one of the extensions. This is how extensions like omni.kit.pip_archive are made, we install at build time and prepackage all packages in the extensions.

Another thing to know is that pip installation errors go to stdout and if you run from the Launcher, you won’t see them (to be fixed in next versions). You can run an app from terminal, using sh/bat file (like omnu.create.sh) then you would probably have more information on what failed. Run with --clear-data to make sure it tries to install again.

6

@anovoselov

I checked with 2022 and 2023 versions of Create and still facing the same issue. I tried with installing the LangChain in external as well as in internal directory and provided those as explicit import paths but facing the same issue.

I am able to import langchain on command line with the provided versions of python in omniverse, but its not working with script editor or as an extension.

import sys
# appending a path Create 2023
#sys.path.append(r'C:/Users/pranav.buradkar/AppData/Local/ov/data/Kit/USD.Composer/2023.1/pip3-envs/default')
#sys.path.append(r'C:/Users/pranav.buradkar/AppData/Local/ov/data/Kit/USD.Composer/2023.1/pip3-envs/default/langchain')

# appending a path Create 2022
# sys.path.append(r'C:/Users/pranav.buradkar/AppData/Local/ov/data/Kit/Create.Next/2022.3/pip3-envs/default')
# sys.path.append(r'C:/Users/pranav.buradkar/AppData/Local/ov/data/Kit/Create.Next/2022.3/pip3-envs/default/langchain')

print (f"path ---> {sys.path}")

import omni.ext
import omni.ui as ui

#import openai
from langchain.llms import AzureOpenAI
#from langchain.llms.openai import AzureOpenAI, OpenAI

LOG -

[35.172s] RTX ready
[141.846s] [ext: omni.kit.registry.nucleus-0.0.0] startup
[154.293s] [ext: omni.hello.world-1.0.0] startup
2023-07-18 06:27:52 [154,209ms] [Warning] [omni.kit.pipapi.pipapi] extension omni.hello.world-1.0.0 has a [python.pipapi] entry, but use_online_index=true is not set. It doesn't do anything and can be removed.
path --->   ['c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\kernel\\plugins\\python310.zip', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\DLLs', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\lib', 'C:\\Users\\pranav.buradkar\\AppData\\Local\\ov\\pkg\\create-2023.1.1\\kit', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\lib\\site-packages', '.', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/plugins/bindings-python', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/bindings-python']
2023-07-18 06:27:52 [154,247ms] [Error] [omni.ext._impl.custom_importer] Failed to import python module omni.hello.world. Error: No module named 'langchain'. Traceback:
Traceback (most recent call last):
  File "c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\custom_importer.py", line 76, in import_module
    return importlib.import_module(name)
  File "c:\users\pranav.buradkar\appdata\local\ov\pkg\create-2023.1.1\kit\python\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/__init__.py", line 1, in <module>
    from .extension import *
  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/extension.py", line 18, in <module>
    from langchain.llms import AzureOpenAI
ModuleNotFoundError: No module named 'langchain'

2023-07-18 06:27:52 [154,249ms] [Error] [carb.scripting-python.plugin] Exception: Extension python module: 'omni.hello.world' in 'c:\projects\omniverse\kit-extension-template-main_test_langchain\exts\omni.hello.world' failed to load.

At:
  c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\_internal.py(212): startup
  c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\_internal.py(323): startup_extension
  PythonExtension.cpp::startup()(2): <module>
  c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.kit.window.extensions-1.1.9+ed961c5c/omni/kit/window/extensions/ext_commands.py(29): do
  c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.kit.commands-1.4.6+ed961c5c.wx64.r.cp310/omni/kit/undo/undo.py(80): execute
  c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.kit.commands-1.4.6+ed961c5c.wx64.r.cp310/omni/kit/commands/command.py(463): execute
  c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.kit.window.extensions-1.1.9+ed961c5c/omni/kit/window/extensions/common.py(288): toggle_extension
  c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.kit.window.extensions-1.1.9+ed961c5c/omni/kit/window/extensions/ext_components.py(89): toggle

2023-07-18 06:27:52 [154,249ms] [Error] [omni.ext.plugin] [ext: omni.hello.world-1.0.0] Failed to startup python extension.

EDIT-

I have changed the sys.append path to Windows style, I think its able to find the module but still cant locate the langchain.llms


  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/extension.py", line 20, in <module>
    from langchain.llms import AzureOpenAI

  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/extension.py", line 21, in <module>
    from langchain.llms.openai import AzureOpenAI, OpenAI

Full Log -

[435.338s] [ext: omni.hello.world-1.0.0] shutdown
[435.368s] [ext: omni.hello.world-1.0.0] startup
2023-07-18 07:03:11 [435,300ms] [Warning] [omni.kit.pipapi.pipapi] extension omni.hello.world-1.0.0 has a [python.pipapi] entry, but use_online_index=true is not set. It doesn't do anything and can be removed.
path --->   ['c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\kernel\\plugins\\python310.zip', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\DLLs', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\lib', 'C:\\Users\\pranav.buradkar\\AppData\\Local\\ov\\pkg\\create-2023.1.1\\kit', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python', 'c:\\users\\pranav.buradkar\\appdata\\local\\ov\\pkg\\create-2023.1.1\\kit\\python\\lib\\site-packages', '.', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/plugins/bindings-python', 'c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/bindings-python', 'C:\\Users\\pranav.buradkar\\AppData\\Local\\ov\\data\\Kit\\USD.Composer\\2023.1\\pip3-envs\\default', 'C:\\Users\\pranav.buradkar\\AppData\\Local\\ov\\data\\Kit\\USD.Composer\\2023.1\\pip3-envs\\default\\langchain']
2023-07-18 07:03:11 [435,311ms] [Error] [omni.ext._impl.custom_importer] Failed to import python module omni.hello.world. Error: cannot import name 'NotRequired' from 'typing_extensions' (c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.services.pip_archive-0.6.0+wx64/pip_prebundle/typing_extensions.py). Traceback:
Traceback (most recent call last):
  File "c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\custom_importer.py", line 76, in import_module
    return importlib.import_module(name)
  File "c:\users\pranav.buradkar\appdata\local\ov\pkg\create-2023.1.1\kit\python\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/__init__.py", line 1, in <module>
    from .extension import *
  File "c:/projects/omniverse/kit-extension-template-main_test_langchain/exts/omni.hello.world/omni/hello/world/extension.py", line 20, in <module>
    from langchain.llms import AzureOpenAI
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\__init__.py", line 6, in <module>
    from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\agents\__init__.py", line 2, in <module>
    from langchain.agents.agent import (
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\agents\agent.py", line 40, in <module>
    from langchain.utilities.asyncio import asyncio_timeout
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\utilities\__init__.py", line 3, in <module>
    from langchain.utilities.apify import ApifyWrapper
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\utilities\apify.py", line 5, in <module>
    from langchain.document_loaders import ApifyDatasetLoader
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\document_loaders\__init__.py", line 45, in <module>
    from langchain.document_loaders.embaas import EmbaasBlobLoader, EmbaasLoader
  File "C:\Users\pranav.buradkar\AppData\Local\ov\data\Kit\USD.Composer\2023.1\pip3-envs\default\langchain\document_loaders\embaas.py", line 7, in <module>
    from typing_extensions import NotRequired, TypedDict
ImportError: cannot import name 'NotRequired' from 'typing_extensions' (c:/users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/extscache/omni.services.pip_archive-0.6.0+wx64/pip_prebundle/typing_extensions.py)

2023-07-18 07:03:11 [435,312ms] [Error] [carb.scripting-python.plugin] Exception: Extension python module: 'omni.hello.world' in 'c:\projects\omniverse\kit-extension-template-main_test_langchain\exts\omni.hello.world' failed to load.

At:
  c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\_internal.py(212): startup
  c:\users/pranav.buradkar/appdata/local/ov/pkg/create-2023.1.1/kit/kernel/py\omni\ext\_impl\_internal.py(323): startup_extension
  PythonExtension.cpp::startup()(2): <module>

2023-07-18 07:03:11 [435,313ms] [Error] [omni.ext.plugin] [ext: omni.hello.world-1.0.0] Failed to startup python extension.
7

That probably happens because typing_extensions that we include is of an older version and it is imported first.
You probably can work it around if you del sys.modules["typing_extensions"] before importing langchain. That will triggers re-import and If the right version is higher on a sys.path it should be picked up.

8

Still the same issue persists.