Adding extensions to the containerized text-generation-webui is a bit tricky because the container looses all modifications once stopped. I inherited @dusty_nv container and created a new image to add the five extensions below, the only modifications is the addition of those extensions.
-
code_syntax_highlight
- it formats code that might be present on chat responses making it easy to read
- source: GitHub - DavG25/text-generation-webui-code_syntax_highlight: An extension for oobabooga's text-generation-webui that adds syntax highlighting to code snippets
-
chatbot_clinic
- it allows creating multiple prompts and compare the results to fine tune system prompts
- source: GitHub - p-e-w/chatbot_clinic: Science-driven chatbot development
-
dynamic_context
- it adds behind the scene current date and time context to improve llm performance
- source: GitHub - elPatrixF/dynamic_context: An extension for oobabooga´s Text Generation WebUI
-
hello_outside_world
- if the user enters an url during chat it reads the page and shows on screen the llm summarizes it
- source: GitHub - src-r-r/hello_outside_world: text-generation-webui extension that parses any links and summarizes them, allowing the AI assistant to use the content for answers.
- LLM_Web_search
- it adds capability to web search to the llm dramatically improving the usefulness of llm generative ai chat
- source: GitHub - mamei16/LLM_Web_search: An extension for oobabooga/text-generation-webui that enables the LLM to search the web using DuckDuckGo
I uploaded the new image to hub.docker.com. You can find it at rpontual/text-generation-webui. The inherited image is the image from dusty/text-generation-webui:36.2.0 (digest d4d006425e00 from 2/3/24) as such it requires JetPack 6. To run it use the same command you use to run the @dusty_nv version except that the image needs to start with rpontual/ (e.g. " ./run.sh $(./autotag rpontual/text-generation-webui)")
----- Fortune favors the bold… -----
In case you are interested in creating your own derived images, these are the steps I took initially to add extensions:
- Open terminal window, ssh to Jetson, launch the default text-generation-webui
- Go to a browser in your computer, reach the text-generation-webui with your browser
- Navigate to Session tab and add load the extension using the “Install or update an extension”
- Open a second terminal window, ssh to Jetson, execute Docker PS to get container ID
- docker exec -it bash
- inside the container go to /opt/text-generation-webui/extensions
- locate the directory with the extension name and enter it (cd <extension_name>)
- look for requirements.txt and run pip install (pray for not getting into pip conflicts…)
- exit (to exit container)
- at the terminal execute “docker commit containerid /yourrepository/text-generation-webui:36.2.0”
- you can see your new image executing “docker images”