Adding persistance to text-generation-webui (as packaged by dusty_nv in jetson-containers)

Autonomous Machines Jetson & Embedded Systems Jetson Projects
1

I have been using the @dusty_nv jetson-containers project with my Jetson Orin NX 16GB. Mostly I use the text-generation-webui, but loosing my settings each time I restart was bothering me. Here is a quick and dirt solution that worked for me.

# The assumptions:
    - jetson-containers is installed
    - you can run text-generation-webui

# The issue we are resolving:
    - Each time the container is stopped the user settings are lost

# Procedure to add persistance to settings, characters and presets
  cd ~/jetson-containers

  # We need to have the container running during this phase
  # Note the -d flag to detach the container as we need the command prompt
  ./run.sh -d $(./autotag text-generation-webui)

  # Let’s get the container ID
  docker ps 

  # Create an environment variable to make our live simpler
  export CONTAINER_ID=<container_id>

  # Create directories to store settings on the jetson host
  mkdir -p data/settings/text-generation-webui

  # Move to new directory to simplify next step
  cd  data/settings/text-generation-webui

  # Copy files and directories
  docker cp $CONTAINER_ID:/opt/text-generation-webui/characters .
  docker cp $CONTAINER_ID:/opt/text-generation-webui/presets .
  docker cp $CONTAINER_ID:/opt/text-generation-webui/settings.yaml .

  # Stop and remove the container
  docker stop $CONTAINER_ID

  # Let's launch the service with persistence activated
  cd ~/jetson-containers
  ./run.sh -v $PWD/data/settings/text-generation-webui/settings.yaml:/opt/text-generation-webui/settings.yaml -v $PWD/data/settings/text-generation-webui/presets/:/opt/text-generation-webui/presets/ -v $PWD/data/settings/text-generation-webui/characters/:/opt/text-generation-webui/characters/ $(./autotag text-generation-webui)
1 Like