pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded

When running bentoml serve after building my bento I get the following error:

Traceback (most recent call last):
  File "/home/tom/Desktop/ml-reconciliation/venv/bin/bentoml", line 8, in <module>
    sys.exit(cli())
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 362, in wrapper
    return func(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 333, in wrapper
    return_value = func(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 290, in wrapper
    return func(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/env_manager.py", line 122, in wrapper
    return func(*args, **kwargs)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/serve.py", line 260, in serve
    serve_http_production(
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/serve.py", line 327, in serve_http_production
    json.dumps(runner.scheduled_worker_env_map),
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 356, in scheduled_worker_env_map
    for worker_id in range(self.scheduled_worker_count)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 341, in scheduled_worker_count
    return self.scheduling_strategy.get_worker_count(
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/strategy.py", line 68, in get_worker_count
    resource_request = system_resources()
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/resource.py", line 46, in system_resources
    res[resource_kind] = resource.from_system()
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/resource.py", line 248, in from_system
    pynvml.nvmlInit()
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 1770, in nvmlInit
    nvmlInitWithFlags(0)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 1760, in nvmlInitWithFlags
    _nvmlCheckReturn(ret)
  File "/home/tom/Desktop/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 833, in _nvmlCheckReturn
    raise NVMLError(ret)
pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded

Environment :
BentoML: 1.1.11
Python: 3.10
torch: 2.2.1
Ubuntu: 22.04
no Nvidia GPU

Thank you in advance

Since you have no GPU, its not surprising that tehre is no GPU driver loaded. I’m not familiar with BentoML, and I doubt this is the forum to ask BentoML questions, but if that software stack (whatever it is) is configured to use or is expecting to use a GPU, then you probably need to get a different version of the software stack that doesn’t require a GPU, or else try it on a system with a properly installed GPU driver.

You may wish to ask questions about using BentoML here.

Thanks for your answer,
BentoMl needs those drivers regarless of having a GPU or not, a co-worker had the same issue with his pc that doesn’t have a GPU either. He was able to install the drivers and it solves the problem on his side. I followed the same steps, I have now some nvidia drivers installed but it looks like they are not detected.

$ dkms status
nvidia/470.239.06, 6.5.0-27-generic, x86_64: installed

Thank you for the link to the BentoML slack, I also submitted my issue there and I’m waiting for their answer