# Setting up nvidia-persistenced

**URL:** <https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986>\
**Category:** CUDA Setup and Installation\
**Created:** [February 20, 2017, 2:38pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986 "2017-02-20T14:38:32Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![muttlol](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@muttlol](https://forums.developer.nvidia.com/u/muttlol)\
**Post date:** [February 20, 2017, 2:38pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/1 "2017-02-20T14:38:32Z")

</div>

Hi,

I am running the latest CUDA on Ubuntu 16.04. I would like the persistence mode to be enabled at startup.

I see that the boot process is starting the persistence daemon with the persistence mode OFF:

nvidia-+ 1867 0.0 0.0 17100 1704 ? Ss 09:08 0:00 /usr/bin/nvidia-persistenced --user nvidia-persistenced --no-persistence-mode --verbose

Where is the configuration for this? How do I enable persistence mode at startup?

Thanks,  
M.

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [February 20, 2017, 2:42pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/2 "2017-02-20T14:42:12Z")

</div>

use the nvidia-smi tool.

run

nvidia-smi --help

---

<div class="post-metadata">

**Author:** ![muttlol](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@muttlol](https://forums.developer.nvidia.com/u/muttlol)\
**Post date:** [February 20, 2017, 3:26pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/3 "2017-02-20T15:26:11Z")

</div>

Hi,

thank you for the quick answer. Unfortunately, it doesn’t answer my question.

I am sorry I haven’t been clear enough, so I’ll reformulate.

I want to use nvidia-persistenced (not nvidia-smi) to set up my persistence mode.

If I try to do that from bash, it complains that there’s already a nvidia-persistenced daemon running with the persistence mode switched off. Indeed, if check with ps I get the following:

nvidia-+ 1867 0.0 0.0 17100 1704 ? Ss 09:08 0:00 /usr/bin/nvidia-persistenced --user nvidia-persistenced --no-persistence-mode --verbose

Now, I suppose that this daemon is started by Nvidia at boot time. I want the daemon to be started in the same way, but with the persistence mode on, so my questions are:

- Is there a Nvidia configuration file/script that I must edit to control the startup of the daemon?
- What is the syntax?

Thanks,  
M.

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [February 20, 2017, 3:34pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/4 "2017-02-20T15:34:01Z")

</div>

[url][Driver Persistence :: GPU Deployment and Management Documentation](http://docs.nvidia.com/deploy/driver-persistence/index.html#persistence-daemon%5B/url%5D)

---

<div class="post-metadata">

**Author:** ![muttlol](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@muttlol](https://forums.developer.nvidia.com/u/muttlol)\
**Post date:** [February 20, 2017, 4:29pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/5 "2017-02-20T16:29:41Z")

</div>

Ok, I may have found a solution, I did the following:

- Download the nvidia-persistenced archive from ftp://download.nvidia.com/XFree86/nvidia-persistenced
- Uncompress the archive  
\*Edit the nvidia-persistenced.conf.template in the subdirectory corresponding to your startup service (can be upstart, systemd or sysv)
- add “–persistence-mode --verbose” to the line where the service is started
- run install.sh to kill the currently running daemon and install the config with the new parameters

---

<div class="post-metadata">

**Author:** ![Matthew\_K](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Matthew\_K](https://forums.developer.nvidia.com/u/Matthew_K)\
**Post date:** [January 26, 2018, 11:39am UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/6 "2018-01-26T11:39:01Z")

</div>

> [@](#):
>
> Ok, I may have found a solution, I did the following:
> 
> - Download the nvidia-persistenced archive from ftp://download.nvidia.com/XFree86/nvidia-persistenced
> - Uncompress the archive  
> \*Edit the nvidia-persistenced.conf.template in the subdirectory corresponding to your startup service (can be upstart, systemd or sysv)
> - add “–persistence-mode --verbose” to the line where the service is started
> - run install.sh to kill the currently running daemon and install the config with the new parameters

Thanks for this.

I think adding “–persistence-mode --verbose” is unneeded. The daemon appears to enable PM by default.

On my system, the archive was here: /usr/share/doc/NVIDIA\_GLX-1.0/sample/nvidia-persistenced-init.tar.bz2. I didn’t need to download it. See [http://docs.nvidia.com/deploy/driver-persistence/index.html#installation](http://docs.nvidia.com/deploy/driver-persistence/index.html#installation) for more details.

My system: Ubuntu 16.04, NVIDIA 384.111.

---

<div class="post-metadata">

**Author:** ![andorjkiss](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andorjkiss/32/13041_2.png) [@andorjkiss](https://forums.developer.nvidia.com/u/andorjkiss)\
**Post date:** [December 13, 2019, 8:00pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/7 "2019-12-13T20:00:36Z")

</div>

Is there some kind of step-by-step guide to installing this?

$ sudo nvidia-persistenced  
nvidia-persistenced failed to initialize. Check syslog for more details.

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [December 13, 2019, 8:15pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/8 "2019-12-13T20:15:45Z")

</div>

[https://docs.nvidia.com/deploy/driver-persistence/index.html#usage](https://docs.nvidia.com/deploy/driver-persistence/index.html#usage)

---

<div class="post-metadata">

**Author:** ![andorjkiss](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andorjkiss/32/13041_2.png) [@andorjkiss](https://forums.developer.nvidia.com/u/andorjkiss)\
**Post date:** [December 19, 2019, 12:22pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/9 "2019-12-19T12:22:28Z")

</div>

No instructions there on how to do it; just a description of what it’s supposed to do. Running nvidia-persistenced as superuser spits an error (see above).

Or do I have to use like:

$ sudo nvidia-persistenced --user username

where username is my login?

---

<div class="post-metadata">

**Author:** ![andorjkiss](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andorjkiss/32/13041_2.png) [@andorjkiss](https://forums.developer.nvidia.com/u/andorjkiss)\
**Post date:** [December 19, 2019, 9:24pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/10 "2019-12-19T21:24:03Z")

</div>

> [@](#):
>
> No instructions there on how to do it; just a description of what it’s supposed to do. Running nvidia-persistenced as superuser spits an error (see above).
> 
> Or do I have to use like:
> 
> ```auto
> $ sudo nvidia-persistenced --user username
> 
> ```
> 
> where username is my login?

That did not work, however,

```auto
$ sudo -i

# nvidia-smi -pm 1

Persistence mode is already Enabled for GPU 00000000:21:00.0.
Persistence mode is already Enabled for GPU 00000000:31:00.0.
All done.

# exit

```

This did work in Ubuntu 18.04. Thanks!

---

<div class="post-metadata">

**Author:** ![mikechen6688](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mikechen6688](https://forums.developer.nvidia.com/u/mikechen6688)\
**Post date:** [July 18, 2020, 10:04am UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/11 "2020-07-18T10:04:30Z")

</div>

The setup of nvidia.persistenced is formulated as follows.

**1. check the status of nvidia-persistenced**

`$ sudo systemctl status nvidia-persistenced`

**2. Enable nvidia-persistenced**

`$ sudo systemctl enable nvidia-persistenced`

**3. Reboot for execution**

`$ sudo reboot`

If there are issues in the above-mentioned second step, developers need to do the following.

**4. Open nvidia-persistenced.service**

`$ sudo gedit /lib/systemd/system/nvidia-persistenced.service`

**5. Modify and add the content of the file**

**1). Modify the line in section of [Service]**

[Service]

Change

`ExecStart=/usr/bin/nvidia-persistenced --user nvidia-persistenced --no-persistence-mode --verbose`

To:

`ExecStart=/usr/bin/nvidia-persistenced --user nvidia-persistenced --persistence-mode --verbose`

**2). Add the following lines into the file:**

```
[Install] 
WantedBy=multi-user.target 
RequiredBy=nvidia.service

```

Notes:

The temporal method to set nvidia.persistenced is to use the following command:

$ sudo nvidia-smi -pm 1

Cheers

---

<div class="post-metadata">

**Author:** ![mikechen6688](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mikechen6688](https://forums.developer.nvidia.com/u/mikechen6688)\
**Post date:** [July 18, 2020, 10:24am UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/12 "2020-07-18T10:24:59Z")

</div>

My question is nvidia.persistenced brings totally different speed effects on the training oxford\_flowers102.

For instance, I use two machines to train 100 epochs AlexNet for TensorFlow.

**Machine One: Nvidia RTX 2070 Super**

**1. Power & GPU Memory:**

Initial Power: 3W/215W  
Power in Training: 187W/215W

Initial GPU Memory: 300MiB  
GPU Memory during Training: 5500MiB

Effect: 26 minutes for completing the training.

It increase form 3Ｗ to 187w in the power consumption. In addition, the GPU Ｍemory consumption increases quite fast. Therefore, its training speed is much more fast.

**2. Processes:**

Beside of GID, it shows GI ID and CI ID with the command of nvidia-smi.

**Machine Two: Nvidia RTX 2060**

**1. Power & GPU Memory**

Initial Power: 9W/160W  
Power in Training: 16W/160W

Initial GPU Memory: 300MiB  
GPU Memory during Training: 500MiB

Its power constantly fixes on 16W and GPU memory grows quite slow. And its training speed is about 4 hours, much more slower than RTX 2070.

**2. Processes:**

It shows GID but no GI ID and CI ID.

My question is what is GI ID and CI ID. Do they influence the training speed?

Note:

I enable GPU Fan Settings from 45% to 74%. But is has no effect on the training speed except the a little big noise.

---

<div class="post-metadata">

**Author:** ![mikechen6688](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mikechen6688](https://forums.developer.nvidia.com/u/mikechen6688)\
**Post date:** [July 19, 2020, 8:58pm UTC](https://forums.developer.nvidia.com/t/setting-up-nvidia-persistenced/47986/13 "2020-07-19T20:58:21Z")

</div>

I myself has solved the issue. The RTX series can greatly speed up the training by installing Driver v450.57, CUDA 11.0 and cuDNN 8.0.1.
