More than single bitstream output buffer per input

I am using NVENC 9.2 to encode. Now, I download the bitstream in separate thread. This is not very
efficient if I use single input buffer that maps to a single output bitstream buffer as the first should wait for the bitsream to finish download (?). What I thought was to have a pool of output buffers and bind the next free one to the encode params each time,so that the encoder can do its work independently of the download of the NALs. Is it safe to work like that? Or I must always provide input and output buffers as a single work unit?