[reformat.cpp::executeCutensor::384] Error Code 1: CuTensor (Internal cuTensor permutate execute failed)

powerdi123 · June 9, 2023, 9:20am

We can infer the model and get exactly results in main thread. But if we put the dynamic libraries in the multi thread app, it shows the error:

[06/09/2023-17:17:43] [E] [TRT] 1: [reformat.cpp::executeCutensor::384] Error Code 1: CuTensor (Internal cuTensor permutate execute failed)

AastaLLL · June 12, 2023, 2:45am

Hi,

Please create a TensorRT context for each thread.
You can find the implementation for multi-thread below:

github.com

NVIDIA/TensorRT/blob/release/8.6/samples/common/sampleInference.cpp

/*
 * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

#include <algorithm>
#include <array>
#include <chrono>

This file has been truncated. show original

Thanks.

powerdi123 · June 12, 2023, 3:00am

Hey. Thanks the comment. We make a test that in a single thread that create one TensorRT context. But it still not work. That means we do not use sereval threads but only one.

AastaLLL · June 13, 2023, 5:10am

I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks
Hi,

Do you have a reproducible source so we can check it further?
Thanks.

kayccc · June 28, 2023, 3:22am