TensorRT model inference fully on DLA is slow due to abnormally slow cudaEventSynchronize time

[AastaLLL]

Let’s follow the new sparsity issue on the separate topic you created:

Sparsity does not provide any speedup for TensorRT on DLA

My question is: am I doing the sparsity correctly and if not, how to get the claimed speedup from adding structured sparsity? Here is what I did to prune and quantize the model. First I applied sparsity as follows from apex.contrib.sparsity import ASP model_sparse.model.cuda() optimizer_sparse = torch.optim.AdamW(model_sparse.parameters(), lr=learning_rate, weight_decay=0.05) ASP.prune_trained_model(model_sparse, optimizer_sparse) trainer.fit(model=model_sparse, train_dataloaders=train_loade…

Thanks