Microsoft Trains Turing-NLG, World’s Largest Transformer Language Model

Originally published at: https://developer.nvidia.com/blog/using-nvidia-dgx-2-systems-microsoft-trains-worlds-largest-transformer-language-model/

Microsoft today announced a breakthrough in conversational AI, training the largest transformer-based language generation model with 17 billion parameters, using NVIDIA DGX-2 systems. Also today, Microsoft open-sourced DeepSpeed, a deep learning library that can help developers with latency and inference. Named Turing Natural Language Generation (T-NLG), the model is the largest transformer model available that…