NVIDIA Developer Forums

Scaling Language Model Training to a Trillion Parameters Using Megatron

AI & Data Science Data Science of the Day

btegegn June 7, 2021, 1:00pm 1

You know whats cooler than a billion parameters? A TRILLION parameters.
Click the image to read the article
Find more #DSotD posts

Have an idea you would like to see featured here on the Data Science of the Day?

Topic		Replies	Views	Activity
Training a Multilingual Model Works Better When You Use Enough Data and a Larger Model Data Science of the Day fun-facts , natural-language-processing-nlp , data-science	0	909	May 19, 2021
Training a Recommender System with Over 100 Billion Parameters? Data Science of the Day fun-facts , deep-learning , artificialintelligence	0	1202	April 29, 2022
Multi-Task Learning with Transformers Data Science of the Day fun-facts , deep-learning , machine-learning , nlp	0	1215	November 5, 2021
Transformers Explained Visually Data Science of the Day ai , fun-facts , natural-language-processing-nlp , deep-learning , data-science	0	2032	November 1, 2021
What are Vector Embeddings? Data Science of the Day fun-facts , natural-language-processing-nlp , ml	0	1131	June 1, 2021
How to Train Vision Transformers? Data Science of the Day fun-facts , data	0	1612	July 23, 2021
Deploying Deep Learning Models with Model Server Data Science of the Day fun-facts , deep-learning , data-science	0	1384	December 10, 2021
Transformers from Scratch Data Science of the Day fun-facts , deep-learning , nlp	0	1662	December 3, 2021
AI Models for Answering Questions Data Science of the Day ai , fun-facts	0	953	January 25, 2022
NLP is Evolving at a Rapid Pace. What is a Transformer? What is BERT? What is Next? Data Science of the Day fun-facts , natural-language-processing-nlp	0	1324	May 27, 2021