GTC 2020 S21961
Presenters: Ian Hummel,Bloomberg ; CJ Zheng,Bloomberg
Abstract
We’ll explain how we leverage the Bloomberg Data Science Platform to accelerate training large neural networks on our enormous in-house corpora. We’ll discuss how we integrate open-source solutions, such as Kubernetes and KubeFlow, and review our results from pretraining BERT entirely from scratch. The Bloomberg Terminal provides data, analytics, news, and communication for professionals in business, finance, government, and philanthropy. Natural language processing (NLP) plays a growing role in our toolkit due to the increasing importance of textual data in modern finance. Transformer models such as BERT, XLNet and OpenAI’s GPT series achieve superior performance on a wide variety of NLP tasks by replacing recurrence of neurons with self-attention mechanisms. At Bloomberg, we train customized transformer LMs on finance-related corpora to improve performance of production tasks like named entity recognition, document classification, question answering, and more.
Watch this session
Join in the conversation below.