GTC 2020: Optuna: An Eager Hyperparameter Optimization Library

GTC 2020 S21291
Presenters: Shotaro Sano,Preferred Networks
Abstract
We’ll present Optuna, an open-source, next-generation hyperparameter optimization framework with three novel design criteria: (1) an eager API that allows users to concisely construct dynamic, nested, or conditional search spaces; (2) efficient implementation of both sampling and early-stopping strategies; and (3) an easy-to-set-up, versatile architecture that can be deployed for various purposes, ranging from simple scaling on distributed GPUs to lightweight experimentation, conducted via interactive interface. Optuna is the first optimization software designed to run in Eager mode. We’ll present the basic usage of Optuna, describe the design techniques that we needed to meet the above criteria during development, and share our experiences leveraging Optuna to tune hyperparameters on single and multiple GPUs.

Watch this session
Join in the conversation below.