Loading a pre-trained model into NeMo


I’m just getting started with NeMo and I’m trying to load a pre-trained model. The steps I’d like to accomplish are:

  1. Load a pre-trained QA (question answering) model
  2. Load some context
  3. Ask a question and get an answer

I’m using NeMo and I’m loading a Financial Megatron model finmegatron345m_gpt2_bpe. I feel like my problem is in the configuration of NeMo. I’m using the basic QA conf that is provided in the Github (qa_conf.yaml) and then following this Notebook (Question_Answering.ipynb).

I’ve been tweaking the configuration to point to my pre-trained model but I’m getting a variety of errors, most recently that I need to load a language model checkpoint.

Traceback (most recent call last):
  File "/home/ubuntu/finance.py", line 14, in <module>
    model = GPTQAModel.restore_from(restore_path='/home/ubuntu/finmegatron_gpt2_en_uncase.nemo', override_config_path='/home/ubuntu/qa_conf.yml')
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/classes/modelPT.py", line 330, in restore_from
    instance = cls._save_restore_connector.restore_from(
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/connectors/save_restore_connector.py", line 235, in restore_from
    loaded_params = self.load_config_and_state_dict(
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/connectors/save_restore_connector.py", line 158, in load_config_and_state_dict
    instance = calling_cls.from_config_dict(config=conf, trainer=trainer)
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/classes/common.py", line 507, in from_config_dict
    raise e
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/classes/common.py", line 499, in from_config_dict
    instance = cls(cfg=config, trainer=trainer)
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/collections/nlp/models/question_answering/qa_gpt_model.py", line 47, in __init__
    self.language_model = MegatronGPTModel.restore_from(cfg.language_model.lm_checkpoint, trainer=trainer)
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/site-packages/nemo/core/classes/modelPT.py", line 319, in restore_from
    restore_path = os.path.abspath(os.path.expanduser(restore_path))
  File "/home/ubuntu/.pyenv/versions/3.10.9/lib/python3.10/posixpath.py", line 231, in expanduser
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

My code is super-simple

import os
import wget

import pytorch_lightning as pl
from omegaconf import OmegaConf

from nemo.collections.nlp.models.question_answering.qa_gpt_model import GPTQAModel
from nemo.utils.exp_manager import exp_manager


model = GPTQAModel.restore_from(restore_path='./finmegatron_gpt2_en_uncase.nemo', override_config_path='./qa_conf.yml')

When reading through the pre-trained model info, it doesn’t say anything about needing to include a language model checkpoint. Is there any simple documentation on just loading up a pre-trained model in NeMo and running an inference pass?


.nemo file is just a archive that stores config, model, checkpoint. Try to unrar nemo file to know what files does it have. I think you should define full path to finmegatron_gpt2_en_uncase.nemo