I am trying to a ASR which can transcribe multiple languages, by keeping a tokenizer like this:
model:
…
tokenizer:
type: “agg” # aggregate tokenizer
langs:
en:
dir: “”
type: “bpe” # can be “bpe” or “wpe”
es:
dir: “”
type: “bpe” # can be “bpe” or “wpe”
Can I also specify a different decoder for each “Lang” ? Instead of specifying the same decoder type for each lang.
model:
…
decoder:
target: nemo.collections.asr.modules.ConvASRDecoder
feat_in: *enc_final
num_classes: -1 # filled with vocabulary size from tokenizer at runtime
vocabulary: # filled with vocabulary from tokenizer at runtime
For example: One decoder could be a transcription decoder, another can be a classification decoder to predict intent.