# Curating Non-English Datasets for LLM Training with NVIDIA NeMo Curator

**URL:** <https://forums.developer.nvidia.com/t/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/299374>\
**Category:** Technical Blog\
**Created:** [July 10, 2024, 4:00pm UTC](https://forums.developer.nvidia.com/t/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/299374 "2024-07-10T16:00:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jwitsoe](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/jwitsoe/32/16591_2.png) [@jwitsoe](https://forums.developer.nvidia.com/u/jwitsoe)\
**Post date:** [July 10, 2024, 4:00pm UTC](https://forums.developer.nvidia.com/t/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/299374/1 "2024-07-10T16:00:32Z")

</div>

Originally published at: [https://developer.nvidia.com/blog/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/](https://developer.nvidia.com/blog/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/)  
   
Data curation plays a crucial role in the development of&nbsp;effective and fair large language models (LLMs). High-quality, diverse training data directly impacts LLM performance, addressing issues like bias, inconsistencies, and redundancy. By curating high-quality datasets,&nbsp; we can ensure that&nbsp; LLMs are accurate, reliable, and generalizable.&nbsp;&nbsp; When training a localized multilingual LLM, especially for low-resourced languages,…

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [July 10, 2024, 4:26pm UTC](https://forums.developer.nvidia.com/t/curating-non-english-datasets-for-llm-training-with-nvidia-nemo-curator/299374/2 "2024-07-10T16:26:14Z")

</div>