Data preparation and curation

Fine tuning quality is determined more by data quality than by hyperparameters. A small, clean, diverse dataset outperforms a large, noisy one. Data curation is the process of selecting, cleaning, formatting, and validating training examples to maximize learning signal and minimize noise.