The robust and accurate identification of different forms of manure stands as a pivotal imperative within the domain of agriculture. Near-infrared (NIR) Spectroscopy has emerged as an expeditious, efficient, non-destructive, and reliable approach to addressing this challenging task. NIR spectroscopy has the potential to serve as a valuable tool for the classification and identification of manure varieties. In order to enhance fertilizer identification performance, this study proposes a novel model called NIRsViT which classifies fertilizers by employing a combination of deep learning Vision Transformer model on NIR spectral data. The introduced model’s performance outperforms existing deep learning models, with an F1-Score of 86.42% and an accuracy rate of 95.19%. Additionally, the model’s classification performance has been significantly improved by proposed imbalanced data processing approaches, Focal Loss, and Upsample, with an F1-Score up to 93.91%, the improved F1-score proved that imbalanced data was considerably solved. The proposed method is a promising approach to handling imbalanced NIR spectral data and acts as a pioneering benchmark for subsequent models in manure identification through NIR spectroscopy. Future research gears toward improving the NIRsViT model’s temporal efficiency and computational load, while also testing the introduced imbalanced data handling approach for efficiency comparison across various models and larger datasets.