Constitutive models are fundamental for analyzing the mechanical behavior of structural steels. Classical phenomenological or physics-based formulations provide explicit equations but often struggle to capture strong nonlinearity. Consequently, neural-network-based (NN) constitutive models have attracted growing attention for their ability to approximate complex response surfaces efficiently. In this work, we develop data-driven NN models for HSLA-65 steel across a wide range of temperatures and strain rates. We train a Multilayer Perceptron (MLP) and a Recurrent Neural Network (RNN), and evaluate generalization with five different metrics. To further interpret the models, we apply SHAP (SHapley Additive exPlanations) to quantify input-feature contributions. The results show that temperature (thermal softening) and strain rate (rate sensitivity) are dominant drivers, while plastic strain provides a positive cumulative contribution (work hardening) whose global weight is modest in the MLP but approaches that of temperature and strain rate in the RNN. Compared with the MLP, the RNN achieves higher accuracy and demonstrates superior robustness and generalization on unseen data. Overall, the study advances NN-based constitutive modeling by combining high-fidelity predictions with physics-consistent explanations and offering a reproducible workflow applicable to other alloys and loading conditions.