Hybrid CNN–Transformer Architecture for Multivariate Behavioral Time-Series Modeling in Precision Livestock Pregnancy Prediction

An Intelligent Data-Driven Approach for Early Pregnancy Prediction in Precision Livestock Farming

Abstract

This work presents CNN–Transformer, a new hybrid deep learning model based on CNN and Transformer for predicting the pregnancy of a cow with multivariate sequential behavioral data extracted from CowView sensors. The percentage features were divided into three cow states – ALLEY (motion), BOOST (eating) & LOGETTE (sleeping) – and summarized into 30 minutes, resulting in daily sequences comprised of 48 time points. To mitigate the limitation of scarce data and facilitate generalization, simulation-based augmentation is performed by utilizing activity frequency matrix and transition probability matrix learned from the real data. Among these, Logistic Regression, SVM, Random Forest, LSTM and CNN-LSTM are chosen and a comparative analysis of the five models is presented. Traditional machine learning algorithms had moderate performance: Logistic Regression and SVM obtained an accuracy of 71% and 73%, respectively; Random Forest obtained 66%. The results are improved by deep learning methods, where LSTM obtains 77% (ROC-AUC: 0.82), and CNN-LSTM achieves 82% (Precision: 0.86, F1-score: 0.81, ROC-AUC: 0.87). Our CNN–Transformer model beats all the baselines at the accuracy of 86%, precision of 0.90, F1-score of 0.86, and ROC-AUC of 0.92. Models trained on the real data, on average, had an accuracy of 73.8%, significantly better than that with the simulated data (53.8%). It was robust, holding 78% accuracy at 20% noise, and early prediction accuracy was 81% at 24 hours before diagnosis. Statistical validation demonstrated significant improvement (p < 0.01), thus confirming hybrid temporal models for precision livestock farming.