Predicting Acceptance and Review Effort in Human and Agent Pull Requests
Kartik Ghanshyambhai Pansuriya, Ehsan Ghorbani, Deepak Singh, Eman Abdullah AlOmar
IEEE COMPSAC 2026 · SETA Symposium (Software Engineering Technologies & Applications) · Full Paper · Madrid, Spain, July 2026
Can a pull request’s fate be predicted the moment it is opened — before reviewer discussion, CI feedback, or the merge decision? Using the AIDev dataset of human- and AI-agent-authored PRs, this work builds a leakage-aware pipeline that relies only on submission-time signals (PR text, metadata, repository context, temporal signals, and lightweight diff statistics), then compares classical models across pooled, human-only, agent-only, and balanced contributor views with SHAP-based feature analysis.
Key result: tree-based models predict PR acceptance with F1 > 0.95 from early signals, with textual clarity and metadata the most influential predictors — while review effort (comment count, time-to-merge) stays much harder to estimate, pointing to early PR models as advisory triage tools rather than automated decision-makers.
arXiv:2607.12057 [cs.SE]