Fine-Grained Just-In-Time Defect Prediction at the Block Level in Infrastructure-as-Code (IaC)
Infrastructure-as-Code (IaC) is an emerging software engineering practice that leverages source code to facilitate automated configuration of software systems’ infrastructure. Like other software artifacts, severe defects could occur in IaC manifests, which can result in breaking millions of online services. To help developers early identify and fix IaC defects, several research efforts have introduced IaC defect prediction models at the file level. However, the granularity of the proposed approaches remains relatively coarse. Thus, identifying IaC bugs is a non-trivial task since IaC files may have many lines to inspect. To alleviate this issue, we aim to predict IaC defects at the fine-grained level by focusing on IaC blocks, which are code units that encapsulate specific behaviours within the IaC file. We evaluated our approach by conducting an empirical study on 19 open-source Terraform-based projects. We applied 6 machine learning algorithms to study to what extent they can predict defective blocks based on a mixture of code, process, and change level metrics. The results indicate that the LightGBM model emerged as the best model, achieving an average of 0.21 in the MCC and 0.71 in the AUC scores. We found that the developer’s experience and the relative number of added lines tend to be the most important features. Additionally, we found that blocks belonging to the most frequent types are more defect-prone. Our defect prediction models have also shown sensitivity to concept drift. This indicates that IaC practitioners should regularly retrain the defect models. We also suggest that practitioners stay attentive to changes in the syntax of block attributes, as these tend to be prone to modifications over time by IaC tool builders.
Mon 15 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | Defects, Bugs and IssuesTechnical Papers / MSR Awards / Social Events / Tutorials / Data and Tool Showcase Track / Mining Challenge / Registered Reports / Industry Track / MIP Award / Vision and Reflection / Keynotes at Grande Auditório Chair(s): Wesley Assunção North Carolina State University | ||
11:00 12mTalk | Enhancing Performance Bug Prediction Using Performance Code Metrics Technical Papers Guoliang Zhao Computer Science of Queen's University, Stefanos Georgio , Safwat Hassan University of Toronto, Canada, Ying Zou Queen's University, Kingston, Ontario, Derek Truong IBM Canada, Toby Corbin IBM UK | ||
11:12 12mTalk | CrashJS: A NodeJS Benchmark for Automated Crash Reproduction Technical Papers Philip Oliver Victoria University of Wellington, Jens Dietrich Victoria University of Wellington, Craig Anslow Victoria University of Wellington, Michael Homer Victoria University of Wellington | ||
11:24 12mTalk | An Empirical Study on Just-in-time Conformal Defect Prediction Technical Papers Xhulja Shahini paluno - University of Duisburg-Essen, Andreas Metzger University of Duisburg-Essen, Klaus Pohl | ||
11:36 12mTalk | Fine-Grained Just-In-Time Defect Prediction at the Block Level in Infrastructure-as-Code (IaC) Technical Papers Mahi Begoug , Moataz Chouchen ETS, Ali Ouni ETS Montreal, University of Quebec, Eman Abdullah AlOmar Stevens Institute of Technology, Mohamed Wiem Mkaouer University of Michigan - Flint | ||
11:48 4mTalk | TrickyBugs: A Dataset of Corner-case Bugs in Plausible Programs Data and Tool Showcase Track Kaibo Liu Peking University, Yudong Han Peking University, Yiyang Liu Peking University, Zhenpeng Chen Nanyang Technological University, Jie M. Zhang King's College London, Federica Sarro University College London, Gang Huang Peking University, Yun Ma Peking University | ||
11:52 4mTalk | GitBugs-Java: A Reproducible Java Benchmark of Recent Bugs Data and Tool Showcase Track André Silva KTH Royal Institute of Technology, Nuno Saavedra INESC-ID and IST, University of Lisbon, Martin Monperrus KTH Royal Institute of Technology | ||
11:56 4mTalk | A Dataset of Partial Program Fixes Data and Tool Showcase Track Dirk Beyer LMU Munich, Lars Grunske Humboldt-Universität zu Berlin, Matthias Kettl LMU Munich, Marian Lingsch-Rosenfeld LMU Munich, Moeketsi Raselimo Humboldt-Universität zu Berlin | ||
12:00 4mTalk | BugsPHP: A dataset for Automated Program Repair in PHP Data and Tool Showcase Track K.D. Pramod University of Moratuwa, Sri Lanka, W.T.N. De Silva University of Moratuwa, Sri Lanka, W.U.K. Thabrew University of Moratuwa, Sri Lanka, Ridwan Salihin Shariffdeen National University of Singapore, Sandareka Wickramanayake University of Moratuwa, Sri Lanka Pre-print | ||
12:04 4mTalk | AW4C: A Commit-Aware C Dataset for Actionable Warning Identification Data and Tool Showcase Track Zhipeng Liu , Meng Yan Chongqing University, Zhipeng Gao Shanghai Institute for Advanced Study - Zhejiang University, Dong Li , Xiaohong Zhang Chongqing University, Dan Yang Chongqing University | ||
12:08 5mTalk | Predicting the Impact of Crashes Across Release Channels Industry Track | ||
12:13 5mTalk | Zero Shot Learning based Alternatives for Class Imbalanced Learning Problem in Enterprise Software Defect Analysis Industry Track |