- New! FlakyGuard: Automatically Fixing Flaky Tests at Industry Scale
Chengpeng Li, Farnaz Behrang, August Shi, Peng Liu
IEEE/ACM Conference on Automated Software Engineering
(ASE 2025), pages to-appear, Seoul, South Korea, Nov. 2025 - New! Understanding and Improving Flaky Test Classification
Shanto Rahman, Saikat Dutta, August Shi
ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications
(OOPSLA 2025), pages 320:1-320:27, Singapore, Singapore, Oct. 2025 - New! Ranking Relevant Tests for Order-Dependent Flaky Tests
Shanto Rahman, Bala Naren Chanumolu, Suzzana Rafi, August Shi, Wing Lam
International Conference on Software Engineering
(ICSE 2025), pages 1999-2011, Ottawa, Canada, Apr.-May 2025 - New! OptCD: Optimizing Continuous Development
Talank Baral, Emirhan Oğul, Shanto Rahman, August Shi, Wing Lam
International Conference on Software Engineering Tool Demonstrations Track
(ICSE Demo 2025), pages 45-48, Ottawa, Canada, Apr.-May 2025 - New! Takuan: Using Dynamic Invariants To Debug Order-Dependent Flaky Tests
Nate Levin, Chengpeng Li, Yule Zhang, August Shi, Wing Lam
International Conference on Software Engineering New Ideas and Emerging Results Track
(ICSE NIER 2025), pages 81-85, Ottawa, Canada, Apr.-May 2025 - New! Detecting and Evaluating Order-Dependent Flaky Tests in JavaScript
Negar Hashemi, Amjed Tahir, Shawn Rasheed, August Shi, Rachel Blagojevic
International Conference on Software Testing, Verification and Validation
(ICST 2025), pages 13-24, Naples, Italy, Mar.-Apr. 2025 - Reducing Test Runtime by Transforming Test Fixtures
Chengpeng Li, Abdelrahman Baz, August Shi
IEEE/ACM Conference on Automated Software Engineering
(ASE 2024), pages 1757-1769, Sacramento, California, USA, Oct. 2024
(This paper won an ACM SIGSOFT Distinguished Paper Award.) - Prioritizing Tests for Improved Runtime
Abdelrahman Baz, Minchao Huang, August Shi
IEEE/ACM Conference on Automated Software Engineering New Ideas and Emerging Results Track
(ASE NIER 2024), pages 2273-2278, Sacramento, California, USA, Oct. 2024 - Impact of JVM Configurations on Test Runtime
Abdelrahman Baz, Milos Gligoric, August Shi
International Conference on Software Maintenance and Evolution
(ICSME 2024), pages 249-261, Flagstaff, Arizona, USA, Oct. 2024 - <Test Scheduling Across Heterogeneous Machines While Balancing Running Time, Price, and Flakiness
Hengchen Yuan, Jiefang Lin, Wing Lam, August Shi
International Conference on Software Maintenance and Evolution
(ICSME 2024), pages 449-460, Flagstaff, Arizona, USA, Oct. 2024 - Java JIT Testing with Template Extraction
Zhiqiang Zang, Fu-Yao Yu, Aditya Thimmaiah, August Shi, Milos Gligoric
ACM International Conference on the Foundations of Software Engineering
(FSE 2024), pages 51:1-51:23, Porto de Galinhas, Brazil, Jul. 2024 - Quantizing Large-Language Models for Predicting Flaky Tests
Shanto Rahman, Abdelrahman Baz, Sasa Misailovic, August Shi
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2024), pages 93-104, Toronto, Canada, May 2024 - Automatically Reproducing Timing-Dependent Flaky-Test Failures
Shanto Rahman, Aaron Massey, Wing Lam, August Shi, Jonathan Bell
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2024), pages 269-280, Toronto, Canada, May 2024 - FlakeSync: Automatically Repairing Async Flaky Tests
Shanto Rahman, August Shi
International Conference on Software Engineering
(ICSE 2024), pages 1673-1684, Lisbon, Portugal, Apr. 2024 - Optimizing Continuous Development By Detecting and Preventing Unnecessary Content Generation
Talank Baral, Shanto Rahman, Bala Naren Chanumolu, Başak Balcı, Tuna Tuncer, August Shi, Wing Lam
IEEE/ACM Conference on Automated Software Engineering
(ASE 2023), pages 901-913, Kirchberg, Luxembourg, Sept. 2023 - Systematically Producing Test-Orders to Detect Order-Dependent Flaky Tests
Chengpeng Li, Mahdi Khosravi, Wing Lam, August Shi
International Symposium on Software Testing and Analysis
(ISSTA 2023), pages 627-638, Seattle, Washington, USA, Jul. 2023 - TSVD4J: Thread-Safety Violation Detection for Java
Shanto Rahman, Chengpeng Li, August Shi
International Conference on Software Engineering Tool Demonstrations Track
(ICSE Demo 2023), pages 78-82, Melbourne, Australia, May 2023 - JAttack: Java JIT Testing using Template Programs
Zhiqiang Zang, Fu-Yao Yu, Nathan Wiatrek, Milos Gligoric, August Shi
International Conference on Software Engineering Tool Demonstrations Track
(ICSE Demo 2023), pages 6-10, Melbourne, Australia, May 2023 - Compiler Testing using Template Java Programs
Zhiqiang Zang, Nathan Wiatrek, Milos Gligoric, August Shi
IEEE/ACM Conference on Automated Software Engineering
(ASE 2022), pages 1-13, Oakland Center, Michigan, Oct. 2022
(This paper won an ACM SIGSOFT Distinguished Paper Award.) - Evolution-Aware Detection of Order-Dependent Flaky Tests
Chengpeng Li, August Shi
International Symposium on Software Testing and Analysis
(ISSTA 2022), pages 114-125, Virtual Event, Jul. 2022 - Repairing Order-Dependent Flaky Tests via Test Generation
Chengpeng Li, Chenguang Zhu, Wenxi Wang, August Shi
International Conference on Software Engineering
(ICSE 2022), pages 1881-1892, Pittsburgh, Pennsylvania, May 2022 - Comparing and Combining Analysis-Based and Learning-Based Regression Test Selection
Jiyang Zhang, Yu Liu, Milos Gligoric, Owolabi Legunsen, August Shi
International Conference on Automation of Software Test
(AST 2022), pages 17-28, Virtual Event, May 2022 - FLEX: Fixing Flaky Tests in Machine Learning Projects by Updating Assertion Bounds
Saikat Dutta, August Shi, Sasa Misailovic
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
(ESEC/FSE 2021), pages 603-614, Athens, Greece, Aug. 2021 - Domain-Specific Fixes for Flaky Tests with Wrong Assumptions on Underdetermined Specifications
Peilun Zhang, Yanjie Jiang, Anjiang Wei, Victoria Stodden, Darko Marinov, August Shi
International Conference on Software Engineering
(ICSE 2021), pages 50-61, Virtual Event, May 2021 - Learning from Reproducing Computational Results: Introducing Three Principles and the Reproduction Package
Matthew Krafczyk, August Shi, Adhithya Bhaskar, Darko Marinov, Victoria Stodden
Philosophical Transactions of the Royal Society A
(PTRSA), 379(2197):Paper 20200069, 28 pages, May 2021 - Improving Regression Testing Efficiency and Reliability via Test-Suite Transformations
August Shi
Ph.D. Dissertation, University of Illinois at Urbana-Champaign, Urbana-Champaign, Illinois, Jul. 2020
(This dissertation won an ACM SIGSOFT Outstanding Doctoral Dissertation Award.) - Detecting Flaky Tests in Probabilistic and Machine Learning Applications
Saikat Dutta, August Shi, Rutvik Choudhary, Zhekun Zhang, Aryaman Jain, Sasa Misailovic
International Symposium on Software Testing and Analysis
(ISSTA 2020), pages 211-224, Virtual Event, Jul. 2020 - Dependent-Test-Aware Regression Testing Techniques
Wing Lam, August Shi, Reed Oei, Sai Zhang, Michael D. Ernst, Tao Xie
International Symposium on Software Testing and Analysis
(ISSTA 2020), pages 298-311, Virtual Event, Jul. 2020 - Empirically Revisiting and Enhancing IR-Based Test-Case Prioritization
Qianyang Peng, August Shi, Lingming Zhang
International Symposium on Software Testing and Analysis
(ISSTA 2020), pages 324-336, Virtual Event, Jul. 2020
slides - Reflection-Aware Static Regression Test Selection
August Shi, Milica Hadzi-Tanovic, Lingming Zhang, Darko Marinov, Owolabi Legunsen
ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications
(OOPSLA 2019), pages 187:1-187:29, Athens, Greece, Oct. 2019
slides - Understanding and Improving Regression Test Selection in Continuous Integration
August Shi, Peiyuan Zhao, Darko Marinov
IEEE International Symposium on Software Reliability Engineering
(ISSRE 2019), pages 228-238, Berlin, Germany, Oct. 2019
slides - iFixFlakies: A Framework for Automatically Fixing Order-Dependent Flaky Tests
August Shi, Wing Lam, Reed Oei, Tao Xie, Darko Marinov
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
(ESEC/FSE 2019), pages 545-555, Tallinn, Estonia, Aug. 2019
slides - Mitigating the Effects of Flaky Tests on Mutation Testing
August Shi, Jonathan Bell, Darko Marinov
International Symposium on Software Testing and Analysis
(ISSTA 2019), pages 112-122, Beijing, China, Jul. 2019
slides - Scientific Tests and Continuous Integration Strategies to Enhance Reproducibility in the Scientific Software Context
Matthew Krafczyk, August Shi, Adhithya Bhaskar, Darko Marinov, Victoria Stodden
International Workshop on Practical Reproducible Evaluation of Computer Systems
(P-RECS 2019), pages 23-28, Phoenix, Arizona, Jun. 2019 - A Framework for Checking Regression Test Selection Tools
Chenguang Zhu, Owolabi Legunsen, August Shi, Milos Gligoric
International Conference on Software Engineering
(ICSE 2019), pages 430-441, Montreal, Canada, May 2019
slides - Comparing Mutation Testing at the Levels of Source Code and Compiler Intermediate Representation
Farah Hariri, August Shi, Vimuth Fernando, Suleman Mahmood, Darko Marinov
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2019), pages 114-124, Xi’an, China, Apr. 2019
slides - iDFlakies: A Framework for Detecting and Partially Classifying Flaky Tests
Wing Lam, Reed Oei, August Shi, Darko Marinov, Tao Xie
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2019), pages 312-322, Xi’an, China, Apr. 2019 - SRCIROR: A Toolset for Mutation Testing of C Source Code and LLVM Intermediate Representation
Farah Hariri, August Shi
IEEE/ACM Conference on Automated Software Engineering Tool Demonstrations Track
(ASE Demo 2018), pages 860-863, Montpellier, France, Sept. 2018
slides - Evaluating Test-Suite Reduction in Real Software Evolution
August Shi, Alex Gyori, Suleman Mahmood, Peiyuan Zhao, Darko Marinov
International Symposium on Software Testing and Analysis
(ISSTA 2018), pages 84-94, Amsterdam, Netherlands, Jul. 2018
slides - An Extensible, Regular-Expression-Based Tool for Multi-Language Mutant Generation
Alex Groce, Josie Holmes, Darko Marinov, August Shi, Lingming Zhang
International Conference on Software Engineering Tool Demonstrations Track
(ICSE Demo 2018), pages 25-28, Gothenburg, Sweden, May-Jun. 2018
slides - Approximate Transformations as Mutation Operators
Farah Hariri, August Shi, Owolabi Legunsen, Milos Gligoric, Sarfraz Khurshid, Sasa Misailovic
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2018), pages 285-296, Vasteras, Sweden, Apr. 2018 - STARTS: STAtic Regression Test Selection
Owolabi Legunsen, August Shi, Darko Marinov
IEEE/ACM Conference on Automated Software Engineering Tool Demonstrations Track
(ASE Demo 2017), pages 949-954, Urbana-Champaign, Illinois, Oct.-Nov. 2017 - Optimizing Test Placement for Module-Level Regression Testing
August Shi, Suresh Thummalapenta, Shuvendu Lahiri, Nikolaj Bjorner, Jacek Czerwonka
International Conference on Software Engineering
(ICSE 2017), pages 689-699, Buenos Aries, Argentina, May 2017
(This paper won an ACM SIGSOFT Distinguished Paper Award.)
slides - Mutation Testing Meets Approximate Computing
Milos Gligoric, Sarfraz Khurshid, Sasa Misailovic, August Shi
International Conference on Software Engineering, New Ideas and Emerging Results Track
(ICSE NIER 2017), pages 3-6, Buenos Aries, Argentina, May 2017
slides - An Extensive Study of Static Regression Test Selection in Modern Software Evolution
Owolabi Legunsen, Farah Hariri, August Shi, Yafeng Lu, Lingming Zhang, Darko Marinov
ACM SIGSOFT International Symposium on the Foundations of Software Engineering
(FSE 2016), pages 583-594, Seattle, Washington, Nov. 2016 - NonDex: A tool for detecting and debugging wrong assumptions on Java API specifications
Alex Gyori, Ben Lambeth, August Shi, Owolabi Legunsen, Darko Marinov
ACM SIGSOFT International Symposium on the Foundations of Software Engineering Tool Demonstrations Track
(FSE Demo 2016), pages 993-997, Seattle, Washington, Nov. 2016 - Evaluating the Effects of Compiler Optimizations on Mutation Testing at the Compiler IR Level
Farah Hariri, August Shi, Hayes Converse, Darko Marinov, Sarfraz Khurshid
IEEE International Symposium on Software Reliability Engineering
(ISSRE 2016), pages 105-115, Ottawa, Canada, Oct. 2016 - Evaluating Non-adequate Test-Case Reduction
Mohammad Amin Alipour, August Shi, Rahul Gopinath, Darko Marinov, Alex Groce
IEEE/ACM Conference on Automated Software Engineering
(ASE 2016), pages 16-26, Singapore, Singapore, Sept. 2016
slides - Detecting Assumptions on Deterministic Implementations of Non-deterministic Specifications
August Shi, Alex Gyori, Owolabi Legunsen, Darko Marinov
IEEE International Conference on Software Testing, Verification and Validation
(ICST 2016), pages 80-90, Chicago, Illinois, Apr. 2016
slides - Comparing and Combining Test-Suite Reduction and Regression Test Selection
August Shi, Tifany Yung, Alex Gyori, Darko Marinov
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
(ESEC/FSE 2015), pages 237-247, Bergamo, Italy, Sept. 2015
slides - Reliable Testing: Detecting State-Polluting Tests to Prevent Test Dependency
Alex Gyori, August Shi, Farah Hariri, Darko Marinov
International Symposium on Software Testing and Analysis
(ISSTA 2015), pages 223-233, Baltimore, Maryland, Jul. 2015 - Balancing Trade-Offs in Test-Suite Reduction
August Shi, Alex Gyori, Milos Gligoric, Andrey Zaytsev, Darko Marinov
ACM SIGSOFT Symposium on the Foundations of Software Engineering
(FSE 2014), pages 246-256, Hong Kong, Nov. 2014
slides - Evaluation of a Feature Tracking Vision Application on a Heterogeneous Chip
Ruben Gran Tejero, August Shi, Ehsan Totoni, Maria J. Garzaran
IEEE International Symposium on Computer Architecture and High Performance Computing
(SBAC-PAD 2014), pages 246-253, Paris, France, Oct. 2014