Track: Quality in the age of AI
ABOUT
Quality of AI-enabled systems (Q4AI) is recognized as a difficult challenge in both research and practice. Many of these challenges are driven by the data-dependent nature of AI components in which functionality is determined by characteristics (features) of training and operational data and not by traditional component specifications from which test cases are often derived. This data-dependency also causes AI components to drift over time as characteristics of operational data change over time, therefore requiring QA activities, such as runtime monitoring to be essential components of AI-enabled systems.
A complementary aspect of Quality in the Age of AI is the use of AI to support Quality activities and processes (AI4Q), such as using AI techniques for test data and test case generation, fault localization in source code, and analyzing runtime log data to identify problems and courses of action. Challenges in this area stem from the lack of high quality and quantity of training data and oracles that are important for model performance and accuracy.
With the increase in complexity, size, and ubiquity of AI-enabled systems, as well as advances in AI including the growing popularity of large language models (LLMs), it is necessary to continue exploring Quality in the Age of AI. We therefore seek novel contributions investigating advances in both Q4AI and AI4Q.
TOPICS
The scope of this track is Quality in the Age of AI. The topics of interest include, but are not limited to:
Quality of AI-enabled Systems:
Elicitation and specification of quality requirements for AI systems
Testing techniques for AI components and systems
Data quality processes
Tools to support software quality activities in AI systems
Runtime monitoring of AI systems
Certification processes for AI components and systems
Quality metrics for AI systems and components
AI Supporting Software Quality Processes
AI for test case generation
AI for test data generation
AI for quality requirements generation
AI for runtime log analysis
AI for fault localization
SUBMISSION
Formatting and submission guidelines are available at https://2024.quatic.org/call-for-papers . QUATIC uses a single submission site for all tracks. When submitting, please make sure to select “Quality in the age of AI” as your track.
TRACK COMMITTEE
Chairs: Grace A. Lewis (Carnegie Mellon Software Engineering Institute, United States) and Domenico Bianculli (University of Luxembourg, Luxembourg)
Program Committee:
Alessio Gambi, AIT, Austria
Donghwan Shin, University of Sheffield, United Kingdom
Julien Siebert, Fraunhofer IESE, Germany
Lei Ma, University of Tokyo, Japan and University of Alberta, Canada
Neelofar Neelofar, Monash University, Australia
Rachel Brower-Sinning, Carnegie Mellon Software Engineering Institute, USA
Vincenzo Riccio, University of Udine, Italy
Xiaochen Li, Dalian University of Technology, China
Xiao-Y Zhang, University of Science and Technology Beijing, China
Zhenya Zhang, Kyushu University, Japan
Grace Lewis is a Principal Researcher at the Carnegie Mellon Software Engineering Institute (SEI) where she conducts applied research on how software engineering and software architecture principles, practices and tools need to evolve in the face of emerging technologies. She is the principal investigator for the Automating Mismatch Detection and Testing in Machine Learning Systems project that is developing toolsets to support these two activities, in addition to other projects that are advancing the state of the practice in software engineering for machine learning (SE4ML). Grace is also the lead for the Tactical and AI-Enabled Systems (TAS) applied research and development team at the SEI that is creating and transitioning innovative solutions, principles, and best practices for (1) architecting and developing systems to support teams operating at the tactical edge in resource-constrained environments, (2) engineering AI software systems, and (3) using AI/ML at the edge for improved capabilities and mission support. She is currently First Vice President of the IEEE Computer Society. Grace holds a B.Sc. in Software Systems Engineering and a Post-Graduate Specialization in Business Administration from Icesi University in Cali, Colombia; a Master in Software Engineering from Carnegie Mellon University; and a Ph.D. in Computer Science from Vrije Universiteit Amsterdam..
Domenico Bianculli is associate professor/chief scientist 2 at the Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, where he leads the Software Verification and Validation (SVV) research group. He holds a PhD degree from Università della Svizzera italiana (Lugano, Switzerland), a MSc in Computing Systems Engineering and a BSc in Computer Engineering, both from Politecnico di Milano (Milan, Italy). His research career has mainly focused on different aspects of run-time verification for dependable, secure, and compliant software, conducting research and technology transfer with industrial partners in the financial, eGovernment, satellite, and mission-critical systems domains. Current research interests include: run-time verification and monitoring, specification languages, verification and validation for cyber-physical systems, log analysis and AIOps, program analysis, data quality, and regulatory compliance.