Software Testing for Machine Learning

Kishore Sugali Venkata N. Inukollu Chris Sprunger

31 Januari 2021

The history of Artificial Intelligence and Machine Learning dates back to 1950’s. In recent years, there has been an increase in popularity for applications that implement AI and ML technology. As with traditional development, software testing is a critical component of an efficient AI/ML application. However, the approach to development methodology used in AI/ML varies significantly from traditional development. Owing to these variations, numerous software testing challenges occur. This paper aims to recognize and to explain some of the biggest challenges that software testers face in dealing with AI/ML applications. For future research, this study has key implications. Each of the challenges outlined in this paper is ideal for further investigation and has great potential to shed light on the way to more productive software testing strategies and methodologies that can be applied to AI/ML applications.

Naveenkumar Jayakumar Nilofar Mulla

5 April 2021

This study provides information about the use of artificial intelligence (AI) and machine learning (ML) techniques in the field of software testing. The use of AI in software testing is still in its initial stages. Also the automation level is lesser compared to more evolved areas of work.AI and ML can be used to help reduce tediousness and automate tasks in software testing. Testing can be made more efficient and smarter with the help of AI. Researchers recognize potential of AI to bridge the gap between human and machine driven testing capabilities. There are still number of challenges to fully utilize AI and ML techniques in testing but it will definitely enhance the entire testing process and skills of testers and will contribute in business growth. Machine learning research is a subset of overall AI research. The lifecycle of software is increasingly shortening and becoming more complicated. There is a struggle in software development between the competing pressures of developing software and meeting deadlines. AI-powered automated testing makes conducting full test suites in a timely manner on every change. In this article a detailed overview about the various applications of AI in software testing have been demonstrated. Also the implementation of machine learning in software testing has been discussed in detail and use of different machine learning techniques has been explained as well.

Caterina Urban Antoine Min'e

6 April 2021

We review state-of-the-art formal methods applied to the emerging field of the verification of machine learning systems. Formal methods can provide rigorous correctness guarantees on hardware and software systems. Thanks to the availability of mature tools, their use is well established in the industry, and in particular to check safety-critical applications as they undergo a stringent certification process. As machine learning is becoming more popular, machine-learned components are now considered for inclusion in critical systems. This raises the question of their safety and their verification. Yet, established formal methods are limited to classic, i.e. non machine-learned software. Applying formal methods to verify systems that include machine learning has only been considered recently and poses novel challenges in soundness, precision, and scalability. We first recall established formal methods and their current use in an exemplar safety-critical field, avionic software, with a focus on abstract interpretation based techniques as they provide a high level of scalability. This provides a golden standard and sets high expectations for machine learning verification. We then provide a comprehensive and detailed review of the formal methods developed so far for machine learning, highlighting their strengths and limitations. The large majority of them verify trained neural networks and employ either SMT, optimization, or abstract interpretation techniques. We also discuss methods for support vector machines and decision tree ensembles, as well as methods targeting training and data preparation, which are critical but often neglected aspects of machine learning. Finally, we offer perspectives for future research directions towards the formal verification of machine learning systems.

Isabel Praça Tiago Fonseca João Vitorino + 2 lainnya

14 Maret 2023

It is imperative to safeguard computer applications and information systems against the growing number of cyber-attacks. Automated software testing tools can be developed to quickly analyze many lines of code and detect vulnerabilities by generating function-specific testing data. This process draws similarities to the constrained adversarial examples generated by adversarial machine learning methods, so there could be significant benefits to the integration of these methods in testing tools to identify possible attack vectors. Therefore, this literature review is focused on the current state-of-the-art of constrained data generation approaches applied for adversarial learning and software testing, aiming to guide researchers and developers to enhance their software testing tools with adversarial testing methods and improve the resilience and robustness of their information systems. The found approaches were systematized, and the advantages and limitations of those specific for white-box, grey-box, and black-box testing were analyzed, identifying research gaps and opportunities to automate the testing tools with data generated by adversarial attacks.

R. Galanopoulou D. Spinellis Zoe Kotti

17 November 2022

Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009 and 2022, covering 6,117 primary studies. The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML. We propose a number of ML for SE research challenges and actions, including conducting further empirical validation and industrial studies on ML, reconsidering deficient SE methods, documenting and automating data collection and pipeline processes, reexamining how industrial practitioners distribute their proprietary data, and implementing incremental ML approaches.