Theme 1: Search Based Testing for the Future Internet
- Paolo Tonella. Academic developments in search based testing for the Future Internet.
- Abstract:Search based algorithms can be used to address the problem of automatically generating the test data necessary to ensure a given test adequacy level is met (e.g., branch coverage). Search algorithms (genetic algorithms, ant colony optimization, hill climbing, etc.) resort to a fitness function in order to select the most promising solutions for further search space exploration. This makes them quite robust with respect to program properties, such as infeasible paths, pointers, reflection and non linear computations, which are hard to deal with, using more traditional program analysis. In this tutorial, I will introduce the basic principles behind the search based algorithms most widely used in software testing. I will describe how they can be instantiated to solve the test case generation problem. I will then present the extensions required when the program under test is an object oriented program. Finally, I will overview some recent developments that try to combine search based test case generation with other approaches, such as dynamic symbolic execution.
- Peter M. Kruse. Industrial applications of search based testing for the Future Internet.
- Abstract: During the past years, search based testing research has reported encouraging results for automated functional (i.e. black-box) testing. However, despite promising results, these techniques have hardly been applied to complex, real-world systems and as such, little is known about their scalability, applicability, and acceptability in industry.In my talk, I will discuss about tis acceptability and present existing uses of search based software testing. The introduction is a short review on previous case studies and applications for search based testing, both functional black-box testing and structural white-box testing. Current applications include testing of interactive systems (e.g. GUIs) and recent results on test data generation.
- Tanja Vos. Mayor European initiatives in search based testing where academia and industry get together.
- Abstract: As a coordinator of two EU funded initiatives on search based testing, I will tell you the experiences with setting up EU projects, the do´s and the don´ts and what the important things are during proposal preparation. Subsequently, I will tell you the challenges which you have to overcome when executing such a project, especially related to the empirical case study work that needs to be done. To assess tools resulting from an EU funded project, evaluative case study research must involve realistic systems and realistic subjects from industry, and not toy-programs and students as it is the case with most case studies. This type of research is time-consuming, expensive and extremely difficult. Not in the last place because the success of these studies depends heavily on the communication between academia and industry and this has long history of being hindered by different underlying objectives and visions from the different actors in an academia-industry partnership. I will present a general methodological framework that we have developed in the context of the FITTEST project for evaluating software testing techniques and tools, which can be instantiated to define case studies that evaluate the effectiveness, efficiency and subjective satisfaction of specific techniques and tools. This framework has been successfully used in the FITTEST project as a vehicle to improve communication and align objectives during case study research in academia-industry partnerships.
Theme 2: Training and teaching software testing: What a good software tester shall know
- Tao Xie. How to do research on software testing?
- Abstract: Conducting research in a research area such as software testing includes various aspects, including selecting a research problem, designing a solution to address the problem, executing the research project with tool prototyping and evaluation, communicating the research outcomes in both written and oral forms, and laying out a research agenda to realize a longer-term research vision, etc. This talk will give an overview on research skills that a student needs to be equipped with to independently carry out high-quality research in software testing along with general guidelines and lessons learned on various above-mentioned aspects in conducting research in software testing.
Theme 3: Security Testing
- Antonia Bertolino. XML-based approaches for security testing.
- Abstract: The dynamism, pervasiveness and flexibility of modern distributed applications come with increased risks for the security of managed data and resources. Therefore dedicated approaches for security testing are needed in addition to more traditional functional tests. After a brief overview of the most common techniques for security testing, the talk will mainly focus on the validation of mechanisms for access and usage control, and in particular on policy-based approaches using the standard XACML notation. XACML is a platform-independent XML-based language. Several tools for the automated validation of either the specified policies or the PDP (Policy Decision Point) component have been recently proposed. Among different strategies, the rest of the talk will present recent results in combinatorial approaches for test cases (XACML requests) generation as well as in mutation analysis of XACML policies.
- Yves Le Traon. Security testing: a key challenge for software engineering of web apps.
- Wissam Mallouli. Network Monitoring for Security Checking Using MMT.
- Abstract: Network monitoring is a laborious challenging task that is vital for a network operator, a service provider or a corporate network infrastructure in order to keep the network operation stable, smooth and safe. Monitoring provides valuable real time and historical information to understand the network usage trends and dynamics and thus detect misbehaviours and attacks. The vulnerabilities introduced by this “open world”: Critical infrastructures are more than ever open to the Internet, the dematerialization of corporate IT and the success of cloud services are pushing towards proactive mechanisms for detecting and preventing anomalies. In this context, Deep Packet Inspection (DPI) is considered as a catalyser in the shift towards advanced monitoring. DPI is the process of capturing network traffic, analysing and inspecting it closely to determine accurately what is really happening in the network. In this presentation, we will present an events-based network monitoring solution part of MMT tool that inspects network traffic against a set of security properties denoting both security rules and attacks. This solution has been applied to an industrial case study provided by Thales Group that consists of a set of QoS-aware ad-hoc radio communication protocols.
Theme 4: Software Product Line Testing
- Myra Cohen. Sampling, Re-use and Incremental Testing in Software Product Lines.
- Abstract: Software product lines are families of products that are composed from a set of common and variable features, with a well-managed asset base. Testing of software product lines requires testing, not of a single product, but of a set of products. While re-use is the basis of composing products in a software product line, this is not always true of testing. Instead, software product lines are often viewed as individual products during testing and information learned in one test session is lost in the next. In this talk we begin with an overview of software product lines and highlight some of the challenges faced when testing them. We then discuss some of the approaches that have been proposed to overcome these challenges. We examine potential re-use from two perspectives – bottom up, which aims to re-use test cases or test plans, and top down, which re-uses prior test results. We present some top down sampling techniques for product lines that ensure that we have tested a representative set of products. We then show how we can utilize code-level dependence information to reduce the combinatorial space for testing, and how directed integration testing, guided by our analyses, can incrementally increase the overall testing strength of the product line, and lead to targeted test generation. We end with a technique that leverages ideas from regression testing to identify and test the differences between pairs of products, in what we call continuous test suite augmentation.
- Gilles Perrouin. Feature-based Testing of SPLs: Pairwise and Beyond.
- Abstract: Software Product Lines (SPLs) are notoriously difficult to test because of the combinatorial explosion of the number of products one can derive from a limited set of features. As demonstrated in the first talk of this session, sampling techniques such as Combinatorial Interaction Testing (CIT) allow to extract a tractable subset of products for testing. In this presentation, we take a high-level perspective on the problem by combining CIT with a very popular formalism for modeling variability software product lines: Feature Models (FMs).
We will first outline how to adapt CIT ideas to the SPL context using model-based engineering techniques and Alloy. We will then explore competing approaches developed in the meantime and provide some metrics to compare them. Despite huge improvements in the performance and usability of the proposed techniques, scalability remains an issue for large FMs. We propose to use similarity testing and search based techniques to mimic CIT and prioritize results, providing to the tester more control on the product selection process and optimal exploitation of partial results. The side problem of the fidelity of the FM regarding concrete SPL artifacts will be exhibited. Finally, we will sketch a more general perspective where behavioral models are added and in which testing marries with verification to validate SPLs.
Theme 5: Mining software models for analysis and testing + TAROT 2014
- Leonardo Mariani. Automated Failure Analysis in Absence of Specification.
- Rui Abreu. Presenting the TAROT 2014.
A “working group session” is also planned to be run on Wednesday afternoon. A “projects and posters” session is planned for Thursday.