A controlled experiment to explore potentially undetectable defects for testing techniques

Martín Solari, Santiago Matalonga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Software testing practitioners have an array of testing techniques to choose from to test their software. Nevertheless, there is little empirical evidence about the capability of each technique to detect specific types of defects. As a result, when selecting and combining the testing techniques for a project, practitioners must rely on their own experience. This paper studies the behaviour of two specific techniques, equivalence partitioning and decision coverage, to determine which types of defect are potentially undetectable to either one. This paper presents a differentiated experiment replication based on a previous experimental design, but using different artifacts. The experiment confirms the hypothesis that some defect types are undetectable to each technique. Even with a correct application of each technique, some defects will only be detected by chance. This study adds new empirical evidence for constructing a classification of defects that takes into account technique detection capabilities.
Original languageEnglish
Title of host publicationSEKE 2014
Subtitle of host publicationProceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering
Place of PublicationSkokie
PublisherKnowledge Systems Institute Graduate School
Pages106-109
Number of pages4
ISBN (Print)1891706357
Publication statusPublished - 2014
Externally publishedYes
EventThe 26th International Conference on Software Engineering & Knowledge Engineering - Hyatt Regency, Vancouver, Canada
Duration: 1 Jul 20143 Jul 2014

Conference

ConferenceThe 26th International Conference on Software Engineering & Knowledge Engineering
Abbreviated titleSEKE 2014
CountryCanada
CityVancouver
Period1/07/143/07/14

Fingerprint

Defects
Testing
Experiments
Software testing
Design of experiments

Keywords

  • software testing
  • experiment
  • defect detection

Cite this

Solari, M., & Matalonga, S. (2014). A controlled experiment to explore potentially undetectable defects for testing techniques. In SEKE 2014: Proceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering (pp. 106-109). Skokie: Knowledge Systems Institute Graduate School.
Solari, Martín ; Matalonga, Santiago. / A controlled experiment to explore potentially undetectable defects for testing techniques. SEKE 2014: Proceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering. Skokie : Knowledge Systems Institute Graduate School, 2014. pp. 106-109
@inproceedings{6a5875a0cc944953a55091575b951155,
title = "A controlled experiment to explore potentially undetectable defects for testing techniques",
abstract = "Software testing practitioners have an array of testing techniques to choose from to test their software. Nevertheless, there is little empirical evidence about the capability of each technique to detect specific types of defects. As a result, when selecting and combining the testing techniques for a project, practitioners must rely on their own experience. This paper studies the behaviour of two specific techniques, equivalence partitioning and decision coverage, to determine which types of defect are potentially undetectable to either one. This paper presents a differentiated experiment replication based on a previous experimental design, but using different artifacts. The experiment confirms the hypothesis that some defect types are undetectable to each technique. Even with a correct application of each technique, some defects will only be detected by chance. This study adds new empirical evidence for constructing a classification of defects that takes into account technique detection capabilities.",
keywords = "software testing, experiment, defect detection",
author = "Mart{\'i}n Solari and Santiago Matalonga",
year = "2014",
language = "English",
isbn = "1891706357",
pages = "106--109",
booktitle = "SEKE 2014",
publisher = "Knowledge Systems Institute Graduate School",
address = "United States",

}

Solari, M & Matalonga, S 2014, A controlled experiment to explore potentially undetectable defects for testing techniques. in SEKE 2014: Proceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering. Knowledge Systems Institute Graduate School, Skokie, pp. 106-109, The 26th International Conference on Software Engineering & Knowledge Engineering, Vancouver, Canada, 1/07/14.

A controlled experiment to explore potentially undetectable defects for testing techniques. / Solari, Martín; Matalonga, Santiago.

SEKE 2014: Proceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering. Skokie : Knowledge Systems Institute Graduate School, 2014. p. 106-109.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A controlled experiment to explore potentially undetectable defects for testing techniques

AU - Solari, Martín

AU - Matalonga, Santiago

PY - 2014

Y1 - 2014

N2 - Software testing practitioners have an array of testing techniques to choose from to test their software. Nevertheless, there is little empirical evidence about the capability of each technique to detect specific types of defects. As a result, when selecting and combining the testing techniques for a project, practitioners must rely on their own experience. This paper studies the behaviour of two specific techniques, equivalence partitioning and decision coverage, to determine which types of defect are potentially undetectable to either one. This paper presents a differentiated experiment replication based on a previous experimental design, but using different artifacts. The experiment confirms the hypothesis that some defect types are undetectable to each technique. Even with a correct application of each technique, some defects will only be detected by chance. This study adds new empirical evidence for constructing a classification of defects that takes into account technique detection capabilities.

AB - Software testing practitioners have an array of testing techniques to choose from to test their software. Nevertheless, there is little empirical evidence about the capability of each technique to detect specific types of defects. As a result, when selecting and combining the testing techniques for a project, practitioners must rely on their own experience. This paper studies the behaviour of two specific techniques, equivalence partitioning and decision coverage, to determine which types of defect are potentially undetectable to either one. This paper presents a differentiated experiment replication based on a previous experimental design, but using different artifacts. The experiment confirms the hypothesis that some defect types are undetectable to each technique. Even with a correct application of each technique, some defects will only be detected by chance. This study adds new empirical evidence for constructing a classification of defects that takes into account technique detection capabilities.

KW - software testing

KW - experiment

KW - defect detection

M3 - Conference contribution

SN - 1891706357

SP - 106

EP - 109

BT - SEKE 2014

PB - Knowledge Systems Institute Graduate School

CY - Skokie

ER -

Solari M, Matalonga S. A controlled experiment to explore potentially undetectable defects for testing techniques. In SEKE 2014: Proceedings of the Twenty-Sixth International Conference on Software Engineering & Knowledge Engineering. Skokie: Knowledge Systems Institute Graduate School. 2014. p. 106-109