Characterizing testing methods for context-aware software systems: results from a quasi-systematic literature review

Context-Aware Software Systems (CASS) use environmental information to provide better service to the systems’ actors to fulfill their goals. Testing of ubiquitous software systems can be challenging since it is unlikely that, while designing the test cases, the tester can identify all possible context variations. A quasi-Systematic Literature Review has been undertaken to characterize the methods usually used for testing CASS. The analysis and generation of knowledge in this work rely on classifying the extracted information. Established taxonomies of software testing and context-aware were used to characterize and interpret the findings. The results show that, although it is possible to observe the utilization of some software testing methods, few empirical studies are evaluating such methods when testing CASS. The selected technical literature conveys a lack of consensus on the understanding of context and CASS, and on the meaning of software testing. Furthermore, context variation in CASS has only been partially addressed by the identified approaches. They either rely on simulating context or in fixing the values of context variables during testing. We argue that the tests of context-aware software systems need to deal with the diversity of context instead of mitigating their effects.