Code Defenders

Code Defenders is a web-based game in which students practice two central activities in software testing: (1) assessing how well a program is tested by the current test suite, and (2) improving the test suite by creating additional tests. Players can either take the role of attackers, who aim to introduce artificial bugs (“mutants”) into the class under test that reveal weaknesses in the test suite, or the role of defenders wo aim to improve the test suite by adding tests. If a test which passes on the original program fails on a mutant, then that mutant is detected and the defender scores points, whereas the attacker scores points if the mutant is not detected. The number of points a mutant is worth depends on the number of tests it “survives”, which further encourages players to create as subtle as possible mutants, and as strong as possible tests.

Papers and publications

More information about Code Defenders and its use in the classroom can be found in the following publications:

  • Gordon Fraser, Alessio Gambi, José Miguel Rojas. “Teaching Software Testing with the Code Defenders Testing Game: Experiences and Improvements,” In Proc. 1st International Software Testing Education Workshop (TestEd), IEEE, 2020, (PDF).


    ABSTRACT:     Code Defenders is a game that aims to engage students with software testing. Players compete over a Java class under test by producing tests and mutants, i.e., artificial faults, scoring points if tests reveal mutants, or mutants survive tests. While initially created as a fun way to introduce students to testing in one-off fun sessions, we have moved to integrating the game as an assessed core component of a software testing course. This has shown great potential in engaging students, but many challenges have been revealed as part of this integration. In this paper we describe how we integrated Code Defenders into a software testing course, and how we improved the game in order to address the resulting challenges.

  • Gordon Fraser, Alessio Gambi, Marvin Kreis and José Miguel Rojas. “Gamifying a Software Testing Course with Code Defenders,” In Proc. of the ACM Technical Symposium on Computer Science Education (SIGCSE), ACM, 2019 (PDF)

    ABSTRACT: Software testing is an essential skill for software developers, but it is challenging to get students engaged in this activity. The Code Defenders game addresses this problem by letting students compete over code under test by either introducing faults (“attacking”) or by writing tests (“defending”) to reveal these faults. In this paper, we describe how we integrated Code Defenders as a semester-long activity of an undergraduate and graduate level university course on software testing. We complemented the regular course sessions with weekly Code Defenders sessions, addressing challenges such as selecting suitable code to test, managing games, and assessing performance. Our experience and our data show that the integration of Code Defenders was well received by students and led them to thoroughly practice testing. Positive learning effects are evident as student performance improved steadily throughout the semester.

  • Gordon Fraser, Alessio Gambi, José Miguel Rojas. “A Preliminary Report on Gamifying a Software Testing Course with the Code Defenders Testing Game,” In Proc. of The European Conference of Software Engineering Education, pp. 50-54, ACM, 2018. [PDF}

    ABSTRACT:  It is challenging to teach software testing in a way that is engaging for students, and to ensure that they practice effective testing sufficiently. Code Defenders is an educational game that is intended to address this problem: Students compete over code under test by either introducing faults (“attacking”) or by writing tests (“defending”). We have integrated Code Defenders as a mandatory component of a software testing course at the University of Passau, which featured ten game sessions of two hours each and involved 120 students. In this paper, we describe how this integration took place and provide some initial insights into our experiences. Code Defenders and the course material are freely available, allowing others to replicate this setup and to gamify their own testing courses

Technology

Code Defenders is implemented as a web-based game (shown in Figure 1), and can be played in teams or pairs. Players get to see the source code of the Java class under test, with colour highlighting to indicate the coverage of the defenders’ test suite, and with bug-shaped icons labelling the locations and status of the attackers’ mutants. Attackers can create mutants by editing the source code of the Java class, and defenders have a code editor to write tests using the JUnit, which is the de-facto standard test automation framework for the Java programming language. Besides the information displayed with the class under test (code, coverage, and mutant locations), defenders get to see all tests that have been submitted in the game, while attackers see all mutants. There is also a scoreboard that breaks down the game’s current score for each team and player and summarises the status of the mutants.

Getting Code Defenders

Code Defenders can be played online at http://code-defenders.org

Code Defenders is free and open source and can be downloaded from https://github.com/CodeDefenders/CodeDefenders