[DSRP Evidence](https://dsrpevidence.org/)

# Introducing Systems Thinking as a Framework for Teaching and Assessing Threat Modeling Competency

Joshi et al., 2024, ASEE — Computer Science & AI

Patterns: [Distinctions](https://dsrpevidence.org/pattern/distinctions), [Systems](https://dsrpevidence.org/pattern/systems), [Relationships](https://dsrpevidence.org/pattern/relationships), [Perspectives](https://dsrpevidence.org/pattern/perspectives)

## In short

This is direct independent effect evidence for DSRP because DSRP was not merely cited as background theory: it was explicitly taught, operationalized into a D/S/R/P rubric, and used to assess observable student performance. Students exposed to the DSRP-based intervention showed substantially higher aggregate DSRP performance (2.55 vs. 1.65 and 1.45) and greater attention to system-level and interaction-generated threats. The strongest construct-level difference was Relationships (R), where the Fall 2023 group averaged 2.80 compared with 1.40 in both comparison groups.

The evidence should nevertheless be characterized as preliminary/quasi-experimental rather than definitive causal evidence: only five teams per semester were analyzed, assignment prompts changed, and the Fall 2023 cohort simultaneously received DSRP instruction, ThreatModeler software, and assessment changes. The authors themselves describe these as preliminary results and propose longitudinal testing and further construct-validity work.

## What they found (results)

Students receiving STRIDE alone showed generally beginner-to-intermediate systems-thinking performance, with mean DSRP scores of 1.65 (Fall 2021) and 1.45 (Spring 2023). Students receiving systems-thinking plus STRIDE instruction averaged 2.55, with component scores of D = 2.60, S = 2.40, R = 2.80, and P = 2.40.

The authors conclude that students receiving both systems-thinking and STRIDE instruction "performed better" at holistically identifying and mitigating both component- and system-level risks. They specifically argue that adding a one-week systems-thinking module helped students identify system-level threats and threats arising from relationships among components.

## What they set out to do (purpose)

Joshi et al. investigated whether adding explicit DSRP systems-thinking instruction to STRIDE threat-modeling instruction improved upper-level software-engineering students' ability to identify and mitigate system-level as well as component-level cybersecurity threats. They developed separate STRIDE and DSRP assessment rubrics and applied them to student security-case artifacts from Fall 2021, Spring 2023, and Fall 2023. Five team projects were randomly selected from each semester, for 15 analyzed projects. Fall 2021 and Spring 2023 students received STRIDE instruction without systems-thinking instruction; Fall 2023 students received both.

One important methodological caveat: Fall 2023 differed in more than DSRP instruction. The instructor also introduced ThreatModeler tooling and changed the assessment prompts to elicit more detailed reasoning. So this is meaningful independent empirical evidence, but not a clean randomized causal test of DSRP alone.

## Abstract

Computing systems face diverse and substantial cybersecurity threats. Software engineers can mitigate some of these threats through appropriate software design and analysis, provided they are trained in appropriate competencies. One fundamental cybersecurity competency is threat modeling (Xiong & Lagerström, 2019), which is a systematic approach to identifying, mapping, and mitigating design-level security problems (Soares Cruzes et al., 2018). There are many frameworks for teaching threat modeling, but our analysis of these frameworks and existing coursework suggests that (1) these approaches tend to be focused on component-level analysis rather than educating students to reason holistically about a system’s cybersecurity, and (2) there is no rubric for assessing a student’s threat modeling competency.

To address these concerns, we propose systems thinking as a framework for teaching and assessing threat modeling competency. Prior studies by Young & Leveson (2013) and Yan (2020) suggest systems thinking can be a suitable approach for understanding and mitigating cybersecurity threats. Further, Tisdale (2015) synthesizes literature to argue that a holistic approach like systems thinking is needed to address cybersecurity risks. The purpose of this work-in-progress study is therefore to develop and pilot a rubric that uses systems thinking as a way to assess the threat modeling approach of computer engineering students. Based on our findings, we also discuss how systems thinking could be integrated into the teaching of threat modeling.

To conduct this study, we are developing a novel rubric for assessing threat modeling competency based on systems thinking (e.g., System Engineering approach (Ross et al., 2018)). We will use this rubric to assess threat models created during upper-level software design projects at a large midwestern university in the USA (24 student teams in Fall 2021 and 37 student teams in Spring 2023). We will compare these scores to the baseline rubric used in the course, which was derived directly from the industry standard STRIDE threat modeling framework. Our work will contribute by helping educators understand: (1) trends in threat modeling approaches undertaken by students; (2) identifying blindspots in their threat modeling approach; (3) describing a new rubric for assessing threat modeling based on systems thinking; and (4) envisioning in detail the opportunity for using systems thinking in threat modeling teaching and assessment.

These researchers were not testing DSRP. The finding is theirs; the correspondence to DSRP is drawn by this site.

[Source](https://peer.asee.org/introducing-systems-thinking-as-a-framework-for-teaching-and-assessing-threat-modeling-competency)