[DSRP Evidence](https://dsrpevidence.org/) # A Method for Measuring Systems Thinking Learning ## Details **Authors** Taylor, Seth and Calvo-Amodio, Javier and Well, Jay **Year** 2020 **Publisher** Systems (MDPI), vol. 8, no. 2, article 11 **Kind of work** article **Discipline** Education **Secondary disciplines** Systems Science, Engineering **Kind of engagement** scholarly **Applied** false [Read it at the publisher](https://doi.org/10.3390/systems8020011) 10.3390/systems8020011 ## In authors' words ### What they set out to do (purpose) To define a method for measuring whether non-experts learn systems thinking, on the authors' premise that complex cross-disciplinary problems require systems thinking and that its spread is held back by low awareness and shallow understanding of it. ### Who or what was studied (sample) 97 middle school and high school students, taught by multiple instructors. ### How they did it (methods) Pre-post quasi-experiment. Students drew a fish-tank system before instruction (Drawing A) and again after it (Drawing B). The authors defined systems thinking as four skills drawn from DSRP and layered a three-level learning scheme over them (sensibility, literacy, capability). Drawings were scored by counting items in three categories — elements, interactions, and roles/purposes — with each item classed at one of the three levels and weighted 1, 2 or 3. Wilcoxon signed-rank tests compared the paired drawings; ANOVA tested for instructor and grade-band effects. ### What they found (results) Drawing B scores were significantly higher than Drawing A (p < 0.001 at alpha = 0.05, two-tailed and right-tailed). Students identified roughly two to three more elements, one to two more interactions, two more roles or purposes, and about six more items in total, with total scores rising ten to twelve points. The largest gain fell at the literacy level. ANOVA found no significant instructor effect; middle and high school students improved by comparable amounts, with high schoolers scoring higher overall, which the authors attribute to recall. The authors conclude that teaching the patterns as skills increased systems thinking learning, and that students reached knowledge of the concepts without sufficient understanding to reach capability. ## Commentary ### In short Three engineering researchers build a measurement instrument on DSRP and test it on 97 middle and high school students, scoring before-and-after drawings of a fish tank for distinctions, relationships and perspectives. ### In more detail Taylor, Calvo-Amodio and Well take DSRP as the definition of what systems thinking consists of and then ask whether that definition can be measured in people who are not systems thinkers. They restate the four rules in the Cabreras' own terms — that any idea or thing can be distinguished from the others it is with, split into parts or lumped into a whole, related to other things, and made the point or the view of a perspective — and credit the framework to Cabrera. They then convert three of the four into countable evidence in a drawing: distinctions become the elements a student identifies, relationships become the interactions, and perspectives become the roles or purposes attributed to parts of the system. Each identified item is also placed at one of three learning levels of the authors' own devising, so that a student who names a component scores lower than one who explains what it does for the whole. Ninety-seven students drew a fish tank before and after instruction in the skills, and the paired drawings were compared. The result is a significant gain across all three categories, concentrated at the middle learning level. The study's deliberate omission is the Systems rule: the authors state that students were not explicitly taught the concept of systems, and part-whole organization accordingly has no counterpart in the scoring. What the instrument tests is therefore three of the four patterns, taught and counted, in a population of eleven- to eighteen-year-olds. ### Response The paper takes DSRP at its word on the point that matters most for teaching: that the patterns are skills rather than topics, and that a skill taught can be counted in what a learner produces. Its scoring scheme is the useful part — elements for distinctions, interactions for relationships, roles and purposes for perspectives — because it turns each operator into evidence a second rater could check in a drawing. The omission is consequential and the authors name it: systems was not taught and is not scored. Part-whole organization is the pattern doing the heaviest work in a fish tank, which nests inside a room, a building and a water cycle, and a measure that leaves it out cannot report on systems thinking as the theory defines it, only on three of its four operators. The three-level scheme of sensibility, literacy and capability is likewise the authors' addition rather than anything DSRP supplies, and the gain they report at the literacy level is a finding about their own scale as much as about the patterns. **Patterns it shows** D, R, P Shows it makes a difference **Added** 2026-10-14 **How to cite this** Taylor, Seth and Calvo-Amodio, Javier and Well, Jay (2020). A Method for Measuring Systems Thinking Learning. Systems (MDPI), vol. 8, no. 2, article 11.