This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

Abstract. Instructional physics labs are critical junctures for many STEM majors to develop an understanding of experimentation in the sciences. Students can acquire useful experimental skills and grow their identities as scientists. However, many traditionally-instructed labs do not necessarily involve authentic physics experimentation features in their curricula. Recent research calls for a reformation in undergraduate labs to incorporate more student agency and choice in the learning processes. In our institution, we have adopted open-ended lab teaching in the introductory physics courses. By using reformed curricula that provide higher student agency, we analyzed approximately 100 students in the introductory-level lab courses to examine their views towards the open-ended physics labs. Between the start and the end of the semester, we found a statistically significant shift in students’ perceptions about the agency afforded in lab activities. We also examined students’ responses to "Which lab unit was your favorite and why?". The analysis showed that majority of the students preferred Project Lab, which had the highest student agency and coding analysis showed that "freedom" was the most frequent response for students’ reason for picking Project Lab. Finally, we also examined student views across gender and found no significant gender effect on students’ sense of agency.

Sense of agency, gender, and students’ perception in open-ended physics labs

Z. Yasemin Kalender    Martin Stein    N. G. Holmes Laboratory of Atomic and Solid State Physics, Department of Physics, Cornell University, Ithaca, NY, 14850

I Introduction

Scientific thinking is any instance of purposeful thinking that has the objective of enhancing the seeker’s knowledge Khun (2011), and involves many independent and critically engaged decision making processes. Scientific decisions may relate to building models, designing experiments, interpreting results, or communicating the ideas AAPT (2014), among others. To make such decisions, agency plays a critical role, where agency is defined here as the capacity to guide one’s actions towards achieving a goal Bandura (1989, 1996). To learn to think like a scientist, students need to learn how to make such decisions using their agency.

Undergraduate labs offer a great opportunity to gain practice with such decision making. However, the curricula of many physics lab courses do not satisfy these goals, because they are highly-structured, procedural, and content-reinforcing PCA (2012). Therefore, in recent years, there has been a growing recognition for the need to reform undergraduate physics curricula to adopt more open-ended and student-centered tasks and activities.

Open-ended (i.e., interactive or constructive) lab instruction can offer students more choice and agency as they engage with the material. Open-ended labs can benefit students in multiple ways, from developing more expert-like attitudes towards experimentation Wilcox and Lewandowski (2017) to developing positive science identities Doucette et al. (2020); Carlone and Johnson (2007); Kalender et al. (2019a). Supporting students to enact their agency, however, is more than simply removing structure. Students must have the motivation and self-efficacy to take up the agency afforded Bandura (1989). In the present work, we studied students’ beliefs and perspectives about the agency in a physics lab course and compared across different demographics.

II Literature review and research questions

In an open-ended lab, students have control over what they do and what they find in their experiment. Students have more instances to use their agency to become the initiator of their learning processes Tapal et al. (2017). Affording such agency in lab classes can better engage students in activities that interest them Scardamalia and Bereiter (2006) and enhance their learning of science practices and career outcomes Zeiser et al. (2018). Students who perceive themselves as having agency will have more ownership in their learning Hanauer and Dolan (2014). Students with a sense of ownership will become more invested, engaged, and persist in their work Hanauer et al. (2016); Dounas-Frazer et al. (2017).

Supporting agency, however, does not mean full autonomy on students’ part Bandura (1989). In science education, the goal is not simply for students to make decisions – the goal is for them to learn to make decisions in line with scientific practice Ford (2015). Thus, students should be supported in making their own decisions in open-ended course work  Holmes et al. (2020). The literature on social learning suggests using teaching processes such as modeling, coaching, scaffolding, and then slowly fading support Collins et al. (1987). As students develop higher-order skills and learning activities become more complex, students transition from novices to more independent and expert-level learners  Ericsson et al. (1993); Chi et al. (1981) and they become more autonomous (i.e., higher agency) scientists. This apparent structure may seem antagonistic to students’ agency. Given that open-ended labs are rarely fully ‘open’, in what ways are students aware of the agency afforded to them in labs?

Agency in a lab is also influenced by the learning environment and social-classroom dynamics and hierarchies Schenkel and Calabrese Barton (2020), with such dynamics and hierarchies being imported from the wider cultural context in sciences  Seymour and Hewitt (1997). If one student takes charge in a group, other students might not perceive the agency afforded to them Archibeque et al. (2017, 2018). In particular, members of marginalized groups might perceive themselves as having less agency to participate in tasks or group discourses because of their peers. For example, in physics, researchers have found gendered patterns in students’ roles in lab activities such that female students are assigned to "disempowering" roles Danielsson (2012). The accumulation of many such interactions may impact male and female students’ sense of agency differently.

Finally, a critical question to the discussion of agency is whether students value and recognize the benefit of provided agency in physics labs. On one hand, positive attitudes toward agency in physics can influence students’ reactions to challenges and setbacks Zeiser et al. (2018); Dweck (2008); Elliott and Dweck (1988) and students who view agency as positive can become more active-seekers of new knowledge and feel more positively toward self-learning strategies Bandura (1989). On the other hand, prior literature shows that students in active-learning classes often feel less positive about and see less value in self-guided learning compared to traditional learning Cavanagh et al. (2016), despite the fact that they gain more knowledge in the former Deslauriers et al. (2019).

In this study, we focused on the understudied topic of students’ sense of agency in open-ended physics lab courses. We examined three primary research questions.

  • RQ1: How do open-ended labs impact students’ sense of agency? While prior literature emphasizes the importance of agency, there is minimal work on how to actually promote students’ sense of agency. To address this, we examined whether open-ended labs can be one method of increasing students’ sense of agency in physics.

  • RQ2: How does students’ sense of agency in these labs interact with gender? As noted above, student demographics can play an important role in how students engage in physics labs activities. To unpack these complexities in the role of agency in physics labs, we tested for equity of parity (i.e., equity of outcomes) Rodriguez et al. (2012); Van Dusen and Nissen (2019); Burkholder et al. (2020) to examine whether men and women perceive labs differently at the beginning and end of the instruction.

  • RQ3: To what extent do students value having agency in open-ended physics lab? During a lab course, different assignments will vary in the amount of agency afforded to students. This provides an opportunity to test how students perceive agency by asking for their preferred assignments and the reasoning for those preferences.

III Methods

III.1 Participants and lab context

The participants were students in the first semester calculus-based honors physics course sequence at a large research university, who mostly intended to major in physics. There were 88 and 66 students who took the pre and post-test. Students self-identified their gender on the survey and approximately 30% of the classroom identified as female, one student preferred not to disclose their gender, and the remaining students identified as male. Labs were combined with the main lecture course and taught by three graduate teaching assistants. Each lab session was two hours long and each lab unit typically spanned two sessions. In a 15-week semester, there were 9 weeks of lab. Students conducted their experiments in groups of 2-3 students. Between each lab unit, students picked different lab partners. Each unit focused on one or multiple goals such as Model Testing, Ethics, or Model Extending. Afforded student agency in each lab unit (Lab 1, Lab 2, Lab 3) was gradually increased towards a student-guided "Project Lab" unit. In this final Project Lab unit, students picked their own research question and topic and designed the whole investigation. In the final lab session, students presented their project results to the whole classroom. More detailed information about each unit can be found on PhysPort Holmes (2020).

III.2 Survey development and validation

Table 1: SoA items and their factor loadings and p-values are shown below.
Items Factor loadings p-values
I am in control of setting the goals for the experiments. 0.859 < 0.001
I have the freedom to design and conduct the best possible experiment to attain my goals. 0.860 < 0.001
I am in control of choosing the appropriate analysis tools to evaluate experimental outcomes. 0.733 < 0.001
I am in control of doing interesting experiments in a physics lab. 0.723 < 0.001

We administered a short survey in the first and last lab session to measure students’ attitudes towards experimentation in physics. We developed Sense of Agency (SoA) survey items to measure students’ beliefs about the agency afforded in a physics lab context, adapted from an existing survey Tapal et al. (2017). We conducted interviews with three undergraduate and two graduate students to ensure that students interpret the items in the way we intend. These initial student interviews helped us modify the SoA items. The SoA items used a Likert Scale and they were chosen to measure different aspects of physics labs such as setting goals, designing and conducting experiments, and choosing analysis tools. We also conducted tests of internal consistency and construct validity by using Cronbach-alpha Cronbach and Meehl (1955). The inter-item reliability between SoA questions was 0.87 in the pre-test scores, 0.89 in the post-test, which were considered "good" Cronbach (1951). The survey included additional constructs (e.g., self-efficacy, mindset), and initial factor analysis in R RCoreTeam (2013) showed separability between constructs. The fit parameters for confirmatory factor analysis were CFI = 0.928, TLI = 0.915, RMSEA = 0.078, SRMR = 0.066, which are in the range of acceptable model fit Hooper et al. (2008). Based on the reliability results, among five of the SoA items, four of them met the psychometric standards so we used in our analysis. The items and their factor loading values are given in Table 1.

In the post attitude survey, we included an additional open-ended question about students’ favorite lab unit and asked for a short explanation of why they liked that particular lab unit. We also interviewed students later in the semester to better understand their experiences in these labs. We are not doing a formal analysis of these data here but will present some student quotes in the discussion section that are relevant to the current study.

III.3 Analysis

III.3.1 Students’ Sense of Agency

In order to answer RQ1 and RQ2, we performed tt-tests and calculated effect sizes (i.e., Cohen’s d Cohen (1998)) and confidence intervals on whether students’ sense of agency changed from pre to post survey, and whether the scores differed between male and female students at either time point. We use alpha value of 0.05 as a threshold for statistical significance level.

III.3.2 Coding for "Favorite Lab"

To answer RQ3, we analyzed the open-ended post survey question regarding students’ favorite lab and generated a word cloud by using a library in R RCoreTeam (2013). The word cloud serves only to provide a qualitative description of students’ responses. We emergently and iteratively created a coding scheme from students’ open-ended responses. The final coding scheme is presented in Table 2. Note that multiple codes could apply to a single response. Two authors independently coded all of the items with the final coding scheme. An inter-rater reliability showed "good" or "excellent" agreement between the coders. Table 2 shows percentage of agreement and Cohen’s Kappa values. All disagreements were discussed to reach consensus.

Table 2: Coding table with code names, their definition, and an example sentence from students. Percent agreement and Cohen’s Kappa values are also given for an inter-reliability to measure between the coders. A particular student response can be labeled with multiple codes.
Code name Definition Example Percent Cohen’s
Agreement Kappa
Learning concepts Experiment helped learning theory and/or concepts "Allowed me to study aspects of Hooke’s law that I did know before" 94.8 0.69
Interest Experiment was fun / interesting / enjoyable / engaging "I was genuinely excited by the material…" 92.2 0.79
Logistics Experiment was short / organized (or not) / clear (or not) / structured (or not) "It was more organized …" 97.4 0.79
Epistemological belief Experiment provoked the most critical thinking, had unexpected results "That unit proved to me that the lab is also about disconfirming predictions for an experiment…." 93.5 0.67
Group work Experiment was the favorite because of group mates "Good group, worked well together…" 97.4 0.82
Freedom Flexibility, creativity regarding the research questions, design or in general the unit itself "Because in project lab session, we have the freedom to explore the subject we are interested in and be able to design our own experiment." 98.7 0.97
Other No response/"No lab in high school"/too vague to code "Better than the other ones" 90.0 0.76

IV Results

To answer RQ1, we first checked the variance between pre and post scores and found no difference (p = 0.213). Then we performed Students’ tt-test where we assumed equal variance based on the previous analysis and found that students’ average SoA scores increased from MM = 2.96, SDSD = 0.98 at pre-test to post-test MM = 3.73, SDSD = 0.84 with t(56)=5.29t(56)=5.29, p < 0.001. The mean difference in SoA between pre and post test is 0.77 with 95% Confidence Interval [0.48,1.07] (see Fig 1). The effect size between pre and post SoA scores is 0.76, which is considered as between "medium" and "large" Cohen (1998).

To address RQ2, we compared SoA scores between female and male students at both pre and post-test separately. We did not find any gender difference in students’ SoA either in the pre-test tt(84) = 0.01, p = 0.99 or the post-test tt(64) = 0.35, p = 0.72.

When we looked at the distribution of students’ favorite lab, we found that the majority of the students (n = 45) chose the Project Lab unit as their favorite, followed by Lab 3 (n = 30). The word cloud analysis presented in Fig. 2 indicates that students most frequently stated "freedom" as the reason for why they liked any particular lab unit. Based on our coding analysis, as seen in Figure 3, the most common explanation given by students who selected the Project Lab as their favorite unit was the freedom that the lab afforded them. "Interest" was the second most common explanation. The first two lab units were selected by very few students as their favorite lab, 6 and 10 students, respectively. The coding patterns for Lab 1 and Lab 2 were very similar.

Refer to caption
Figure 1: Raincloud plot Allen et al. (2018) presents the shift in students’ Sense of Agency scores from pre to post-test by gender. Box plot, error bars, and the distribution of students’ scores are included in the plot. The dotted lines shows the trend in students’ agency belief shift by gender.
Refer to caption
Figure 2: Using rquery.wordcloud() function, word cloud generator shows most frequently used word for students’ open-ended responses to why they liked a particular lab unit. "Freedom" is the most frequently written word, i.e., 25% students wrote "freedom" for why they liked a particular lab unit.
Refer to caption
Figure 3: Coding results for students’ explanation to their preferred lab unit. There are seven main codes: Learning concept (L), Interest(I), Epistemology (E), Group (G), Freedom (F), Logistic (L) and Other (O). There are four lab units (Lab 1, Lab2, Lab3, and Project Lab) and each unit takes two weeks to complete. Number of students who choose specific lab as their favorite unit is shown with "n".

V Conclusion and Discussion

How students take up agency and whether they view agency as positive is critical in physics lab courses Dounas-Frazer et al. (2017); Zeiser et al. (2018). We found support that open-ended labs can increase students’ perceptions about their agency. Specifically, addressing RQ1, we examined the relationship between students’ SoA and time, and found that students indicated having more agency (or a greater sense of agency) by the end of the course. In interviews at the end of the semester, many students referred to the notion of freedom explicitly, such as: "There was more freedom for us to choose how to design and carry out an experiment." Given that prior literature has shown students’ attitudes in physics tend to decline as the semester continues Adams et al. (2006), our findings provide evidence that open-ended labs can elevate students’ views towards agency, potentially buffering against these typical declines in motivation.

In order to address RQ2, we also tested whether students’ SoA varied between male and female students. Although previous work suggests that female students experience a larger drop in attitudes than male students in physics courses Kalender et al. (2019b), we found that both female and male students’ sense of agency increased over the semester. Given that male students typically dominate group conversations in physics Heller and Hollabaugh (1992), one possibility that could have happened was that male students could have increased in their SoA more than female students. However, we found that there were no gender differences in SoA scores, both at the start and at the end of the semester. This suggests that open-ended labs with student agency and frequent peer collaboration can help both female and male students’ motivation. We note that this analysis focused on the gender as binary, however, we do acknowledge that gender is a fluid, multi-level, and deeply complex construct.

The design and instruction of these open-ended labs incrementally reduced structure, making room for student agency; instruction built towards the student-guided "Project Lab" unit instead of beginning with it. Thus, we examined students’ reasoning for their preferred lab in addressing RQ3. We found that students most frequently preferred the Project lab unit, suggesting that they embraced self-directed learning (i.e., freedom, see Fig.3). Students’ responses to why they liked Project Lab most often include "freedom" in terms of coming up with their own research question, designing experiments, or being able to be creative in their experimentation. Even though prior work found that students resist against self-guided learning Seidel and Tanner (2013), our results suggest that this might not always be the case and that the particular method of implementing self-guided learning is crucial. Students may have felt positively, rather than frustrated, about the open-endedness because the student agency was expanded in each unit and students were never fully autonomous. Prior work highlights the importance of balancing structure and open-endedness in teaching via gradual increase in student agency Collins et al. (1987). In an interview at the end of the semester, one student described the balance as follows: "I definitely think that it is a good idea to have structure. That being said, I created the structure myself and am able to bend it. Maybe designing our labs has helped. That feels like we had ownership."

Classroom activities with hands-on tasks offer opportunities to teach students to be more agentic in their learning processes. The learning environment and interactions with peers during class can enhance or hinder student agency, which can influence their confidence and identity Godwin et al. (2016). Particularly, first-year college experiences can have important downstream consequences regarding motivation in specific courses and broader academic (e.g., enrolling in more physics classes) and career (e.g., retention in STEM fields) choices Godwin et al. (2016); Hazari et al. (2010). Other research indicates that such agency-enhancing activities facilitate student learning, and the research here indicates that students are likely to perceive that agency positively and without differential impacts by gender.

Acknowledgements.
This material is based upon work supported by the National Science Foundation under Grant No. 1836617. We also thank the Cornell Active Learning Initiative for their support. Finally, we thank the Cornell Physics Education Research Lab members for their constructive feedback throughout this study

References

  • Khun (2011) D. Khun, What is scientific thinking and how does it develop (Wiley-Blackwell, New York, 2011).
  • AAPT (2014) AAPT, AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum (2014).
  • Bandura (1989) Albert Bandura, “Human agency in social cognitive theory.” American Psychologist 44, 1175–1184 (1989).
  • Bandura (1996) Albert Bandura, “Reflections on Human Agency: Part I,” Constructivism in the Human Sciences 1, 3–12 (1996).
  • PCA (2012) “Engage to Excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics,”  (2012).
  • Wilcox and Lewandowski (2017) B. R. Wilcox and H. J. Lewandowski, “Developing skills versus reinforcing concepts in physics labs: Insight from a survey of students’ beliefs about experimental physics,” Phys. Rev. PER 13, 010108 (2017).
  • Doucette et al. (2020) Danny Doucette, Russell Clark,  and Chandralekha Singh, “Hermione and the secretary: how gendered task division in introductory physics labs can disrupt equitable learning,” European Journal of Physics 41, 035702 (2020).
  • Carlone and Johnson (2007) Heidi B. Carlone and Angela Johnson, “Understanding the science experiences of successful women of color: Science identity as an analytic lens,” Journal of Research in Science Teaching 44, 1187–1218 (2007).
  • Kalender et al. (2019a) Z. Yasemin Kalender, Emily Marshman, Christian D. Schunn, Timothy J. Nokes-Malach,  and Chandralekha Singh, “Why female science, technology, engineering, and mathematics majors do not identify with physics: They do not think others see them that way,” Phys. Rev. Phys. Educ. Res. 15, 020148 (2019a).
  • Tapal et al. (2017) A. Tapal, O. Ela, D. Dar,  and B. Eitam, “The sense of agency scale: A measure of consciously perceived control over one’s mind, body, and the immediate environment,” Frontiers in Psychology 8, 1552 (2017).
  • Scardamalia and Bereiter (2006) M. Scardamalia and C. Bereiter, The Cambridge handbook of: The learning sciences, edited by R. K. Sawyer (Cambridge University Press, New York, 2006).
  • Zeiser et al. (2018) Kristina Zeiser, Carrie Scholz,  and Victoria Cirks, Maximizing Student Agency Implementing and Measuring Student-Centered Learning Practices (2018).
  • Hanauer and Dolan (2014) David I. Hanauer and Erin L. Dolan, “The project ownership survey: Measuring differences in scientific inquiry experiences,” CBE-Life Sci Educ 13, 149–158 (2014).
  • Hanauer et al. (2016) David I. Hanauer, Mark J. Graham,  and Graham F. Hatfull, “A measure of college student persistence in the sciences (pits),” CBE-Life Sciences Education 15, ar54 (2016).
  • Dounas-Frazer et al. (2017) Dimitri R. Dounas-Frazer, Jacob T. Stanley,  and H. J. Lewandowski, “Student ownership of projects in an upper-division optics laboratory course: A multiple case study of successful experiences,” Phys. Rev. Phys. Educ. Res. 13, 020136 (2017).
  • Ford (2015) Michael J. Ford, “Educational implications of choosing "practice" to describe science in the next generation science standards,” Science Education 99, 1041–1048 (2015).
  • Holmes et al. (2020) N. G. Holmes, Benjamin Keep,  and Carl E. Wieman, “Developing scientific decision making by structuring and supporting student agency,” Phys. Rev. Phys. Educ. Res. 16, 010109 (2020).
  • Collins et al. (1987) A. Collins, J. S. Brown,  and S. E. Newman, Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics (Technical Report No. 403) (Centre for the Study of Reading, University of Illinois, Cambridge, MA, 1987).
  • Ericsson et al. (1993) K. Anders Ericsson, Ralf Thomas Krampe,  and Clemens Tesch-Römer, “The role of deliberate practice in the acquisition of expert performance,” Psychological Review 100, 363–406 (1993).
  • Chi et al. (1981) Michelene Chi, Paul J. Feltovich,  and Robert Glaser, “Categorization and representation of physics problems by experts and novices,” Cognitive science 5, 121–152 (1981).
  • Schenkel and Calabrese Barton (2020) Kathleen Schenkel and Angela Calabrese Barton, “Critical science agency and power hierarchies: Restructuring power within groups to address injustice beyond them,” Science Education 104, 500–529 (2020).
  • Seymour and Hewitt (1997) E. Seymour and N. M. Hewitt, Talking about Leaving: Why Undergraduates Leave the Sciences (Westview Press, Boulder, CO, 1997).
  • Archibeque et al. (2017) B. A. Archibeque, M. B. Kustusch, F. Genz, S. Franklin,  and E. C. Sayre, “Quantitative measures of equity in small group,” in Physics Education Research Conference 2017, edited by A. Traxler, Y. Cao,  and L Ding (Cinncinati,OH, 2017).
  • Archibeque et al. (2018) B. A. Archibeque, M. B. Kustusch, F. Genz, S. Franklin,  and E. C. Sayre, “Qualitative measures of equity in small group,” in International Society of the Learning Sciences (2018).
  • Danielsson (2012) Anna T. Danielsson, “Exploring woman university physics students ’doing gender’ and ’doing physics’,” Gender and Education 24, 25–39 (2012).
  • Dweck (2008) Carol S Dweck, Mindset: The New Psychology of Success (Random House Digital, Inc., 2008).
  • Elliott and Dweck (1988) Elaine S Elliott and Carol S Dweck, “Goals: An approach to motivation and achievement.” Journal of Personality and Social Psychology 54, 5 (1988).
  • Cavanagh et al. (2016) Andrew J. Cavanagh, Oriana R. Aragon, Xinnian Chen, Brian Couch, Mary Durham, Aiyana Bobrownicki, David I. Hanauer,  and Mark J. Graham, “Student buy-in to active learning in a college science course,” CBE-Life Sci Educ 15, ar76 (2016).
  • Deslauriers et al. (2019) Louis Deslauriers, Logan S. McCarty, Kelly Miller, Kristina Callaghan,  and Greg Kestin, “Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom,” Proceedings of the National Academy of Sciences 116, 19251–19257 (2019).
  • Rodriguez et al. (2012) Idaykis Rodriguez, Eric Brewe, Vashti Sawtelle,  and Laird H. Kramer, “Impact of equity models and statistical measures on interpretations of educational reform,” Phys. Rev. ST Phys. Educ. Res. 8, 020103 (2012).
  • Van Dusen and Nissen (2019) Ben Van Dusen and Jayson Nissen, “Modernizing use of regression models in physics education research: A review of hierarchical linear modeling,” Phys. Rev. Phys. Educ. Res. 15, 020108 (2019).
  • Burkholder et al. (2020) Eric Burkholder, Cole Walsh,  and N. G. Holmes, “Examination of quantitative methods for analyzing data from concept inventories,” Phys. Rev. Phys. Educ. Res. 16, 010141 (2020).
  • Holmes (2020) N. G. Holmes, Thinking Critically in Physics Labs (2020).
  • Cronbach and Meehl (1955) L. J. Cronbach and P. E. Meehl, “Construct validity in psychological tests,” Psychological Bulletin 52, 281–302 (1955).
  • Cronbach (1951) L. J. Cronbach, “Coefficient alpha and the internal structure of tests,” Psychometrika 16, 297–334 (1951).
  • RCoreTeam (2013) RCoreTeam, R: A Language and Environment for Statistical Computing (2013).
  • Hooper et al. (2008) D. Hooper, Coughlan J.,  and Mullen M., “Structural equation modeling: Guidelines for determining model fit,” Electronic Journal on Business Research Methods 6, 53 (2008).
  • Cohen (1998) J. Cohen, Statistical Power Analysis for the Behavioral Sciences (L. Erlbaum Associates, 1998).
  • Allen et al. (2018) Micah Allen, Davide Poggiali, Kirstie Whitaker, Tom Rhys Marshall,  and Rogier Kievit, “Raincloud plots,”  (2018).
  • Adams et al. (2006) W. K. Adams, K. K. Perkins, N. S. Podolefsky, M. Dubson, N. D. Finkelstein,  and C. E. Wieman, “New instrument for measuring student beliefs about physics and learning physics: The colorado learning attitudes about science survey,” Phys. Rev. ST Phys. Educ. Res. 2, 010101 (2006).
  • Kalender et al. (2019b) Z. Yasemin Kalender, Emily Marshman, Christian D. Schunn, Timothy J. Nokes-Malach,  and Chandralekha Singh, Diversity Across the Disciplines: Research on People, Policy, Process, and Paradigm, edited by A.J. Murrell, J.L. Petrie-Wyman,  and A. Soudi (Information Age Publishing, 2019).
  • Heller and Hollabaugh (1992) Patricia Heller and Mark Hollabaugh, “Teaching problem solving through cooperative grouping. part 2: Designing problems and structuring groups,” American Journal of Physics 60, 637–644 (1992).
  • Seidel and Tanner (2013) Shannon B. Seidel and Kimberly D. Tanner, “"what if students revolt?"-considering student resistance: Origins, options, and opportunities for investigation,” CBE-Life Sci Educ. 12, 586–595 (2013).
  • Godwin et al. (2016) Allison Godwin, Geoff Potvin, Zahra Hazari,  and Robynne Lock, “Identity, critical agency, and engineering: An affective model for predicting engineering as a career choice,” Journal of Engineering Education 105, 312–340 (2016).
  • Hazari et al. (2010) Zahra Hazari, Gerhard Sonnert, Philip M. Sadler,  and Marie-Claire Shanahan, “Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study,” Journal of Research in Science Teaching 47, 978–1003 (2010).