Past Research Projects

The Impact of Course Format on Student Perceptions of the Classroom Learning Environment and Teamwork.

Research Team: Shayna. A. Minosky, Michael Wiechers, & Leonardo Landaverde-Umana.

Traditionally, education has been largely delivered in an in-person format; however, an increasing number of courses are being delivered entirely online or with a blend of online and in-person components. These formats differ along various dimensions, such as the quantity and quality of interpersonal interactions and connections, which will likely lead to different student experiences. Using a sample of 200 undergraduate student responses from an online survey, we compared five different course formats (in-person, synchronous online, asynchronous online, blended with alternating weeks, and blended exam only) on students’ perceptions of various elements of their learning environment, including teaching presence, cognitive presence, social presence, sense of community, and teamwork. A between groups ANOVA demonstrated significant differences for seven of the eight variables examined. In each case, the in-person format was rated the most positively and the blended exam only format tended to receive the poorest ratings. Overall, our results suggest that live interaction among students, and between students and instructors, whether it is from an in-person format or a blended alternating format, appears to be linked to more positive perceptions of the social learning environment.  

This paper has been accepted for publication in Active Learning in Higher Education.

Attitudes Towards Working in Student Teams: A Systematic Review of Validity Evidence.

Research Team: Shayna A. Minosky & Leonardo Landaverde-Umana.

Working in small groups of peers is a common activity in classroom settings; however, students are mixed on whether they perceive teamwork positively or negatively. Being able to assess students’ attitudes towards teamwork can help identify facilitators and barriers towards working with peers and can contribute to fostering a more positive learning environment. In this paper, we present a systematic review that identifies measures that are available to assess student attitudes toward teamwork and the validity evidence for these measures.  The scope of this review consisted of studies that provided quantitative self-report data relevant to adolescent or adult students’ attitudes towards working in a team of peers in a classroom setting. We excluded studies that focused on interprofessional medical teams (clinicians and students), workplace teams, and sports teams. We also excluded studies that focused on the assessment of team performance rather than general attitudes about working in a team. We conducted our search through to May 27, 2021. Of the 15,346 initial citations, 10 studies, representing eight unique measures, were included in this review. An overall validity percentage score was calculated for each of the studies, with the average score equal to 46%. The Feelings Toward Group Work Questionnaire Greek version (Goudas et al., 2009) achieved the highest score at 88%. Taken together, these findings provide guidance for those looking for a measure of teamwork attitudes.  

This paper has been submitted for publication and is currently under review.

Narratives of Empowerment through Attaining a Black Belt in Mixed Martial Arts .

Research Team: Shayna A. Minosky, Amanda R. Dumoulin.

In this qualitative study, we explored the experiences of 10 adults who trained in mixed martial arts (MMA) to understand the meaning they ascribed to attaining the black belt and their martial arts journal overall. Using a conventional content analysis, four themes were derived from the data: importance of the black belt, benefits of training in MMA, dealing with injuries, and being part of the MMA community. Training in MMA was very positive, with both individual benefits (improved physical and mental health, skill development, and personal growth) and interpersonal benefits (relationship development and sense of community) being reported. Self-determination theory (Ryan & Deci, 2000) and goal setting theory (Locke & Latham, 2002) are used to discuss participants’ motivation in their pursuit of the black belt and continued training. 

This paper has been accepted for publication in Martial Arts Studies.

Does Level of Teaching and Teaching Experience Indicate a Teacher’s Teaching Perspective? A Descriptive Analysis Using the Teaching Perspective Inventory.

Research team: Amanda R. Dumoulin, Brandon J. Justus, Shayna A. Rusticus, and Daniel D. Pratt.

The Teaching Perspectives Inventory (TPI) assesses teacher’s beliefs, intentions, and actions as they relate to the five teaching perspectives of transmission, apprenticeship, nurturing, developmental, and social reform. In this study we explored whether there was a relationship between teachers’ dominant teaching perspectives and both level of teaching (elementary to graduate) and teaching experience (novice to very experienced) in a sample of 54,176 respondents from Canada and the United States. A chi-square test of independence revealed a statistically significant and medium-sized relationship between teaching perspective and level of teaching, with a nurturing perspective being more common among elementary and secondary teachers and an apprenticeship perspective being more common among undergraduate and graduate teachers. A second chi-square test of independent revealed a statistically significant but small relationship between teaching perspective and teaching experience. All experience levels had the same rank ordering of perspectives: nurturing, apprenticeship, developmental, transmission, then social reform. Understanding patterns and diversity in teaching perspectives can help institutions foster faculty development to provide a more personalized approach to supporting their educators, allowing for the benefits to be felt by the students. 

The paper has been submitted for publication and is currently under review.

Dumoulin, A. R., Justus, B. J., Rusticus, S. A., & Pratt, D. D. (2021, June). Teaching perspectives are related to level of education and teaching experience [Poster]. 82nd Canadian Psychological Association Annual National Convention, Virtual.

View the poster here (Download)

Validation of the Teamwork Expectations and Attitudes Measure 

Research team: Brittney Stobbe, Jonathan B. K. Lau, Brandon J. Justus, and Shayna A. Rusticus 


Teamwork is essential in any group projects and being able to identify attitudes and expectations could support students, teachers, and instructors with learning. To help identify attitudes and expectations in groups, our study developed a scaled called the Teamwork Expectations and Attitudes Measure (TEAM). First, two pilot studies were conducted to develop and refine the items. Starting with 75-items, the TEAM scale was completed into a 14-item unidimensional scale. Second, the pilot studies were followed up with a validation study which confirmed the unidimensional structure of the scale and provided evidence of convergent, discriminant, and criterion validity. In the end, the purpose of the developing the TEAM scale was to create something that could assist instructors and students on the perception of teamwork in class, which in turn could shred knowledge about the class’ overall view about group work. Therefore, with the significant findings from this study, the TEAM scale can be useful tool in undergraduate environments that need extra support in examining teamwork within classes.  

Justus, B. J., Rusticus, S. A., Stobbe, B. L. P., Lau, J. B. K. (2021, June). Using the Teamwork Expectations and Attitudes Measure (TEAM) to assess student perceptions of working in teams [Poster] 2021 Annual Meeting of the National Council on Measurement in Education, Virtual.

Employable-Skills Self-Efficacy Survey: A Validation Study

Amanda R. Dumoulin and Shayna A. Rusticus


The Employable Skills Self-Efficacy Survey (ESSES; Ciarocco & Strohmetz, 2018) is a scale that measures the self-efficacy of undergraduate psychology students. This measure is intended to assess an important collection of constructs and has many potential benefits, including assisting institutions in ensuring their students accomplish the goals laid out by the American Psychological Association (2013) for undergraduate psychology students. The purpose of this study was to provide additional validity evidence for the ESSES by looking at its internal structure, reliability, and convergent and discriminant validity. As identified through confirmatory factor analysis, the ESSES does not have an eleven-factor structure, but ten of the eleven subscales were found to be unidimensional. However, only three of the unidimensional subscales had acceptable reliability. There was evidence of convergent validity, but limited evidence of discriminant validity. Revisions are necessary before this scale should be used to measure the employable skills self-efficacy of undergraduate psychology students.  

The paper has been accepted for publication in the Scholarship of Teaching and Learning in Psychology.

Dumoulin, A. R., & Rusticus, S. A. (2021, June). Employable Skills Self-Efficacy Survey: A validation study [Poster]. 2021 Annual Meeting of the National Council on Measurement in Education, Virtual.

Validating a Modified Version of the Self-Directed Learning Readiness Scale (MSDLR) for use Among Undergraduate Students 

Amanda R. Dumoulin, Brandon J. Justus, Jonathan B. K. Lau, and Shayna A. Rusticus 


Self-directed learning readiness (SDLR) refers to the degree to which a learner is ready to be accountable for their own learning and learning needs and is a skill that students can develop. Understanding student levels of SDLR can help optimize the learning environment for more effective teaching and learning strategies. The purpose of this study was to provide additional validity evidence for a modified version of the SDLR scale. Evidence of internal structure and relations with other variables was examined in a sample of 203 undergraduate students. A confirmatory factor analysis did not support the three-factor structure of the modified SLDR scale; however, a follow-up exploratory factor analysis suggested that there were three factors, with some items not loading onto their intended factors. Evidence was provided for convergent validity and mixed evidence was found for discriminant validity. Overall, these results suggest that some modifications may be needed for this scale, but there is potential for this measure to be suitable for assessing readiness for self-directed learning.  

Dumoulin, A. R., Justus, B. J., Lau, J. B. K., & Rusticus, S. A. (2021). Validating a modified version of the Self-Directed Learning Readiness scale (MSDLR) for use among undergraduate students. Kwantlen Psychology Student Journal, 3.

Dumoulin, A. R., Justus, B. J., Lau, J. B. K., & Rusticus, S. A. (2021, June). Validating the self-directed learning readiness scale for use with undergraduate students [Poster]. 82nd Canadian Psychological Association Annual National Convention, Virtual.

Does self-directed learning readiness predict undergraduate students’ instructional preferences?  

Brandon J. Justus, Shayna A. Rusticus, and Brittney Stobbe  


 Self-directed learning is a process by which students take the lead, with or without the help of others, in determining their learning needs and managing their learning strategies and outcomes. Relatedly, self-directed learning readiness (SDLR) looks at the attitudes, abilities, and personality characteristics necessary for self-directed learning. In study one, we shortened, and slightly modified, the SDLR scale (Fisher et al., 2001) for use among undergraduate university students and examined its factor structure and reliability.  In a sample of 194 students, the three-factor structure of this scale (self-management, desire to learn, and self-control) was confirmed with acceptable reliability. In study two, we examined whether SDLR subscales predicted a preference for a teacher-directed or student-directed class format in a sample of 256 undergraduate students. We conducted a series of four multiple linear regressions to examine whether the three dimensions of SDLR were predictive of four classroom preference styles (knowledge construction, teacher direction, cooperative learning, and passive learning). Three of these analyses were statistically significant with small to medium effect sizes. These findings have the potential to identify factors that may be linked to greater student engagement, more positive learning environments, and greater success in the learning process.  

Justus, B. J., Rusticus, S. A., & Stobbe, B. (2020, July). Does self-directed learning readiness predict undergraduate students’ instructional preferences? [Poster]. 81st Canadian Psychological Association Annual National Convention, Montréal, Quebec, Canada.

This paper has been accepted for publication in The Canadian Journal for the Scholarship of Teaching and Learning

Comparing Student and Teacher Formed Teams on Group Dynamics, Satisfaction and Performance 

Shayna A. Rusticus and Brandon J. Justus  


We compared student and teacher-formed teams on aspects of group dynamics, satisfaction, and performance. Two sections of an introductory psychology research methods course were randomly assigned to either student-formed (n = 28) or teacher-formed (n = 33) teams. We conducted t-tests on 10 measures related to group dynamics, satisfaction, and success. Academic performance and group work contribution were the only measures found to be statistically different, with the student-formed teams scoring higher than the teacher-formed teams. Follow up individual interviews or focus groups conducted with 13 of these students suggested a slight preference for the teacher-formed method because it was transparent and eliminated the stress of having to choose one’s, team members. We further recommend this method because of its simplicity and closer approximation to real-world scenarios. Several factors identified as being important for effective team functioning, regardless of group formation methods are also discussed.   

Rusticus, S. A. & Justus, B. (2019). Comparing student- and teacher-formed teams on group dynamics, satisfaction, and performance.  Small-Group Research, 50(4), 443–457.  

Establishing Equivalence Thresholds Using a Distribution-Based Approach  

Shayna A. Rusticus, Kyla Javier, and Kevin W. Eva  


Establishing group equivalence, as opposed to group differences, is a common goal in many educational/research contexts. Tests of equivalence are used to address such goals; however, a key methodological consideration is how to operationalize equivalence. This study sought to verify if a distribution-based approach, based on effect size, can establish a generalizable criterion for identifying equivalence. A sample of 331 students was presented with a series of numerical statements or bar graphs representing three measures: (1) overall academic achievement, (2) an individual exam score, and (3) a course evaluation survey. Descriptive statistics and a mixed ANOVA examined the effects on equivalence ratings of (a) the difference between means, (b) spacing of the differences (narrow/wide), and (c) presentation format (bar graph/numerical). Across the measures and conditions, the equivalence threshold (i.e., the point at which 50% of participants rated the mean difference as non-equivalent) ranged from an effect size of d = 0.37 to d = 1.15, suggesting that a single effect size criterion for establishing the equivalence threshold may not be achievable. Guidelines are provided for setting an appropriate equivalence threshold.    

What are the Key Elements of a Positive Learning Environment?  
Perspectives from Students and Faculty 

Shayna A. Rusticus, Tina Charmchi, and Andrea Mah  


The learning environment comprises the psychological, social, cultural, and physical setting in which learning occurs and has an influence on student motivation and success. The purpose of the present study was to qualitatively explore, from the perspectives of both students and faculty, the key elements of the learning environment that supported and hindered student learning. We recruited a total of 22 students and 9 faculty to participate in either a focus group or individual interview session on their perceptions of the learning environment at their university. We analyzed the data using directed content analysis and organized the themes around the three key dimensions of personal development, relationships, and institutional culture. Within each of these dimensions, we identified subthemes that facilitated or hindered student learning, and faculty work, experiences. We also identified and discussed similarities in subthemes identified by students and faculty. 

This study has been been accepted by Learning Environments Research.