Science of teaching and learning

J. Cape

Faculty at the University of Manitoba may be engaging in the scholarship of teaching and learning (SoTL) far more than institutional data suggest—they just aren't calling it that. This scoping review examined 502 higher education studies published by UM-affiliated authors over 14 years, identifying 58 that investigated teaching and learning questions characteristic of SoTL without being labeled as such. Of the 131 unique authors represented, 38 were students. Most research appeared in discipline-specific journals rather than SoTL outlets, with healthcare programs particularly well-represented. Strikingly, none of the 58 articles used "SoTL" in their titles, abstracts, or keywords — even though SoTL or teaching-learning funding supported 18% of the studies. These findings suggest that SoTL-type inquiry is already underway across the institution and may be significantly underrecognized. Mapping where this work is happening could help universities better target support programming, more accurately assess the impact of SoTL funding, and connect researchers who share an interest in teaching and learning inquiry.

H. Malyk

Diverse definitions and interpretations of SoTL concepts often lead to confusion and hinder engagement, limiting original research and teaching methods (McKinney, 2002). Terminological inconsistency pushes SoTL to the margins of pedagogy, making it undervalued and vague (Simmons, 2017; Chick, 2013). This study addresses these issues by examining SoTL's core concepts, definitions, and terminology to enable cultural change for improved educational practices within the 4 M Framework of SoTL Impact (Simmons, 2020) and clarify the learning process through SoTL (Tagg, 2010).

L. A. Doan & B. M. Stoesz

Self-reflection is vital for professional development among higher education instructors (Szűcs, 2018). Creative, expressive, and reflective writing can positively shape professional identity (Lengelle et al., 2016). Most research has focused on K-12 and community college teachers, while university educators and educational developers are less studied (Jaramillo Cherrez & Jin, 2020). This study aims to explore whether educational developers in higher education engage in self-reflection and how it affects their growth and identity.

B. M. Stoesz, M. Quesnel, J. Plohman, I. Arowolo, & W. Snow

Training postsecondary educators to implement pedagogies and deal with various teaching challenges more effectively supports the goal of striving for teaching excellence at the University of Manitoba. A novel way to prepare graduate students for success in teaching involves the use of extended/virtual reality (XR/VR). This project aims to determine the effectiveness of such an approach to teacher training. 

L. Scholle-Collen (Queen’s University) & B. M. Stoesz with support from L. Doan

Chapter in press

Many Canadian universities began as religious institutions focused on ministry, but have since shifted to broad academic offerings. Analyzing 264 Senate meeting minutes from Queen’s University and the University of Manitoba highlighted gradual cultural changes toward academic integrity. Traditional qualitative analysis detected subtle improvements, while generative AI proved inadequate for independent data analysis. These findings inform university policy development, emphasizing the importance of adapting to evolving standards of integrity and technology to promote honesty, trust, and transparency.

W. M. Snow & A. E. De Jaeger

Many fields, including higher education, use self-report surveys, which are increasingly conducted online for greater anonymity and lower costs. However, digital surveys raise the risk of fraudulent responses. This study assessed suspected invalid answers in an online survey on adult job-related training in Manitoba and improved an algorithm for detecting fraud. The findings aim to help design stricter data collection methods and advocate for standardized reporting of fraudulent activity to boost trust and data integrity in online research.

B. M. Stoesz, R. Attas, & H. Sanni-Anibire

Within the topic of academic integrity, accommodations for students experiencing learning challenges have largely been ignored in research and practice. Instructors’ lack of knowledge and accompanying skills may result in misconceptions about the intersection between academic integrity and the learning needs of students; in particular, the misconception that accommodations are “situationally sanctioned cheating” that allow students to “gain an unfair advantage” rather than facilitating learning and success must be addressed. This chapter explores questions that students, instructors, and other postsecondary community members may have about accessibility and equity, and how these considerations show respect, fairness, trust, responsibility, honesty, and courage and contribute to building cultures of academic integrity. To read more, click here. 

M. Quesnel & B. M. Stoesz

Contract cheating is a significant concern in higher education, requiring a multifaceted teaching and learning approach to address it. Quizzing students about their writing to promote engagement, confirm authorship and detect cheating has not yet been investigated systematically. Therefore, in this study, our objective was to explore the validity and usability of the quizzing software Auth+ to verify assessment authorship and detect contract cheating. A total of 24 third-year computer science students submitted their assignments to Auth+ and were quizzed, then completed a working memory assessment and a user experience survey. Students reported that Auth+ was easy to use and would discourage cheating but increased anxiety and contributed little to their learning. Quiz scores were correlated with individual differences in working memory capacity but not with other contract cheating indicators. These findings suggest . . . To read more, click here.  

Student ratings of instruction (SRI) are commonly used to evaluate courses and teaching in higher education. Much debate about their validity in evaluating teaching exists, which is due to concerns of bias by factors unrelated to teaching quality (Spooren et al., 2013). Our objective was to identify peer-reviewed original research published in English from January 1, 2012, to March 10, 2021, on potential sources of bias in SRIs. Our systematic review of 63 articles demonstrated strong support for the continued existence of gender bias, favoring male instructors and bias against faculty with minority ethnic and cultural backgrounds. These and other biases must be considered when implementing SRIs and reviewing results. Critical practices for reducing bias when using SRIs include implementing bias awareness training and avoiding use of SRIs as a singular measure of teaching quality when making decisions for teaching development or hiring and promotion.

Stoesz, B., De Jaeger, A., Quesnel, M., Bhojwani, D. & Los, R. (2022). Bias in student ratings of instruction: A systematic review of research from 2012 to 2021. Canadian Journal of Educational Administration and Policy/Revue canadienne en administration et politique de l’éducation, (201), 39–62. https://doi.org/10.7202/1095482ar 

Approximately 2,034,957 Canadians were enrolled in courses at postsecondary institutions in 2015/2016 (Statistics Canada, 2017), and an estimated 610,487 (30%) are completing online courses in any given year (Bates, 2015). This project examined students’ visual perceptions of the learning management systems (LMS) they use and how it influences their learning experiences. Findings contribute to our knowledge of LMS interface design.  

Stoesz, B. M., & Niknam, M. (2023). Student perceptions of the visual design of learning management systems. Canadian Journal of Learning and Technology, 48(3), 1-22. https://cjlt.ca/index.php/cjlt/article/view/28154/20689  

Researchers from four Canadian universities and an industry partner are collaborating to better understand faculty perceptions and needs related to academic integrity in Canadian higher education. The survey designed for this project was informed by previous academic integrity research (e.g. McCabe, 1993), and the findings are expected to lead to the development of specific supports for faculty in Canadian higher education institutions to promote academic integrity.

Eaton, S. E., Stoesz, B. M., Crossman, K., Garwood, K., & McKenzie, A. (2023). Faculty Perspectives of Academic Integrity During COVID-19: A Mixed Methods Study of Four Canadian Universities. Canadian Journal of Higher Education, 52(3), 42–58. https://doi.org/10.47678/cjhe.vi0.189783 

Postsecondary educators and students were among the first affected by COVID-19 pandemic safety protocols and were required to transition quickly from face-to-face to unfamiliar remote teaching and learning environments. To support this transition, support staff at teaching and learning centres (TLCs) also pivoted their support strategies and developed and delivered more online resources and virtual professional development workshops. The purpose of this study was to examine whether the demands for online and remote teaching and learning support was evident in the Google Analytics data of the TLC webpages of a research-intensive Canadian university. To facilitate interpretation of observed trends, we aligned the timing of messages emailed to faculty and new resource launches on the TLC webpages to the analytics data. Results confirmed that TLC webpages are vital sources of information for faculty development , and targeted communications increased faculty engagement with teaching resources and professional development opportunities. As pandemic safety protocols ease and educators and students return to their postsecondary campuses, they will face new challenges. In response, TLCs must continue to monitor faculty engagement and their changing support needs, and they must continue to adjust approaches to offering information and professional development opportunities as necessary. The examination of website analytics data is just one measure that can support evidence-informed decision making for this purpose.

Stoesz, B. M. (2022). Using Google analytics to measure engagement with a teaching and learning centre during COVID-19. Journal on Centers for Teaching and Learning, 14, 39-57. https://openjournal.lib.miamioh.edu/index.php/jctl/article/view/245 

The topic of ethics has received much attention within the SoTL community; however, limited studies have examined both REB reviewers’ and SoTL researchers’ perspectives on partnerships when preparing or reviewing SoTL research. In this chapter, we describe the findings from our study that aimed to address this gap by gathering researchers’ and REB reviewers’ experiences with and perceptions of SoTL research and situate findings within the micro-meso-macro-mega (4M) framework.

De Jaeger, A. E., Stoesz, B. M., Doan, L. A. (2022). Tensions and Partnerships: Understanding Research Ethics in the Scholarship of Teaching and Learning (SoTL). In: Fedoruk, L.M. (eds) Ethics and the Scholarship of Teaching and Learning. Ethics and Integrity in Educational Contexts, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-031-11810-4_4 

This study surveyed 60 international students at Canadian postsecondary institutions about their knowledge and emotions related to academic integrity. Most participants understood academic integrity policies and found expectations similar between Canada and their home countries, but confusion existed over duplicate submission/self-plagiarism and contract cheating. Confidence was common, yet some felt fear or anxiety, often linked to less understanding of the rules. The results highlight a need for clearer education on specific integrity issues and support for international students, as well as improvements to policies and resources.

Sanni-Anibire, H., Stoesz, B. M., Gervais, L., & Vogt, L. (2021). International students’ knowledge and emotions related to academic integrity at Canadian postsecondary institutions. International Journal of Educational Integrity, 17, 21 (2021). https://doi.org/10.1007/s40979-021-00088-4 

Readiness for teaching online can be defined as having the competencies required to deliver a high-quality online course, where readiness can be equated with the qualities exemplified by the ideal online instructor. This project seeks to determine the characteristics that exemplify effective online instructors and create a first-of-its-kind online teaching readiness assessment with psychometric properties. The assessment will provide instructors with immediate feedback as to the areas they excel and can improve upon, assist in identifying faculty training/development opportunities, determine strengths and areas in need of enhancement in instructors’ online teaching skillsets, and further understand the barriers and attitudes towards online teaching.

Los, R., De Jaeger, A. & Stoesz, B. M. (2021). Development of the Online and Blended Teaching Readiness Assessment (OBTRA). Frontiers in Education, 6(673594). https://doi.org/10.3389/feduc.2021.673594 

Delivering courses online can be effective but learning via learning management systems (LMS) may be attenuated by poor visual design. This project aimed to determine if existing models of complexity for webpages can be extended to LMS and explore how individual differences influence ratings of complexity. Findings inform the development of online learning environments.  

Stoesz, B. M., Niknam, M., & Sutton, J. (2020). Defining the visual complexity of learning management systems using image metrics and subjective ratings. Canadian Journal of Learning and Technology, 46(2), 1-21. https://www.cjlt.ca/index.php/cjlt/article/view/27899/20497