Multivariate Generalization Analysis for Students’ Evaluation on Teaching Level of Teachers in Medical Colleges

Nan CHENG; Haoyuan DENG; Jianzhong YIN; Meng WU; Yuan LUO; Qiong MENG

doi:10.12259/j.issn.2095-610X.S20220702

Volume 43 Issue 7

Jul. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Kunming Medical University > 2022 > 43(7): 156-161

Nan CHENG, Haoyuan DENG, Jianzhong YIN, Meng WU, Yuan LUO, Qiong MENG. Multivariate Generalization Analysis for Students’ Evaluation on Teaching Level of Teachers in Medical Colleges[J]. Journal of Kunming Medical University, 2022, 43(7): 156-161. doi: 10.12259/j.issn.2095-610X.S20220702

Citation:

Nan CHENG, Haoyuan DENG, Jianzhong YIN, Meng WU, Yuan LUO, Qiong MENG. Multivariate Generalization Analysis for Students’ Evaluation on Teaching Level of Teachers in Medical Colleges[J]. Journal of Kunming Medical University, 2022, 43(7): 156-161. doi: 10.12259/j.issn.2095-610X.S20220702

Citation:

Nan CHENG, Haoyuan DENG, Jianzhong YIN, Meng WU, Yuan LUO, Qiong MENG. Multivariate Generalization Analysis for Students’ Evaluation on Teaching Level of Teachers in Medical Colleges[J]. Journal of Kunming Medical University, 2022, 43(7): 156-161. doi: 10.12259/j.issn.2095-610X.S20220702

PDF( 623 KB)

Multivariate Generalization Analysis for Students’ Evaluation on Teaching Level of Teachers in Medical Colleges

doi: 10.12259/j.issn.2095-610X.S20220702

1.
Institute of Public Health，Kunming Medical University，Kunming Yunnan 650500
2.
Baoshan College of Traditional Chinese Medicine，Baoshan Yunnan 678000
3.
School of Public Health，Nanjing Medical University Nanjing，Jiangsu 210000
4.
School of Public Health，Guizhou Medical University，Guiyang Guizhou 550025，China

Received Date: 2022-04-11
Publish Date: 2022-07-25

Abstract

Abstract

Objective To evaluate the reliability of the Student Evaluation Scale for the Teaching Level of Teachers in Medical Universities （SESTLTMU） and determine the appropriate number of students in the teaching evaluation based on Multivariate Generalizability Theory （MGT） . Methods The data of 422 students from a medical university who were surveyed by this scale were collected and analyzed by using mGENOVA, a special software of multivariate generalizability theory. The variance components of various error sources were estimated in Generalizability Study （G-study）, and then several Decision Studies （D-studies） with varying numbers of items and numbers of students were analyzed to obtain reliability coefficients including generalizability coefficient （G） and the indexes of dependability （Ф） in order to evaluate the reliability of the scale. Results In the G-study, the most prominent variation in every domain was introduced by student nested in teacher effect. In the D-study, at the level of domain, the G coefficients and the Ф coefficients for three of the five domains were approximately equal to or greater than 0.80, except for the teaching organization domain and teaching method domain （> 0.70 but < 0.80）. For the overall scale, the compositeG and composite Ф coefficients were larger than 0.85. Under the premise that the Ф is 0.80 or above, the minimum number of students selected from each class should be 25. Under the premise that the G is 0.80 or above, the minimum number of students selected from each class should be 28. Conclusions The scale has good reliability as a whole based on the results of MGT. If this scale needs to be revised in the future, it can be considered to adjust the content in the teaching organization domain and the teaching method domain. It is the most appropriate to select 28 students from each class for investigation in the practice of teaching evaluation by university students.
- Multivariate generalizability theory,
- Teaching evaluation by students,
- Teaching level,
- Scale

FullText(HTML)

References(14)

References

[1]	本书编委会. 全国普通高校本科教育教学质量报告（2020年度）[M]. 北京: 高等教育出版社, 2021: 1-264.
[2]	陈银燕. 高校发展性评价体系构建:教师和机构的双维度评价[J]. 内蒙古师范大学学报(教育科学版),2016,29(03):76-78.
[3]	Debroy A,Ingole A,Mudey A. Teachers’ perceptions on student evaluation of teaching as a tool for faculty development and quality assurance in medical education[J]. Educ Health Promot,2019,8:218-225.
[4]	Constantinou C,Wijnen-Meijer M. Student evaluations of teaching and the development of a comprehensive measure of teaching effectiveness for medical schools[J]. BMC Med Educ,2022,22(1):113. doi: 10.1186/s12909-022-03148-6
[5]	黎光明,甄锋泉,王幸君,等. 多元概化理论在教育测量与评价中的多维化分析[J]. 教育测量与评价(理论版),2016,180(2):13-17.
[6]	孟琼,张美霞,陈莹,等. 医科院校教师教学水平学生评价量表的信度效度分析[J]. 卫生软科学,2016,30(7):46-48+53. doi: 10.3969/j.issn.1003-2800.2016.07.012
[7]	张志明, 张雷. 测评的概化理论及其应用[M]. 北京: 教育科学出版社, 2003: 52-53.
[8]	Nicaise V,Bois J E,Fairclough S J,et al. Girls’ and boys’ perceptions of physical education teachers' feedback:effects on performance and psychological responses[J]. Sports Sci,2007,25(8):915-926. doi: 10.1080/02640410600898095
[9]	Wolbring T,Riordan P. How beauty works. Theoretical mechanisms and two empirical applications on students’ evaluation of teaching[J]. Soc Sci Res,2016,57:253-272. doi: 10.1016/j.ssresearch.2015.12.009
[10]	Doubleday A F,Lee L M. Dissecting the voice:Health professions students’ perceptions of instructor age and gender in an online environment and the impact on evaluations for faculty[J]. Anat Sci Educ,2016,9(6):537-544. doi: 10.1002/ase.1609
[11]	Briesch A M,Swaminathan H,Welsh M,et al. Generalizability theory:A practical guide to study design,implementation,and interpretation[J]. Sch Psychol,2014,52(1):13-35. doi: 10.1016/j.jsp.2013.11.008
[12]	Vispoel W P,Morris C A,Kilinc M. Applications of generalizability theory and their relations to classical test theory and structural equation modeling[J]. Psychol Methods,2018,23(1):1-26. doi: 10.1037/met0000107
[13]	Keller L A,Clauser B E,Swanson D B. Using multivariate generalizability theory to assess the effect of content stratification on the reliability of a performance assessment[J]. Advances in Health Sciences Education,2010,15(5):717-733. doi: 10.1007/s10459-010-9233-8
[14]	Ibrahim A M. Using generalizability theory to estimate the relative effect of class size and number of items on the dependability of student ratings of instruction[J]. Psychol Rep,2011,109(1):252-258. doi: 10.2466/03.07.11.PR0.109.4.252-258

Relative Articles(20)

Relative Articles

[1]	Jihua CUI, Shujin LI, Yu SONG, Yanfei YANG, Yanhui SUN, Jingjing SUN, Yu LING. The Predictive Value of Temperament in Children with Attention Deficit Hyperactivity Disorder. Journal of Kunming Medical University, 2023, 44(5): 125-131. doi: 10.12259/j.issn.2095-610X.S20230518
[2]	Jiping ZHOU, Junling ZHANG, Qing LIU, Xingmei LI, Peiyou FAN, Heling JIANG. The Application of “Modular Teaching Method” in the Nursing Teaching of Nephrology Department. Journal of Kunming Medical University, 2022, 43(5): 168-172. doi: 10.12259/j.issn.2095-610X.S20220523
[3]	Jiang ZHANG, Bingkun YANG, Wenbo PING, Xijuan ZHAO, Yan SU, Jiang WU. Application of Rain Classroom Combined Scenario Simulation Based on Target Teaching Theory in Teaching Intern Nursing Students in Radiotherapy Department. Journal of Kunming Medical University, 2022, 43(11): 172-177. doi: 10.12259/j.issn.2095-610X.S20221123
[4]	Chuntao WU, Ke LIN. Construction and Application of the Health Literacy Assessment Scale for Day Surgery Patients and Its Influence on Early Postoperative Recovery Quality. Journal of Kunming Medical University, 2022, 43(5): 179-184. doi: 10.12259/j.issn.2095-610X.S20220515
[5]	Yanqin RUAN, Maojuan LI, Limei HE, Ying SONG, Qiaoyun HUANG, Ying CHEN, Chang LIU. Screening of Special Abnormal Module Items in PRO Scale for Patients with Inflammatory Bowel Disease.. Journal of Kunming Medical University, 2022, 43(3): 37-43. doi: 10.12259/j.issn.2095-610X.S20220310
[6]	Shun-ding TANG, Ying SONG, Wei CHANG, Lin-xiong WU, Xiao-qing ZHANG. Application of Comprehensive Index Method in Teaching Performance Evaluation of University Teachers. Journal of Kunming Medical University, 2021, 42(4): 161-167. doi: 10.12259/j.issn.2095-610X.S20210431
[7]	Xiao-xiao SONG, Shan BAI, Yue-na YANG, Bing-bing YANG, Sai-xian LI, Ying-hui LIN, Zhe MA, Jia ZENG, Fan-wei QU. Reliability and Validity of A Questionnaire for International Students in China with Educational Service Satisfaction in English Version-Taking Kunming Medical University as A Case. Journal of Kunming Medical University, 2021, 42(3): 161-165. doi: 10.12259/j.issn.2095-610X.S20210337
[8]	Sun Chun Yi , Qing Qing , Li Dai Zhu , Xu Chang Jun , Huang Ya Juan , Liu Yang . . Journal of Kunming Medical University, 2020, 41(02): 153-157.
[9]	Tan Rui Jing, Liao Rui , He Li Fen , Wang Jun Ying , Ding Li , Liu Min , Li Hong Mei . An Application of Modularization and Project-driven Teaching in the Improvement of Students' Information Literacy. Journal of Kunming Medical University, 2018, 39(04): 141-144.
[10]	Yang Lin Lin , Yang Hong Ying , Zhang Lei , Zhao Ling Feng , Wang Wei , Tan Shu Fen . The Research on the Application of Bloom's Taxonomy of Educational Objectives in the Teaching of the Theory Course of Obstetrics and Gynecology. Journal of Kunming Medical University, 2018, 39(04): 145-148.
[11]	Lei Wen , Wu Wen Juan , Li Zhen Kun , Li Jing , Yang Mei Juan , Dong Zhao Xing . Participatory Teaching Class for MBBS Students. Journal of Kunming Medical University, 2018, 39(05): 138-140.
[12]	Ding Zhe Yu , Cun Ying Li , Cha Yong , Dai You Guo , Zhang Wen , Zhang Zhong Hong , Du Feng , Wan Chong Hua . Revision of the Quality of Life Instruments for Cancer Patients-Stomach Cancer (V 2.0). Journal of Kunming Medical University, 2017, 38(07): 50-54.
[13]	Zhang Xi Hong . The Study of Oral Health-Related Quality of Life （OHR-QOL） among 471 Elderly Patients with Chronic Periodontitis in Kunming City. Journal of Kunming Medical University,
[14]	Luo Sai Mei . . Journal of Kunming Medical University,
[15]	Zhang Xiao Qing . . Journal of Kunming Medical University,
[16]	Zhang Xiao Qing . . Journal of Kunming Medical University,
[17]	Zhang Dan Xia . . Journal of Kunming Medical University,
[18]	Meng Qiong . Development and Evaluation on Chinese Version of Measurement Instrument of Quality of Life for Patients with Stomach Cancer: EORTC QLQ-STO22. Journal of Kunming Medical University,
[19]	Tang Xin Ming . . Journal of Kunming Medical University,
[20]	. Application of Generalizability Theory in Evaluation of Academic Record in Final Examination of Medical Statistics. Journal of Kunming Medical University,