-
Validity evidence and reliability of a Simulated Patient Feedback Instrument
Background:
In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF).
Methods:
Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings.
Results:
All but two items of the mQSF received importance ratings of >2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter.
Conclusions:
The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.
-
Do individual learning styles influence the choice to use a web-based ECG learning programme in a blended learning setting?
Background:
The compressed curriculum in modern knowledge-intensive medicine demands useful tools to achieve approved learning aims in a limited space of time. Web-based learning can be used in different ways to enhance learning. Little is however known regarding its optimal utilisation. Our aim was to investigate if the individual learning styles of medical students influence the choice to use a web-based ECG learning programme in a blended learning setting.
Methods:
The programme, with three types of modules (learning content, self-assessment questions and interactive ECG interpretation training), was offered on a voluntary basis during a face to face ECG learning course for undergraduate medical students. The Index of Learning Styles (ILS) and a general questionnaire including questions about computer and Internet usage, preferred future speciality and prior experience of E-learning were used to explore different factors related to the choice of using the programme or not.
Results:
93 (76 %) out of 123 students answered the ILS instrument and 91 the general questionnaire. 55 students (59%) were defined as users of the web-based ECG-interpretation programme. Cronbach's alpha was analysed with coefficients above 0.7 in all of the four dimensions of ILS. There were no significant differences with regard to learning styles, as assessed by ILS, between the user and non-user groups ; Active/Reflective; Visual/Verbal; Sensing/Intuitive; and Sequential/Global (p=0.56-0.96). Neither did gender, prior experience of E-learning or preference for future speciality differ between groups.
Conclusion:
Among medical students, neither learning styles according to ILS, nor a number of other characteristics seem to influence the choice to use a web-based ECG programme. This finding was consistent also when the usage of the different modules in the programme were considered. Thus, the findings suggest that web-based learning may attract a broad variety of medical students.
-
Can achievement goal theory provide a useful motivational perspective for explaining psychosocial attributes of medical students?
Background:
Psychosocial competence and frustration tolerance are important characteristics of skilled medical professionals. In the present study we explored the usefulness of applying a comprehensive motivational theory (Goal orientations), for this purpose. According to goal orientation theory, learning motivation is defined as the general goals students pursue during learning (either mastery goals - gaining new knowledge; or performance goals - gaining a positive evaluation of competence or avoiding negative evaluation). Perceived psychosocial abilities are a desirable outcome, and low frustration tolerance (LFT), is a negative feature of student behavior. The hypothesis was that the mastery goal would be positively associated with psychosocial abilities while performance goals would be positively associated with LFT.
Methods:
143 first-year medical students completed at the end of an annual doctor-patient communication course a structured questionnaire that included measures of learning goal orientations (assessed by Pattern of Adaptive Learning Scale - PALS), psychosocial abilities (assessed by Psychological Medicine Inventory- student version -PMI-S) and Low Frustration Tolerance (LFT).
Results:
All study variables were found reliable (Cronbach's alpha ranged from .66 to .90) and normally distributed. Hierarchical multiple regression analysis revealed significant associations supporting the hypotheses. The mastery goal orientation was positively associated with perceived psychosocial abilities (PMI-S) (beta=.16, p<.05) and negatively associated with low frustration tolerance (beta=-.22, p<.05) while performance goal orientation was significantly associated with low frustration tolerance (beta=.36, p<.001).
Conclusions:
The results suggest that the goal orientations theory may be a useful theoretical framework for understanding and facilitating learning motivation among medical students. Limitations and suggestions for practice within medical education context are discussed.
-
Comparing gender awareness in Dutch and Swedish first-year medical students - results from a questionaire
Background:
To ascertain good and appropriate healthcare for both women and men implementation of gender perspectives in medical education is needed. For a successful implementation, knowledge about students' attitudes and beliefs about men, women, and gender is crucial. The aim of this study was to compare attitudes to gender and gender stereotyping among Dutch and Swedish male and female medical students.
Methods:
In this cross-sectional study, we measured the attitudes and assumptions about gender among 1096 first year medical students (616 Dutch and 480 Swedish) with the validated Nijmegen Gender Awareness in Medicine Scale (N-GAMS). The response rate was 94% in the Netherlands and 93% in Sweden. Univariate analysis of variance (ANOVA) was used to compare the scores between Dutch and Swedish male and female students. Linear regressions were used to analyze the importance of the background variables.
Results:
There were significant differences in attitudes to gender between Dutch and Swedish students. The Swedish students expressed less stereotypical thinking about patients and doctors and the Dutch were more sensitive to gender differences. The students' sex mattered for gender stereotyping, with male students in both countries agreeing more with stereotypical statements. Students' age, father's birth country and mother's education level had some impact on the outcome.
Conclusions:
There are differences between cultures as well as between men and women in gender awareness that need to be considered when implementing gender in medical education.This study suggests that to arouse the students' interest in gender issues and make them aware of the significance of gender in medical work, the examples used in discussions need to be relevant and challenging in the context of the specific country. Due to different levels of knowledge and different attitudes within the student population it is important to create a climate for dialogue where students feel permitted to disclose their ideas and attitudes in order to become aware of what these are as well as their possible consequences on interaction and decision-making in medical work.
-
A psychometric appraisal of the DREEM
Background:
The quality of the Educational environment is a key determinant of a student centred curriculum. Evaluation of the educational environment is an important component of programme appraisal. In order to conduct such evaluation use of a comprehensive, valid and reliable instrument is essential. One of most widely used contemporary tools for evaluation of the learning environment is the Dundee Ready Education Environment Measure (DREEM). Apart from the initial psychometric evaluation of the DREEM, few published studies report its psychometric properties in detail. The aim of this study was to examine the psychometric quality of the DREEM measure in the context of medical education in Ireland and to explore the construct validity of the device.
Methods:
239 final year medical students were asked to complete the DREEM inventory. Anonymised responses were entered into a database. Data analysis was performed using PASW 18 and confirmatory factor analysis performed.
Results:
Whilst the total DREEM score had an acceptable level of internal consistency (alpha 0.89), subscale analysis shows that two subscales had sub-optimal internal consistency. Multiple group confirmatory factor analysis (using Fleming's indices) shows an overall fit of 0.76, representing a weak but acceptable level of fit. 17 of the 50 items manifest fit indices less than 0.70. We sought the best fitting oblique solution to the 5-subscale structure, which showed large correlations, suggesting that the independence of the separate scales is open to question.
Conclusions:
There has perhaps been an inadequate focus on establishing and maintaining the psychometric credentials of the DREEM. The present study highlights two concerns. Firstly, the internal consistency of the 5 scales is quite variable and, in our sample, appears rather low. Secondly, the construct validity is not well supported. We suggest that users of the DREEM will provide basic psychometric appraisal of the device in future published reports.
-
Mobile Medical Education (MoMEd) - how mobile information resources contribute to learning for undergraduate clinical students: a mixed methods study.
Background:
Mobile technology is increasingly being used by clinicians to access up-to-date information for patient care. These offer learning opportunities in the clinical setting for medical students but the underlying pedagogic theories are not clear. A conceptual framework is needed to understand these further. Our initial questions were how the medical students used the technology, how it enabled them to learn and what theoretical underpinning supported the learning.
Methods:
387 medical students were provided with a personal digital assistant (PDA) loaded with medical resources for the duration of their clinical studies. Outcomes were assessed by a mixed-methods triangulation approach using qualitative and quantitative analysis of surveys, focus groups and usage tracking data.
Results:
Learning occurred in context with timely access to key facts and through consolidation of knowledge via repetition. The PDA was an important addition to the learning ecology rather than a replacement. Contextual factors impacted on use both positively and negatively. Barriers included concerns of interrupting the clinical interaction and of negative responses from teachers and patients. Students preferred a future involving smartphone platforms.
Conclusions:
This is the first study to describe the learning ecology and pedagogic basis behind the use of mobile learning technologies in a large cohort of undergraduate medical students in the clinical environment. We have developed a model for mobile learning in the clinical setting that shows how different theories contribute to its use taking into account positive and negative contextual factors.The lessons from this study are transferable internationally, to other health care professions and to the development of similar initiatives with newer technology such as smartphones or tablet computers.
-
Factors confounding the assessment of reflection: a critical review
Background:
Reflection on experience is an increasingly critical part of professional development and lifelong learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection.DiscussionWe critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors affecting the assessment of reflection.SummaryTo address the problem of inconsistency, we identified processes that were common to a number of widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment instruments. We arrived at the conclusion that, until further progress has been made in defining standards, assessment must depend on developing and communicating local consensus between stakeholders (students, practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests. Major factors that complicate assessment are the subjective nature of reflection's content and the dependency on descriptions by persons being assessed about their reflection process, without any objective means of verification. To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the subjective content of reflection and where possible to consider objective information about the triggering situation to verify described reflections. Finally, internal and external
contextual factors such as motivation, instruction, character of assessment (formative or summative) and the ability of individual learning environments to stimulate reflection should be considered.
-
Burnout syndrome among dental students: a short version of the "Burnout Clinical Subtype Questionnaire" adapted for students(BCSQ-12-SS)
Background:
Burnout has been traditionally defined in relation to the dimensions of "exhaustion", "cynicism", and "inefficiency". More recently, the Burnout Clinical Subtype Questionnaire (BCSQ-12) further established three different subtypes of burnout: the "frenetic" subtype (related to "overload"), the "under-challenged" subtype (related to "lack of development"), and the "worn-out" subtype (related to "neglect"). However, to date, these definitions have not been applied to students. The aims of this research were (1) to adapt a Spanish version of the BCSQ-12 for use with students, (2) to test its factorial validity, internal consistency, convergent and discriminant validity, and (3) to assess potential socio-demographic and occupational risk factors associated with the development of the subtypes.MethodWe used a cross-sectional design on a sample of dental students (n = 314) from Santiago and Huesca universities (Spain). Participants completed the Burnout Clinical Subtype Questionnaire Student Survey (BCSQ-12-SS), the Maslach Burnout Inventory Student Survey (MBI-SS), and a series of socio-demographic and occupational questions formulated for the specific purpose of this study. Data were subjected to exploratory factor analysis (EFA) using the principal component method with varimax orthogonal rotation. To assess the relations with the criterion, we calculated the Pearson correlation coefficient (r), multiple correlation coefficient (Ry.123), and the coefficient of determination (R2y.123). To assess the association between the subtypes and the socio-demographic variables, we examined the adjusted odds ratio (OR) obtained from multivariate logistic regression models.
Results:
Factorial analyses supported the theoretical proposition of the BCSQ-12-SS, with α-values exceeding 0.80 for all dimensions. The "overload-exhaustion" relation was r = 0.59 (p < 0.001), "lack of development"-"cynicism", r = 0.49 (p < 0.001), "neglect"-"inefficiency", r = 0.47 (p < 0.001). The "overload"-"lack of development" relation was r = 0.21 (p < 0.001), "overload"-"neglect", r = 0.20 (p < 0.001), and "lack of development"-"neglect", r = 0.38 (p < 0.001). The BCSQ-12-SS explained 38.44% of the variability in "exhaustion", (Ry.123 = 0.62), 30.25% in "cynicism" (Ry.123 = 0.55), and 26.01% in "inefficiency" (Ry.123 = 0.51). "Hours spent on studying" was found to be associated with "overload" (p = 0.001), "campus" with "lack of development" (p = 0.013), and ""failed subjects" with "neglect" (p = 0.011).
Conclusions:
The results support the definition of burnout as established by the BCSQ-12-SS. As such, the BCSQ-12-SS can be used for the recognition of clinical profiles and for the suggestion of potential intervention strategies specific to the characteristics of each particular case.
-
Propensity score matching in estimating the effect of managerial education on academic planning behavior
Background:
In many academic settings teaching a particular topic is applied to every student enrolled in the same academic year, it is a difficult task for researchers to design a randomized control group study. This research aimed to estimate the effect of teaching management and planning on increasing academic planning behavior (APB), using propensity score matching (PSM).
Methods:
In a cross-sectional survey utilizing a self-reported structured questionnaire on a systematic random sample of 421 students in Hanoi Medical University, one of the eight medical schools in Vietnam, this evaluation study adopted regression procedures to assess model fit, then PSM to create a matched control group in order to allow for evaluating the effect of management education.
Results:
The study showed both direct and indirect effects of the education on behavior. After PSM to adjust for the possible confounders to balance statistically two groups - with and without management education, there is statistically a significant difference in APB between these two groups, making a net difference of 18.60% (p < .05). The estimated 18.6 percentage point increase can be translated into the practice of APB by 670 students in the population. This number of academic planners can be attributed to a high recall of important management and planning education.
Conclusions:
The study provided theoretical as well as practical implications to guide the design of the education and evaluation of teaching.
-
The role of deliberate practice in the acquisition of clinical skills
Background:
The role of deliberate practice in medical students' development from novice to expert was examined for preclinical skill training.
Methods:
Students in years 1-3 completed 34 Likert type items, adapted from a questionnaire about the use of deliberate practice in cognitive learning. Exploratory factor analysis and reliability analysis were used to validate the questionnaire. Analysis of variance examined differences between years and regression analysis the relationship between deliberate practice and skill test results.
Results:
875 students participated (90%). Factor analysis yielded four factors: planning, concentration/dedication, repetition/revision, study style/self reflection. Student scores on 'Planning' increased over time, score on sub-scale 'repetition/revision' decreased. Student results on the clinical skill test correlated positively with scores on subscales 'planning' and 'concentration/dedication' in years 1 and 3, and with scores on subscale 'repetition/revision' in year 1.
Conclusions:
The positive effects on test results suggest that the role of deliberate practice in medical education merits further study. The cross-sectional design is a limitation, the large representative sample a strength of the study. The vanishing effect of repetition/revision may be attributable to inadequate feedback. Deliberate practice advocates sustained practice to address weaknesses, identified by (self-)assessment and stimulated by feedback. Further studies should use a longitudinal prospective design and extend the scope to expertise development during residency and beyond.
|
