CTE
CTEVirginia Commonwealth UniversityCenter for Teaching ExcellenceCTE
CTE HomeAbout the CTEProgramsWorkshopsResourcesWhat's New
CTE
Bookmark This Site CTE Site Map Printer Friendly Text Size:SmallNormalLargeExtra Large
  Programs

Enhancing Student Engagement and Critical Thinking During Chaplain Intern Orientation

Alexander Tartaglia, D. Min. & Diane Dodd-McCue, DBA
Program in Patient Counseling
School of Allied Health Professions

 

Background

Patient Counseling offers an intern program in collaboration with MCV Hospitals where students serve as hospital chaplains for a 10-week summer rotation.  Students participate in an extensive orientation to hospital as well as department policies and procedures that directly impact their role.  Formal orientation typically involves a 20-hour content-based program.  Content focused orientation process runs counter-intuitive to the primarily experiential methods used for pastoral care training.

Objectives

The primary objective of the project was to revise the orientation process and align it with the experience-based, action-reflection method already employed within the program.  Corollary objectives include:

  1. Incorporation of a critical thinking component to orientation without sacrificing coverage of hospital/departmental policies and procedures
  2. Assessment of utilization of videotaped, case-based scenarios rather than didactic content-focused delivery of orientation information
  3. Enhancement of student engagement through a shared learning experience

Description

Eleven students without clinical experience were enrolled in the 2008 summer intern program.  The study focused on four key components of orientation: Advance Directives, Ethics Committee and Ethics Consultation, Pastoral Care Scope of Practice, and Pastoral Care in a Multi-faith Context.  Role plays were conducted using faculty as standardized patients/family members and advanced M.S. degree students as chaplains.  Role plays were videotaped and posted into a Blackboard course using the updated computer and software purchased through this CTE grant.  Prior to the introduction of each component, students viewed the videotaped scenario.  The accompanying assignment was to identify key content learned and key critical thinking issues raised in the role play.  The faculty responsible for each orientation component then used the assignment to initiate discussion related to key content and critical thinking elements.  The study was approved by VCU’s Human Subjects Review Board.

Assessment

Each student completed a knowledge-based pre-test on each of the four selected orientation topics.  The pre-tests, developed and piloted by faculty, included 10 statements eliciting an agree/disagree response.

Following each orientation discussion during the first week, students completed a post-test comprised of the same items as the pre-test.  A second post-test was completed by the students during week 10 - the final week - of the program.  Instead of leaving the questions as a simple “agree” and “disagree” option, the second post-test tool included a “reason” cell to help determine the “thinking” process for each response. 

Student evaluation of the videotaped scenario methodology was conducted during week 10 of the program.  In addition, faculty and experienced M.S. students evaluated the methodology’s effectiveness.

Results

The use of videotaped scenarios to introduce orientation topics demonstrated both immediate and sustained acquisition of new knowledge for clinical practice within the hospital setting.  Here are the results of the pre and post tests.  Improvement is demonstrated by a reduced number of incorrect responses.

Advance Directives:  Ten of 11students demonstrated improved knowledge from pre-test to post-test 1.  Ten of 11 students demonstrated improved knowledge from pre-test to post-test 2.  One student who demonstrated no change from pre-test to post-test 1 demonstrated improvement in post-test 2.  Four students demonstrated further improved knowledge from post-test 1 to post-test 2.  Five demonstrated no change and one did worse.

Ethics Committee and Consultation Service:  Nine of 11 students demonstrated improved knowledge from pre-test to post-test 1.  Two students demonstrated no change.  Nine of 11 students demonstrated improved knowledge from pre-test to post-test 2.  Six students demonstrated further improved knowledge from post-test 1 to post-test 2.  Four demonstrated no change and one did worse.

Pastoral Care Scope of Practice:  Ten of 11 students demonstrated improved knowledge from pre-test to post-test 1.  Nine of 11 students demonstrated improved knowledge from pre-test to post-test 2.  One student who did worse from pre-test to post-test 1 also did worse from pre-test to post-test 2.  Six students demonstrated further improved knowledge from post-test 1 to post-test 2.  Two students demonstrated no change.  Both of these students provided four incorrect answers in the pre-test and zero in each of the post-tests.  Three students did worse from post-test 1 to post-test 2.  Two students demonstrated dramatic change.  Both provided four incorrect answers on the pre-test and zero on each of the post-tests.

Pastoral Care in Multi-faith Situation:  Eight of 11 students demonstrated improved knowledge from pre-test to post-test 1.  One student demonstrated no change and two demonstrated a negative change.  Nine of 11 students demonstrated improved knowledge from pre-test to post-test 2.  One demonstrated no change and one demonstrated a negative change.  Seven students demonstrated further improved knowledge from post-test 1 to post-test 2.  Four demonstrated no change.  The most dramatic change was demonstrated by the student who provided five incorrect answers in the pre-test and two incorrect answers in post-test 1 and one incorrect answer in post-test two.

Evaluation of Video Activity

Table A

Assessment of Use of Videos (11 Interns)
1-5 Scale with 5 as High

 

Range

Average

Median

Mode

Increased Knowledge of Hospital Policies

1 - 5

3.1

3

4

Helped Apply Critical Thinking to Hospital Scenarios

2 - 5

3.5

4

2,4,5

Scenarios Helped to Promote Critical Thinking

1 - 5

3.5

4

4

Scenarios Emphasized Criteria for Problem Solving

2 - 4

3.5

4

4

Scenarios Emphasized Possibility of Several Right Answers

2 - 5

3.3

3

3

Scenarios Promoted an Atmosphere to Enhance Student Engagement

1 - 5

3.3

3

3,4

Scenarios Promoted my Leaning

1 - 5

3

3

2

Overall Use of Scenarios Recommended over other Teaching Methods

1 - 4

2.9

3

4

Advance Directive Scenario was Useful for Pastoral Practice

1 - 5

2.8

3

2

Scope of Practice/Prioritizing Scenario was Useful for Pastoral Practice

1 - 4

3.2

4

4

Ethics Consultation Scenario was Useful for Pastoral Practice

2 - 5

3.4

4

4

Multi-Faith Scenario was Useful for Pastoral Practice

2 - 5

3.5

4

4

 

Table B

Assessment of Use of Videos (11 Faculty and Residents)
1-5 Scale with 5 as High

 

Range

Average

Median

Mode

Increased Student Knowledge of Hospital Policies

3-5 

4.1 

Videotaping Helped Interns Apply Critical Thinking to Hospital Scenarios

3-5 

4.0 

Scenarios Helped Promote Critical Thinking for Interns

2-5 

3.7 

Scenarios Emphasized Criteria for Problem Solving

3-5 

4.1 

Scenarios Emphasized Possibility of Several Right Answers

3-5 

3.9 

Scenarios Promoted an Atmosphere to Enhance Student Engagement

3-5 

4.6 

Scenarios Promoted Intern Leaning

4-5 

4.5 

Overall Use of Scenarios Recommended over Content only Methods

2-5 

4.5 

5

Discussion

Student Learning: Relative to orientation topic content evaluated across the three points in time, the results point to improved student understanding in each of the four scenario content areas.  Results of the second post-test demonstrate a sustained learning pattern.  Also, the relative change recorded for students who initially had the lowest number of correct answers to the survey questions was often dramatic.  Relative to the use of scenarios as the delivery method for orientation content, the results support the theory that interactive interventions would produce positive student outcomes. 

One limitation of this study is the lack of a control group.  While orientation has consistently been the most critiqued element of the summer intern program, no controlled data has been collected evaluating the traditional didactic only method of presenting the content areas.   Without a control group of students, it is difficult to assess the extent to which this was an improvement when compared to the traditional didactic/content based orientation. 

Student and Faculty/Resident Assessment:  Balanced understanding of these findings mandates consideration of two perspectives, the students and the faculty/residents.  Two conclusions can be clearly drawn from the subjective evaluations completed by the two groups.  First, subjective student intern evaluation of the use of videotaped scenarios failed to reflect the extent of new and sustained student learning.  This may well reflect differences in student learning styles, with some students more apt to prefer traditional methodologies.  Other students may remain less comfortable with ambiguous clinical situations preferring a teaching style that offers clear and defined expectations.  Student evaluations could also reflect consistency in historical feedback that orientation still contains too much information to absorb in a limited timeframe. 

Second, evaluation of the orientation initiative was more highly regarded by faculty and experienced CPE residents than new student interns.  The difference between students and faculty may be attributable to the latter’s inherent desire for the project to be successful.  This difference may be somewhat mitigated by the fact that residents, who had considerably less investment in the initiative, also evaluated the project more positively.  The more optimistic interpretation is that faculty was able to identify beneficial use of videotaped scenarios in the student intern learning that the students themselves had not consciously internalized.  There is some support for this notion in that process learning includes the development of learning how to learn.

Other Learning:  Relative to future efforts to incorporate scenarios into student orientation, three additional lessons were learned.  First, greater attentiveness should be given to assure that the evaluation methodology is congruent with the objective of increased critical thinking. Initially the pre- and post-test surveys relied on absolute binomial responses. In retrospect, this counters the objective of increasing student critical thinking and efforts to promote greater recognition of the complexity of the topics covered.  Thus, the revised final post-test instrument, by providing students the opportunity to provide substantiation for their responses, is a more appropriate tool.

Second, collection of data at multiple points in time is strongly recommended.  In this case, collection of data students at three points in time allowed not only for evaluation of the effectiveness of content delivery but for identification of areas for clarification. 

Third, evaluation of pedagogical methodology from the perspective of both students and faculty (and residents) is strongly encouraged.  Although the primary goals of the orientation scenarios were sustained student knowledge specific content as well as enhanced student engagement, the faculty’s perspective of the efficiency and ease of implementation is critical for sustaining this orientation delivery methodology in the long run.  

Future

Sufficient documentation of student learning as well as student and faculty satisfaction exists to support repetition and replication.  Adjustments noted in this report in developing videotaped scenarios for the 2009 cohort of summer interns.  We have approached our colleagues at another academic medical center with the possibility of duplicating this study at their hospital-based clinical pastoral education program next summer.

The equipment and software purchased with the Center of Teaching Excellence grant will allow us to sustain this initiative.  As we refine this study we anticipate submitting it as poster presentation at the Mid-Atlantic Region, ACPE Annual Meeting.  An article will be submitted for review to a pastoral care and education journal.

« Back to Small Grants Program home

   

top
 
Virginia Commonwealth University | Center for Teaching Excellence
Last modified: June 20, 2013
Contact webmaster
  Academic Learning Commons
1000 Floyd Avenue, Suite 4102
Richmond, VA 23284
(804) 828-4470