Sunday, July 28, 2013

Assessing Residents in the Era of Entrustable Professional Activities and Milestones

The Accreditation Council for Graduate Medical Education (ACGME) is implementing the "Next Accreditation System"(NAS) for medical and surgical post-graduate training programs in the United States.  As a part of NAS, programs will be reporting the progress of their residents on milestones of clinical competence that have been developed by each specialty.  The milestones for specialties that will begin reporting this year can be viewed at http://www.jgme.org/toc/jgme/5/1s1.

With the continuing shift to outcomes-based assessment, there is a need for ongoing education of both faculty and residents about this evolution in assessment.  Pediatric residency programs will begin reporting on 21(of the 48) milestones in June 2014. 

Below is a proposed method for education about the Milestones that I am going to pilot for my department (ambulatory pediatrics).  Challenges for education in our department include faculty working in different locations/ different shifts and variability in familiarity with the milestones.

After reviewing the proposed faculty development plan, please feel free to comment and offer suggestions.

Proposed Faculty Development Plan regarding new resident evaluations.

Independent Learning (estimated time 30-45 minutes)
  
1) Review the 15 minute narrated PowerPoint- overview of the milestones and entrustable professional activities (EPAs). Video uses Windows Media player.

video


2) Do a self- assessment using selected milestones to become familiar with content and method of assessment.  Please complete PC-1,4,5, MK-1, PBL-4, ICS-1, PROF-1, SBP-1,3 of Pediatric Milestones http://www.acgme-nas.org/assets/pdf/Milestones/PediatricsMilestones.pdf

3) Additional readings (optional)

a) CL Carraccio, JB Benson, LJ Nixon, PL Derstine "From the educational bench to the clinical bedside: translating the Dreyfus developmental model to the learning of clinical skills" Acad Med 2008;83:761-767
http://journals.lww.com/academicmedicine/Fulltext/2008/08000/From_the_Educational_Bench_to_the_Clinical.17.aspx

b) PK Hicks et all "The Pediatrics Milestones: Conceptual Framework, Guiding Principles, and Approach to Development" Journal of Graduate Medical Education, Sept 2010; 410-418
www.jgme.org/doi/pdf/10.4300/JGME-D-10-00157.1

c) JR Frank et all "Competency based medical education: theory to practice" Medical Teacher 2010; 32:638-645.
http://groups.medbiq.org/medbiq/download/attachments/15991574/Frank+et+al.+-+Competency-based+medical+education+theory+to+practice.pdf?version=1&modificationDate=1327513526000


Group Learning Session (60- 90 minutes.  Independent learning above done as "pre-work")

1) Guided discussion of the evaluations for our rotation and review of "Snapshots" to provide examples of what novice, advanced beginner, competent, proficient and expert learners would look like for different EPAs.

2) Practice as a group evaluating current or recent residents using the new evaluation system.  Discuss areas that are difficult to evaluate or where there is disagreement.

(For faculty that are community-based we will need to explore additional opportunities for group discussion and learning via webinars or other on-line opportunities)

On-going Application ("post-work" 60 -90 minutes).  You may pick either option. 

1)Attendance at group evaluation sessions of residents (in person or call in)

2) Implement a Plan Do Study Act cycle for an assessment method for the milestones (examples- resident self-assessment on a milestone, chart stimulated recall).   Discuss outcome and lessons learned at monthly provider meeting (or alternate way to share learning for community based physicians)

Faculty would document their work on resident assessment as a part of their annual departmental goals and objectives and would receive institutional faculty development credit for their work.