In response to the specialization of the expanded scope of pharmacist services, the reformation of the pharmacy education system, and the wave of intelligence, the training of hospital pharmacists is becoming increasingly diversified and specialization, especially in the area of clinical pharmacy care. Based on the healthcare system's commitment and social responsibility to the demand for pharmacist expertise and the medication safety of the public, the Department of Pharmacy of Mackay Memorial Hospital has, since 2016, initiated a competency-based pharmacy training program for its two-year postgraduate pharmacists, the group with the largest number of trainees in the department, with an emphasis on post-training outcomes to cultivate clinical pharmacy talent. It establishes a training framework encompassing six core competencies, 17 sub-competencies, and 173 learning milestones. Accordingly, the department has designed courses and training activities to provide guidance for clinical faculty and trainees, and introduced 7 entrustable professional activities (EPAs) and 4 specific EPAs for core clinical tasks. These acts help translate the core competency framework into actionable clinical pharmacy care training and assessment, promoting the development of workplace-based evaluations and milestone plans, and ultimately, in combination with the integrated EPAs, complement each other to form the structure and foundation for the comprehensive assessment of the two-year postgraduate pharmacists training. Meanwhile, the Clinical Competency Committee (CCC) has been established to identify trainees' personal shortcomings and deficiencies in the training program, and to provide personalized feedback and adjustments to the training content. Currently, the Department of Pharmacy consists of 97 pharmacists, with approximately 15 to 20 PGY pharmacists in training. Each PGY pharmacist receives comprehensive training in outpatient, emergency, and inpatient pharmacy services in accordance with the training program. Upon completion of PGY training, pharmacists can be transition to generalist or specialist residency training to develop more advanced clinical skills. (Details of trainees are shown in Annex, Figure 1)

• Introduction to the Services of the Department of Pharmacy (Details shown in the Annex, Table 1)

• The pharmacist training program has a clear blueprint that emphasizes a learner-centered approach, effectively integrates resources, and reduces the duration of the training period

• Assessment tools to ensure quality of assessment

• All professional skills assessment tools have undergone expert evaluation, with an average content validity (CVI) of over 0.8, and an internal consistency reliability of over 0.7, demonstrating that the assessment tools are suitable for trainee evaluation.

• Learning effectiveness and guidance for trainees

• Clear guidelines are in place for addressing learning outcomes, including remediation or exit mechanisms for underperforming or unfit trainees. Unauthorized pharmacists are required to work under the supervision of clinical instructors to ensure patient safety.

• Ongoing development of pharmacy specialization

• The Department of Pharmacy encompasses multiple specialized fields, using systems such as the National Health Insurance (NHI) PharmaCloud System to develop integrated pharmacist outpatient clinics that provide comprehensive pharmaceutical care and continuity of patient care.

Rigorous consensus-building methods, assessment tools, curriculum design (integration of AR and VR in teaching and assessment), an information platform for consolidating learning records and instructor evaluation results, and real-time presentation of outcomes (e.g., Power BI).

1.The Department of Pharmacy uses the Consensus Building Method (Delphi Method) for expert consensus development.

This training framework is established and introduced through a rigorous consensus building and validation process to ensure quality, reliability, and validity. Core competency indicators and milestone items were developed using quantitative, structured analysis to ensure expert agreement, and EPAs are required to meet an EQual rubric Cut score of 4.07. The expert consensus was accomplished in three stages as follows:

Stage 1: Consensus building for milestone indicators

Adopting the Delphi method and collecting experts' opinions as the primary research methods. Pharmacy experts from the Department of Pharmacy were invited to vote on and revise each learning indicator of the pharmacist Milestones through multiple rounds of anonymous surveys. This ensured that the proposed core competencies were verified in terms of their significance and assessability. The first version of the two-year postgraduate pharmacist learning milestones was thereby formulated.

Stage 2: EPAs lesson plan writing and validation

Through expert meetings and content analysis, consensus discussions were held to identify critical clinical tasks for the two-year postgraduate pharmacist training program in hospitals. The milestones established in Stage 1 were integrated with these tasks to translate hundreds of abstract and complex learning objectives into actionable EPAs in the clinical setting.

Stage 3: Mapping of EPAs to milestones

Using Delphi method, representatives of the clinical faculty conducted an in-depth review and validation of the content of the EPAs and their alignment with the Milestone indicators (Milestone-based EPAs). After three stages of refinement, a consensus was reached to confirm that each of the professional activities was sufficiently appropriate for the competency assessment.

2.Nationwide expansion based on MacKay Memorial Hospital’s experience, using the nominal group technique for expert consensus

To enhance the representativeness and acceptance of the EPAs, during July to August 2023, we initiated a nationwide consensus meeting to establish EPAs for newly recruited pharmacists in teaching hospitals. Participants in the consensus meeting were required to be certified instructors in the Medical Personnel Training Program and have received EPAs course training. In the selection of participants, nationwide representativeness was also taken into consideration in terms of hospital level (medical centers, regional hospitals, and district hospitals), regional distribution (North, Central, South, and East), and healthcare system (one representative per system). A total of 27 (100%) clinical educators were recruited from 10 (37.03%) medical centers, 11 (40.75%) regional hospitals, and 6 (22.22%) district hospitals. We also invited scholars with expertise in competency-based medical education and consensus-building methodologies, external experts appointed by Taiwan Society of Health-System Pharmacists, to serve as moderators of the consensus meetings. We employed the nominal group technique to reach a consensus. Prior to the meeting, all expert opinions were collected and made into proposals for discussion. During the consensus meeting, the moderator invited the experts to review each proposal and asked for any suggestions for changes, including additions, deletions, and modifications. Proposals for changes are required to be seconded by at least one other expert before they can be discussed, voted on, and documented. To ensure a high level of agreement, each proposal is extensively discussed and commented on, and then voted on to present a consensus result. More than half of the votes are required for the proposal to stand. The process involved in-depth discussions among all experts to reach a consensus and form a final decision. Voting was conducted anonymously using the Zuvio interactive feedback system and the results were displayed immediately. This is to ensure the transparency and fairness of the process. The framework for eight EPAs tasks for the two-year pharmacist postgraduate training was finalized after four half-day expert consensus meetings. Six core competencies and 15 sub-core competencies were identified through three additional expert consensus meetings.

• Comparison with related organizations in other countries

• A literature review was conducted to examine whether pharmacy associations or healthcare institutions in other countries have similar training blueprints or implementation experiences. It was found that while countries such as the United States, Canada, Australia, and Singapore widely agree on the adoption of competency-based training frameworks as a common consensus and direction, most pharmacy associations in these nations are confined to establishing core competencies and EPA items. Although there are some small EPA programs for pharmacy students or community pharmacy pharmacists, there is no comprehensive training framework for PGY pharmacists that integrates both EPA and milestone plans.

• Oral presentation at the international conferences

• The results of the development of Taiwan's pharmacist competency framework were selected for oral presentation at the Annual Conference of the Association for Medical Education in Europe (AMEE), allowing Taiwan's advancements in pharmacy education to be recognized on the international stage. (as shown in Annex_Figure 3)

• Oral Presentation Topic: Establishing Competency Framework of Taiwanese Pharmacists Using the Nominal Group Technique

• Publications

• Leading the EPA working group of the Taiwan Society of Health-System Pharmacists, based on the draft developed by Mackay Memorial Hospital, a framework of six core competencies for the pharmacist was proposed in December 2022 using the Nominal Group Technique through a rigorous process of consensus building. Through integration with the professional practice model of pharmacy, this framework was refined into 15 sub-competencies specifically for the two-year post-graduate pharmacist training program.

• The core competencies, sub-core competencies, and descriptions of sub-core competencies for pharmacists in Taiwan, along with the complete EPAs, were published on the website of Taiwan Society of Health-System Pharmacists as a blueprint for teaching hospitals to develop pharmacist training programs (https://reurl.cc/NlGoZq). The “2023 Taiwan Pharmacists' Entrustable Professional Activity (EPAs) First Edition” was also posted on the website of Taiwan Society of Health-System Pharmacists (https://bit.ly/40zbONr). These will be continuously updated in the future with the goals of optimizing pharmacists' professional development and aligning with advancements in pharmacy practice.

• Meanwhile, the development process was published in educational journals for the reference of other healthcare professions.

• Establishing Core Competencies of Taiwanese Pharmacists Using the Nominal Group Technique.Ya-Hui Chang et al. Journal of Medical Education；6(4), (2022),P69-78.

• Empowering Pharmacy Faculty Training:Integrating EPAs for Enhanced Workplace Observation and Feedback Skills. Ya-Hui Chang et al. Journal of Medical Education；28(1), (2024), P22-31.

• Establishment of Entrustable Professional Activities for Two-Year Post-Graduate Pharmacists: A Consensus of Multicenter Hospitals in Taiwan. Ya-Hui Chang et al. Formosa Journal of Clinical Pharmacy. 31(4), (2023), pp. 287-299.