As part of the Advanced Teacher Status (ATS) programme, participants are required to undertake a quality improvement project.
SET’s Professional Status team reviewed all 60+ quality improvement projects submitted as part of the October 2020 ATS cohort, shortlisting the top 12 using a scoring matrix, including the one below from Vivienne Stockill, Lecturer at Heart of Worcestershire College.
This editorial provides an overview of Vivienne’s improvement project which focused on models for better evaluating continuing professional development (CPD).
The Tyler et al report (2017) on the role of advanced practitioners (APs) identified that tracking, monitoring and measuring of outcomes from staff development interventions was often inconsistent. However, Clay (2018) states that effective professional development has a significant impact on learners’ achievement and is the educational leadership activity that has the largest impact on learner outcomes. She goes on to add that it should always be undertaken with a clear and specific need in mind, and to assess the impact of the activity, the question that should be addressed is what success would look like if the need is met. Therefore, there is a need to establish what teachers and the organisation expect to gain from the intervention, and how the benefit of the support will be measured.
Moreover, there is an Ofsted requirement that to achieve “outstanding” grading in the inspection of Leadership and Management, the organisation must evidence that leaders ensure teachers receive focused and highly effective professional development (Further Education and Skills Inspection Handbook, para 275).
At present, in this college, a simple form is used at the end of staff development sessions which asks participants if they enjoyed the session, if the aims of the session were met and if the level was appropriate to current levels of skills and knowledge. In addition, they are asked if there is anything that participants will take away from the session to try out within their own practice and whether further input is needed in terms of additional professional development or one-to-one support.
This project examines how professional development programmes delivered by advanced practitioners (and others) may be better evaluated by applying a suggested model in real situations.
To evaluate the impact of staff development activities delivered by advanced practitioners and others, with the objectives to:
A review of relevant literature was conducted to ascertain methodology for the evaluation of staff development.
Data was collected from a number of staff development sessions conducted between September 2020 and April 2021 to evaluate the efficacy of the intervention. The data was obtained using a variety of methods: questionnaires, observations, quizzes and interviews.
The Kirkpatrick model (LinwaysTechnologies, 2020) for evaluation of professional development was followed with each level building upon the next, as illustrated in the diagram.
All responses to questionnaires, interviews and quizzes were collected anonymously to enable confidentiality. Participants were informed that their responses were anonymous and would only be used for the purpose of this study, and that when responses had been collated and incorporated into a report, the data would be held securely within the college’s systems and destroyed after 12 months. This enabled participants to answer questions candidly.
Summary of literature review:
Data was collected from a total of seven staff development sessions and involved 131 participants in sessions.
Using the Kirkpatrick model for evaluation of professional development mentioned above, data was collected at levels one, two and three. Raw data has been redacted to ensure confidentiality but findings are summarised below.
A number of staff commented on the fact that it was enjoyable working with other colleagues and helpful to be able to engage with staff from different departments. However, participants did not understand in many cases why they were required to attend, and no pre-training briefing was carried out.
Further comments that were made suggest that in some cases, participants felt that the material covered had been duplicated in other sessions they had attended. Given that attendance was largely compulsory, this may indicate a need for more careful consideration by line managers as to the specific development needs of staff.
There was a disappointing response to quizzes and tasks which were intended to measure learning from the session. Where this was done at the time of the development session, a 100% response was obtained, but where staff were asked to respond to quizzes and tasks at some time after the session, the response rate was poor. In fact, it was only possible to demonstrate understanding in 48% of attendees. However, of those who responded to tasks and quizzes, 93% demonstrated understanding of the concepts.
Response rates were partly impeded by the fact that some sessions took place online and this did not always lend itself to obtaining immediate feedback, but if the effectiveness of staff development is to be demonstrated, consideration needs to be given to how this may be undertaken.
Responses suggested that teachers take knowledge from the sessions and are keen to put it into practice. However, there was no follow up action by line managers to use opportunities to support practitioners in their new learning, and without evaluation of the impact (level 4), it is not possible to assess whether this is done effectively to improve outcomes for learners.
From the results above, it is difficult to see how subject matter for staff development is determined and how this is communicated to staff. Further, there appears to be no preparation for staff prior to the development session, and no de-briefing or evaluation by line managers after the event.
For evaluation to be effective, the organisation as a whole needs to see the value to be gained from it. Effective performance management in staff development is not seen or valued as a line management activity as there is little evidence of line managers actively working with team members on individual development. Further, observations which would confirm the impact of staff development are seen as an activity that is carried out by managers, rather than using the existing peer review process or allowing APs to carry out learning walks for evaluation purposes.
Clay, B. (2016). Five principles to help you evaluate your CPD. [Online] Available at: https://www.sec-ed.co.uk/best-practice/five-principles-to-help-you-evaluate-your-cpd/ [Accessed 28 October 2020].
Guskey, T. (2002). Does it make a difference? Evaluating professional development. [Online] Available at: http://www.ascd.org/publications/educational-leadership/mar02/vol59/num06/Does-It-Make-a-Difference%C2%A2-Evaluating-Professional-Development.aspx [Accessed 29 October 2020].
Herbert-Smith, K. (2018). How can you measure the impact of CPD? [Online] Available at: https://blog.irisconnect.com/uk/community/blog/how-can-you-measure-the-impact-of-cpd/ [Accessed 28 October 2020].
Higgins, S., Cordingley, P., Greany, T. and Coe, R. (2014). Developing Great Teaching [Online] Available at: https://tdtrust.org/about/dgt/ [Accessed 26 October 2020]. London: Teacher Development Trust.
Joyce, B. and Showers, B. (2002). Student achievement through staff development (3rd edition). London: Longman.
Linways Team (2020). An introduction to the Kirkpatrick Model: The four levels of learning evaluation. [Online] Available at: https://stories.linways.in/an-introduction-to-the-kirkpatrick-model-ac3058a16295 [Accessed 28 October 2020].
Pandey, A. (2018). Using Kirkpatrick’s Model of Training Evaluation. [Online] Available at: https://elearningindustry.com/roi-of-elearning-using-kirkpatricks-model-training-evaluation-determining [Accessed 28 October 2020].
Teacher Development Trust (2020). CPD Processes. [Online] Available at: https://tdtrust.org/about/evidence/#CPDprocesses [Accessed 26 October 2020]. London: Teacher Development Trust.
Thedham, J. (2019). How managers can support and develop advanced practitioners – an organisational approach. [Online] Available at: https://improving-teaching.excellencegateway.org.uk/content/etf3124 [Accessed 7 December 2020]. London: The Education and Training Foundation.
Tyler, E., Marvell, R., Green, M., Martin, A., Williams, J. and Huxley, C. (2017). Understanding the Role of Advanced Practitioners in English Further Education. [Online] Available at: https://www.excellencegateway.org.uk/content/etf2801 [Accessed 7 December 2020]. London: The Education and Training Foundation.
Weston, D. and Clay, B. (2018). Unleashing Great Teaching. London: Routledge.
Content from ATS Academic Poster - Viva preparation – Vivienne Stockill – 2022