The story so far
It all began with a Question
Welcome to our journey, from 2015 to this very day we have invented, invested and
re-invented daily to refine, to tune, to perfect our offerings.
Using a skills based approach to teaching and learning, how do we sustainably and transferrably transform a learners performance in the classroom and exam hall?
foresight and clarity with respect to their own strengths and weaknesses,
aspirations and evidence of being able to plan ahead,
emotional balance, and
ability to respond and cope with stress, anxiety and pressure.
These included the ability to:
retain and recall the content of their modules and assignments with relative ease and little stress and effort;
manage their time, showing a clear set of organisation skills such as prioritising, planning, scheduling and forecasting;
schedule their social life around their academic ambitions and having the discipline to prioritise their academic work when needed;
channel anxiety and cope with pressure in a positive manner, e.g. by increasing their effort to succeed.
This next phase (henceforth – Phase 1) also consisted of a more systematic definition and refinement of psychological and behavioural dimensions identified during Phase 0 and an improvement of PL’s assessment procedures and methods.
With respect to the latter, the company invested in the development of the Performance Learning Online Analysis (PLOA) tools, which served both to record and to analyse students’ responses with respect to 27 traits identified during this phase as common to pupils across the full range of academic ability (lowest to highest performing).
The 27 traits were determined through further research mainly in secondary schools in the UK. The traits described a pupil’s present PL diagnosis regarding grades, attitude to learning, behaviour, class attendance and participation. They also provided information about students’ mindset towards their learning along with possible barriers for their learning, their response to different learning environments (e.g. home or school) and their general well being.
These traits were then further segmented into PL’s five assessment risk levels, ranging from assessment level 1 (extremely high risk) to assessment level 5 (no risk), where the risk was defined concerning the degree to which a learner is believed to be able to reach a target or a predicted grade.
During the assessment, students answer a set of 64 questions with each response being placed in a report outlining their behavioural and psychological characteristics. The PLOA tool offers its assessment back to the students and teachers in the form of a score for each psycho-behavioural category assessed. It also offers each pupil a set of behavioural targets to achieve over the course of PL’s curriculum.
The students are grouped according to their needs and risk levels. Face-to-face sessions are delivered to each group on a weekly and then fortnightly basis, with the view to gradually scaffold the learners into a habit of independent, critical and regular self-appraisal, goal-setting and action. As well as undertaking regular PLOA assessments, pupils receive a printed manual outlining each lesson.
The PL lessons are specific to the pupil reaching their target PLOA score, with the teacher scoring the pupil within the system at the end of each lesson to assess the pupil’s progress towards their PLOA targets.
Crucially to PL’s overarching vision, the improvement in performance of the PL cohort was even more significant for students on free school meals, with the percentage of students who achieved one or more grade higher than predicted being four times above their PL Nil peers’ grades for English, and around three times for math.
Phase 1: implementation
PL is now in Phase 2 of its operation and development. The focus is presently on automating and further refinement of its approach to pupil assessment. Working with University College London’s Knowledge Lab, PL sees a particular opportunity in mining of the data generated to help
understand better the behavioural patterns of relevance to self-reflection and self-regulation and
inform further development of PL’s technology, mainly focusing on real-time modelling of learners’ behaviours and metacognitive competencies’.
To date, PL’s Phase 2 has brought about significant enhancements in its:
method of assessment to allow the students to self-assess using non-discrete social, emotional and mental categories in a way that captures the nuance of their states; this is achieved through a modal interface as shown in Fig.3;
frequency of assessment, which is now conducted at the beginning and at the end of each PL lesson, whereby the learner is asked fundamental questions about individual lessons to assess their motivation, understanding and their willingness and likelihood of applying the skill in their wider academic and personal contexts
delivery of lessons, each lesson is now delivered digitally through game like interactions with the system tracking and recording data such as time on task, accuracy, completion attempts, quantity of usage;
volume and nature of data collected, collected from the teachers’ assessments of the individual pupils as well as pupils self assessments, providing a unique opportunity for a systematic comparison between the two perspectives;
expanded set of behavioural traits along with a definition of a scoring mechanism for modelling and qualifying students behaviours along a spectrum of their strengths and weaknesses.
In this phase, the assessment categories have been expanded from the original 27 to 35 to enable the system to qualify pupils’ self-assessments and teachers’ assessments along with a spectrum of students’ strengths and weaknesses. Furthermore, the initial PLOA assessment categorises the pupil along a set of behaviours, allocating a score per behavioural category and a definition explaining the category and the specific score (see Fig.4).
This, in turn, provides a concrete basis for the pupils’ reflection and gives teachers the ability to check their “gut feeling” perceptions and assessments of their students.
As such, the definitions provide a common ground for both the teachers and the learners to discuss the individual assessments in a way that is targeted, systematic and inspectable over time.
outcome oriented mindset,
While distinct, these domains are interrelated and mutually impacting.
Phase 2: Automation
This work that was undertaken was essential to creating solid, evidence-based foundations. While laborious and time-consuming, the iterative research and development methodology that was adopted during this phase allowed the latest research at the time to be engaged with to ensure a unique product was developed.
The data generated through the technology was invaluable to future development. Specifically, data generated through the platform was created to be mined exhaustively in relation to well-refined research questions of pertinence to metacognition and learning achievement.
They said that these phases of research ‘involved brokering strong relationships between academic researchers and a commercial organisation. This brokerage is a key to taking AI research to scale and demonstrating its impact on learning.’
During the partnership with UCL, we progressed together through three stages of refinement validation and technological implementation. Each stage produced key conclusions that informed each following iteration.
The results of the first intervention involving 28 subject-independent lessons delivered on a one – to – one basis using paper-based training materials were encouraging with the 14 PL students outperforming the rest of their peers who were not using PL.
Following this, further refinement of pupil assessments was commenced along with in-depth analysis of existing literature to provide both research evidence for the behavioural categories in the assessment and to explain the nature of the relationship to metacognitive competencies and academic success.
The goal is to instill a habit, both in teachers and learners, to regularly reflect on key factors as such reflection is known to lead to targeted planning and action and ultimately to better learning outcomes.
Phase 3: Intelligence
Through the existing engagement with The UCL Knowledge Lab, Institute of Education, University College London Institute of Education and our software development partners as well as the hiring of two headteachers who took a secondment from their headship to join us, we began to develop the following areas in our existing model, carried forward from through the last phase of develop and development: