Leveraging Student Evaluations to Improve Feedback Processes at the University of the Witwatersrand with Explorance Blue

About the University of the Witwatersrand

The University of the Witwatersrand (Wits University) is a multicampus public university situated in the northern areas of central Johannesburg. With five faculties, and 34 schools, Wits is home to approximately 3,700 full-time academics and over 40,000 students.

The Challenge: Moving from paper-based to online surveys, and evolving feedback culture

At Wits, the objectives of student feedback are two-fold: accountability (supporting quality assurance and career progression), and development (supporting reflective practice for professional learning and course improvement).

This is underpinned by the Witts Quality Assurance Framework for Educational Provision (WQAFEP), which provides a broad parameter for assuring and enhancing the quality of educational provision. Under the WQAFEP, the system elements that contribute directly to a quality student learning experience include academic offerings, teaching, student learning and engagement, student support, and services, facilities and resources.

The University’s Quality and Academic Planning Office (QAPO), and previously the Centre for Learning, Teaching and Development (CLTD), collects various forms of data from a range of stakeholders on their experience of, and contributions to, educational provision including: student evaluations, peer reviews of teaching, external examiner and internal moderator reports, throughput rates, input from class reps and the Student Representatives Council, advisory boards, reports from Faculty student advisors, support tickets, and emails.

It was the CLTD which, in the 2018-19 academic year, identified a need for an external platform to conduct student evaluations, and move from paper-based to online evaluations. The CLTD recommended that lecturers use student feedback as a formative opportunity, as the results offer useful indicators of trends but have limited value in getting to the heart of facilitation practices. Instead, they are most useful when contextualised in information obtained from more qualitative forms of evaluation, as scores can often be the consequence of multiple factors.

The Solution: Moving forward on the student evaluation journey with Explorance Blue

In 2019, Wits first selected Explorance Blue for end-to-end automation of their course evaluations, and to leverage text analytics to uncover insights hidden in open-ended students’ comments and enrich their reporting capabilities.

Blue was launched across all faculties in 2020, meaning that all academic staff should use the online system for conducting student evaluations. Question personalisation and response rate monitor were two of the tools deemed especially valuable, as the CLTD and now QAPO support the range of different stakeholders using the system, including lecturers and students.

Nompumelelo Mazibuko, QAPO Evaluations Coordinator, has overseen the student evaluation journey with Explorance Blue. “Initially, there was a lot of groundwork that had to be done to get buy-in from the institution to get academics to move from a paper-based to an online system,” she recalled. “However, we had a lot of issues with paper surveys, including around the manipulation of data, and the labour that had to go behind it because we had to manually scan survey forms. We began with a process of engaging with academics and students on how they felt about the core questions we were asking and how they felt about the feedback system, the gaps and changes required.”

With revisions guided by literature and feedback from staff, core questions and policy followed, which were agreed at an institutional level. Further work took place around sourcing data from different faculties, providing training and support for administration staff and academics, and cleaning and verifying data. All this was then fed into Blue, which conducts the evaluations and analyses the data, which is then located within an institution-specific Course Teacher App.

“Through Blue we have opt-in evaluations, where lecturers according to ‘need’ decide whether to conduct evaluations or not, across four evaluation cycles for full-time courses,” Nompumelelo explained. “This helps academics to evaluate the courses while they are teaching the course, and they help the students rather than leaving it right at the end. We have question personalisation, so in addition to the core items we have, we give academics an opportunity to add or design their own questions aligned with their different teaching context. This helps lecturers to create response-able evaluations.

“We also monitor student participation, using the Blue dashboard for that, and academics are given access to the dashboard to see how students are completing the evaluations, whilst they can also send reminders. Reports, for schools and institution-wide, are then accessed before a programme of closing the feedback loop.

The Outcome: From source to analysis; enhancing the quality, and reliability, of evaluations

The Wits journey of evaluations from source to analysis has involved leveraging data to improve feedback processes. Lecturer and student engagement throughout the evaluation process, facilitated through Blue, has been crucial.

“Overall, our goal is to enhance student participation to improve the quality and reliability of evaluations,” Nompumelelo said. “Explorance has supported change in student evaluation practices (designing response-able evaluations and data analysis), explored lecturers’ participation in developing surveys and designing good questions via question personalisation, and helped us to reflect on how we can engage differently for a better experience. Whilst there is more work to do, we now have a system in place whereby we communicate with students on how the feedback is used/has been used, and use student feedback to reflect on teaching and student success.”

She added: “A key consideration throughout has been ‘How do we influence the institutional culture in order to benefit from affordances of the Blue system to ensure better evaluations processes?’ We needed a systems-thinking approach to begin shifting thinking about evaluations from multiple points, and that is now what we are beginning to deliver.”