LJMU Accelerates NSS Analysis: 38,000 Student Comments Processed in Minutes with Explorance MLY

About Liverpool John Moores University

Liverpool John Moores University, with a community of over 25,000 students, is a dynamic institution known for its strong emphasis on research and practical learning. Located in the heart of Liverpool, it offers a wide range of undergraduate and postgraduate programs across various disciplines.

The Challenge: Unlocking Timely and Actionable Insights from Qualitative Analysis

Liverpool John Moores University (LJMU) had been using generic text-mining software to analyse open-text comments derived from student surveys for over 10 years. However, the process proved challenging due to an overreliance on the interpretation of a single researcher, a lack of dynamic learning, and the resource-intensive nature of the system. Since the model was not trained on student feedback, extensive manual thematic analysis was required to contextualise the results generated by the software.

“The value of open-text feedback is that it leads to actionable insights and can actively inform local decision-making and broader institutional strategies related to student retention and success,” explained Fenna Boerkamp, Institutional Research and Evaluation Officer in the University’s Student Voice and Evaluation Team. “Whilst we had used text analytics technology for institutional survey analysis for a long time, we were exploring opportunities to introduce a model that was specifically designed to analyse student feedback to better support timely decision-making.”

The Solution: Implementing Purpose-built Feedback Analytics

LJMU selected Explorance MLY due to the platform being specifically trained to analyse student feedback, and for the additional features it offers. These include MLY’s alerts (instant access to concerning/problematic comments, or comments indicative of a student being in distress), analysis sharing functions, and the option to compare demographic groups.

“MLY not only derives topics but also generates recommendations from the comments which is a useful feature”, Fenna said. “It allows us to breakdown the data by demographics using the filters in the tool, and also create widgets for specific demographic groups, for example the most frequent topics for only male or female students. Additionally, MLY enables sharing and collaboration on the projects, allowing all key stakeholders to participate on interpretation and finetune the findings together.

For our institution specifically, the alert function that MLY offers has been really useful as we still, according to our policy, have to redact all names out of negative comments. That means we read all comments in our module evaluation survey, and we try to figure out which comments we may need to remove names or parts, and the triggers can significantly reduce the time needed to complete this process.”

The University applied MLY to two specific projects during the first six months. The first was for analysis of the 2023 National Student Survey (NSS). The test project was successful, as the majority of themes identified by MLY matched LJMU’s rapid manual analysis. Additionally, the recommendations from MLY were valuable in supporting the institution’s action plans, proving its effectiveness for NSS-type analysis.”

The Outcome: A Faster and More Scalable Approach to Processing Qualitative Data

For LJMU, the application of MLY has already delivered valuable insights that have enhanced the institution’s understanding of the student experience.

“Using MLY to analyse qualitative data is ultimately far less time consuming and resource intensive as MLY does more of the work for you,” Fenna commented. “It is also possible to process very large amounts of data: for example, we have generated initial insights from over 38,000 comments for module evaluation in few minutes. Hence, scalability is a huge benefit of MLY. It is easier to cover a much larger amount of comments and derive more recommendations from the comments than you would manually be able to, and that provides more actionable insights that informs institutional change.”

Fenna concluded: “Adding MLY to LJMU’s portfolio of analytical tools has enhanced understanding of student needs and improved the student experience by providing faster, real-time analysis without the need for lengthy manual data processing. The constantly learning algorithm and new features have definitely added continuous value in analyzing student feedback at scale.”