Optimizing Learning Analytics: Strategies for Boosting Survey Response Rates

A successful learning analytics program relies on the ability to collect data. The higher the response rates for your training surveys, the more confidence you can have in the accuracy of the data. We hope you find this to be a valuable guide to training industry best practices for increasing survey response rates.

Learning Analytics: Measuring Response Rates & Setting Goals

To improve response rates, the first step is to understand the percentage of learners who are submitting surveys. When measuring response rates, begin by separating the data into categories by survey type and delivery method. Determine what your response rates have been historically for each of these categories and how they are trending.

After gathering the data, it is important to set goals for response rates. You will certainly set different goals by survey type and delivery method. You might also set separate goals for your recurring learning events versus your strategic, visible, or costly programs. In setting your goals, ask the following questions:

• What response rate do we feel is needed for reasonable conclusions to be drawn for day-to-day learning measurement?
• What response rate do we feel is needed for a more in-depth measurement exercise?
• How much money and how many resources are we willing to commit to increasing response rates?
• Does management require minimum response rates?

Recommended Response Rates for Analysis

Explorance Metrics That Matter™ (MTM) learning measurement software recommends establishing thresholds for different measurement needs. The following provides some basic guidelines to follow based on different reporting needs and scenarios where data can provide valuable decision support.

To Manage L&D Operations or a Program:

All programs should collect surveys from 65% of the learner population. If rates are below 65% or established goals, target an increased participation rate of 10% annually.

To Manage Courseware:

When conducting an analysis of overall performance, aim for at least 500 surveys. For analysis of any specific course, collect at least 30 surveys and achieve a 65% response rate or higher. If rates for individual courses are below 65% or established goals, target an increased participation rate of 5% annually per course.

To Guide Improvement:

Even a single survey can be used to provide an anecdote of success or an opportunity for improvement. Consider the value of star ratings and comments of users in an app store, or when purchasing books online. This kind of information can help learners to choose learning activities and set expectations. On the other hand, a single piece of constructive feedback provides an opportunity to reach out to a learner to understand an issue further or fix a negative experience.

To Market Training Value:

To demonstrate value for a particular course or to a specific audience (i.e., internal/external client groups), collect at least 30 surveys and achieve a 65% response rate or higher.

Develop Survey Content with Consideration

When looking to increase response rates, the primary goal for the content of your survey should be to make it as easy as possible for respondents to complete it. To achieve this goal, consider the following approaches:

• Streamline your surveys by removing questions that will not be used in reporting.
• For shorter training events, use an abridged version of your primary survey that asks only the essential questions.
• Review each of the questions on the survey to ensure that they are clear, concise, and easy for a student to respond to.
• Provide the surveys in the primary languages of all your learners. If multiple languages are used within a particular audience, consider using the multi-language versioning functionality – a feature available in Explorance Metrics That Matter™.
• Accept anonymous survey submissions for surveys. Note that this could have a negative impact on follow-up surveys if email addresses are not collected for anonymous submissions.

Provide Ease of Access

The most effective way to boost response rates of surveys is to provide easy access and appropriate time for the respondents to complete surveys.

Accessing the Survey

When completing the surveys using Explorance Metrics That Matter™, respondents can reach the survey through several access points:

Surveys for Instructor-Led Training

Instructors should notify learners at the beginning of class that they will be asked to complete surveys and that they provide an opportunity to express their feedback on the event. Instructors should also note that the feedback is used to improve the quality of the training for learners and share an anecdote of an action that was taken because of prior feedback. On the last day of class, the instructor should re-communicate the importance of completing the surveys before providing learners with the URL, QR Code, or emailed link.

Time to complete the surveys should be provided midway through the last day or at the last break of class rather than at the very end of class when learners are anxious to leave. This allows the learners to provide thoughtful feedback without feeling rushed.

If your organization is utilizing email invitations, you can customize the subject and body of the email message to make it unique to your organization. Communicate the importance of the respondents’ feedback in the message and set expectations for the time needed to complete.

Explorance Metrics That Matter™ can also automate one or more reminder emails to be delivered to respondents who do not complete assigned surveys within a set number of days. While respondents may ignore the first email, they are less likely to take no action after receiving two or three. Furthermore, the emails can include a carbon copy to the student’s manager if enrollment and manager information are uploaded or passed via your LMS integration.

Surveys for eLearning

For eLearning, common delivery methods for the survey include:

• Send survey invitation emails to the learner.
• Embed a link to the survey in the eLearning content or learner portal.
• Embed a redirect link to navigate the learner to the survey upon completion.
• Combination of the above, such that the student may complete the survey within the learning experience but will be prompted to do so in an email if they do not.

Utilize Communication

To boost response rates across an organization, the main idea that needs to be impressed upon the learners is that it is important for them to complete the surveys. There are a few different approaches to this that should be used in conjunction with one another:

• When learners are in class, the instructor should communicate at the start, middle, and end of class that they will be receiving a survey. The instructor should explain how it will be delivered and why it is vital that the learners complete it.
• Provide another in-class communication in the form of a handout that details the information regarding the training surveys.
• If surveys are being accessed through automated email messages, customize the email messages to make them unique to your organization and communicate your message for why the learners should complete the survey. Use and customize automated reminder emails to go to learners who ignore the first survey invitation email.
• Share the results of the data collected back to learners so that they understand that the few minutes they take to complete the survey are time well spent. When sharing the survey data with learners, be sure to tie it back to job impact and business results. Ideally, you can show the improvement over time in these areas and give the learners credit for helping you make those improvements.
• Gain buy-in and support from leadership. If learners are told to complete the surveys not just by the training organization but also by their managers, response rates are likely to increase.
• Spur competition among instructors or business units by publishing comparison response rates and rewarding top performers. Instructors are more likely to communicate the importance of completing the survey if they know that they are accountable for the class response rate. Leaders of business units can be motivated by viewing their group’s ranking among the others.

Explorance Metrics That Matter™ for Actionable Insights

Explorance Metrics That Matter™ SmartSheets enables organizations to quickly implement a robust, credible, automated, action-oriented approach to learning analytics. They enable organizations to stand upon decades of research and refinement with the confidence that the measures are valid and proven. While other organization use their “smile sheets” and have endless debates over the wording of questions, organizations that adopt the automated SmartSheets of MTM can redirect their energies toward acting: improving the business impact of their learning programs and communicating forecasts and results to their business stakeholders.