In 2011, the online education industry catered to around 300,000 consumers. In 2021, it served 220 million, thanks in part to increased enrollment during the COVID-19 pandemic. Traditional universities and institutions are increasingly adopting hybrid course formats. For example, the number of full-time online MBA students surpassed in-person MBA students for the first time in the 2020-21 academic year.
Today, online educational platforms like Coursera and edX offer a range of flexible course content, but these firms are faced with a tricky question: Should they release content through a scheduled format that resembles a traditional university course with a subset of lectures and quizzes available at the start of each week, or should they follow in the footsteps of Netflix and Hulu with an on-demand release strategy where all the material is immediately available upon registration?
Exploring this question in a new Journal of Marketing study, we find that the choice of format for content release not only impacts overall user engagement and firm revenue but also user performance and learning outcomes.
Our team studied over 67,000 users taking an introductory marketing course on Coursera consisting of 32 short lecture videos and four quizzes. Our study took advantage of a natural experiment policy change where the platform switched the course from a scheduled format to an on-demand release format while keeping the actual content the same.
The scheduled format closely resembled a traditional university course, with some of the study material available at the beginning of each week for four weeks. Under the on-demand format, all four weeks of content was made available upon registration. All users could take the course for free or opt into paying for a completion certificate, as a one-time fee in the scheduled format and a monthly subscription in the on-demand format.
More Users, Less Engagement
Our findings show that the switch to on-demand content doubled the percentage of paying users from 14% to 28%. Thus, the on-demand format was successful in increasing short-term firm revenue by bringing in more paying users. On the downside, the switch resulted in significantly lower lecture completion rates (three fewer lectures on average) and lower quiz performance (10% lower scores).
The on-demand format also negatively impacted downstream platform engagement. The marketing course was promoted in a “Business Foundations” set with three other courses on operations, accounting, and finance. Compared to users in the scheduled format, those in the on-demand format ended up taking one or two fewer additional courses six months after the focal marketing course.
Our analysis of user activity reveals two new learning patterns:
- First, a subset (13%) of users in the on-demand format continued to return and take quizzes well beyond the recommended 4-week course period. The greater flexibility in the on-demand content release and payment structure likely enabled these users to “stretch out” their consumption.
- Second, the on-demand format increased the practice of binging, with user activity being clumped together (i.e., more binging) as compared to being evenly spaced out (i.e., less binging). In the scheduled format, binging was negatively related to course performance, which is consistent with the intuition that binging reflects procrastination or cramming. However, in the on-demand format, binging was positively related to performance, suggesting that on-demand users may binge as a form of strategic time management by setting aside time to consume in spurts.
Real-World Implications
Our study offers vital lessons for chief marketing officers in the online education space:
- The switch to the on-demand format attracted a set of users who were more likely to pay but were less engaged in the course. Our results show that on-demand content is potentially helpful at bringing in a new user segment or expanding the current user base, similar to universities offering concurrent hybrid MBAs that cater to busy students with full-time jobs. Managers must consider the trade-off between offering structure versus flexibility and may even consider offering different content release options simultaneously, but at different price points by emphasizing their unique features.
- Platforms may need to adapt their content to account for users who binge on content and others who space it out over time. For example, firms can include more recaps or reviews to reduce frustration resulting from users forgetting content. It may even be a viable strategy to embrace the prevalence of binging among users by highlighting or designing sets of lectures that are “bingeable” versus more modular.
- Many online platforms offer episodic content that may be released in installments and thus need to make decisions regarding the content release format. Our work provides insights that help managers anticipate the potential consequences of such decisions. On-demand content offers clear short-term benefits in terms of increased revenue but potentially long-term costs in terms of decreased engagement and new challenges in maintaining user engagement.
Read the Full Study for Complete Details
From: Joy Lu, Eric T. Bradlow, and J. Wesley Hutchinson, “More Likely to Pay but Less Engaged: The Effects of Switching Online Courses from Scheduled to On-Demand Release on User Behavior,” Journal of Marketing.
Go to the Journal of Marketing
Joy Lu is Assistant Professor of Marketing, Carnegie Mellon University, USA.
Eric T. Bradlow is K.P. Chao Professor, Professor of Marketing, Statistics, Economics and Education, and Vice-Dean of Analytics, University of Pennsylvania, USA.
J. Wesley Hutchinson is Stephen J. Heyman Professor Emeritus of Marketing, University of Pennsylvania, USA.