Research Proposal – Student Engagement and Retention in Online Higher Education

Introduction and Background

Fully online delivery has moved from the margins of Australian higher education to its mainstream, accelerated by the remote pivot of 2020 and sustained afterwards because it suits the students the sector has been asked to recruit. The Australian Universities Accord Panel (2024) set a target of lifting tertiary attainment substantially by mid-century and made clear that the growth must come from groups historically under-represented in higher education: students from regional and remote areas, students from low socio-economic backgrounds, First Nations students, and adults returning to study while working and caring. Online delivery is the principal mechanism through which that expansion is expected to occur, because it removes the need to relocate or attend campus at fixed times.

The difficulty is that the mode carrying the sector’s equity ambitions also has the weakest completion record. Departmental cohort analyses show that domestic bachelor students commencing in external mode withdraw at roughly twice the rate of their internal counterparts within the first year, with adjusted first-year attrition near 25% compared with around 12% for on-campus students (Australian Government Department of Education, 2024). The gap widens for part-time enrolments, which dominate online cohorts. Australian Bureau of Statistics (2023) data on education and work confirm that part-time students are disproportionately aged 25 and over and are combining study with substantial paid employment, a pattern that concentrates withdrawal risk precisely among the groups the Accord seeks to attract.

Regulatory expectations have tightened in parallel. The Higher Education Standards Framework (Threshold Standards) 2021 requires providers to monitor progression by equity grouping, to identify students at risk of not progressing, and to make timely support available to them (Tertiary Education Quality and Standards Agency [TEQSA], 2022). Universities have responded with learning analytics dashboards that aggregate learning management system (LMS) activity and generate early alerts when engagement drops. Whether those systems reduce withdrawal, and for whom, remains an open question in the Australian setting.

Problem Statement

Three limitations characterise current knowledge. First, most retention research in online higher education comes from North American institutions with different funding models, census rules and demographics, so its transferability to Australian providers is uncertain. Second, studies typically use either behavioural trace data or self-reported psychological measures, rarely both, so the field can describe what disengaged students do without explaining why. Third, early-alert systems have been evaluated largely on operational metrics such as alert volume and contact rates rather than on their effect on withdrawal, and their differential effects across student groups have received little attention despite documented risks of algorithmic bias (Prinsloo & Slade, 2017).

The problem is practical as well as scholarly. Australian universities are investing in analytics platforms and retention teams without a clear account of which engagement signals matter, when they become informative, and whether the interventions they trigger change outcomes for regional and mature-age students specifically. Institutional effort therefore risks being directed at the students easiest to detect rather than those most likely to leave.

Aim and Research Questions

The aim of this study is to explain how engagement, belonging and institutional early-alert responses jointly shape withdrawal among students in fully online units at an Australian public university, and to identify when intervention is most likely to be effective. Three research questions follow:

1. Which behavioural engagement indicators, drawn from learning management system and student information system records, predict withdrawal from fully online units, and how does predictive accuracy change across the teaching period?
2. To what extent do teaching presence, social presence and sense of belonging account for persistence intentions beyond behavioural engagement and student background characteristics?
3. How do regional and mature-age online students describe the conditions, including contact generated by early-alert systems, that sustain or erode their commitment to completing a unit?

Literature Review

Integration, departure and their online reinterpretation

Tinto’s model of student departure remains the organising theory of retention research, proposing that persistence follows from academic and social integration into the institution, and that departure reflects insufficient integration rather than individual deficit. In later work, Tinto (2017) reframed the model around the student’s own perspective, arguing that self-efficacy, perceived curricular relevance and a sense of belonging are the proximate determinants of persistence. This matters for online study, where the campus rituals through which social integration was originally theorised are absent. Kahu and Nelson (2018), writing from an Australasian standpoint, offer a compatible account in which engagement is generated at an educational interface where institutional design and student circumstance meet, treating engagement as a mutable property of the interface rather than a fixed attribute of the student.

The community of inquiry framework

Where integration theory describes the outcome, the community of inquiry (CoI) framework describes the mechanism available to online educators. Garrison (2017) specifies three interdependent elements: teaching presence (the design, facilitation and direction of learning), social presence (projecting oneself as a person within a mediated environment) and cognitive presence (constructing meaning through sustained discourse). Empirically, teaching presence exerts the strongest influence because it enables the other two, and it supplies validated subscales pointing to design decisions an institution can change, such as announcement cadence, feedback turnaround and structured discussion.

Learning analytics and early-alert systems

Learning analytics offers continuous, unobtrusive measurement of participation, and traces such as login frequency, resource access, forum activity and submission timing have been used to build risk models. Gašević et al. (2015) caution, however, that generic models transfer poorly between units because instructional design determines what a trace signifies; forum inactivity means something different in a unit that assesses discussion than in one that does not. Australian evaluations report that early assessment non-submission is a stronger, more stable predictor than login counts, and that models built without discipline-level calibration generate false-positive volumes that exhaust the staff capacity available to respond.

Belonging in mediated environments

Belonging supplies the affective link between engagement and persistence. Stone and O’Shea (2019), reporting on older, first-in-family online students in Australia, found that perceptions of being known by teaching staff, and of the university expecting them to succeed, distinguished those who continued from those who left, often more powerfully than academic preparedness. Their participants described withdrawal as a quiet drift rather than a decision, suggesting that the interval between disengagement and formal withdrawal is where intervention is feasible. This literature is largely qualitative and rarely connected to trace data.

Conceptual Framework

The framework synthesises these bodies of work into a testable model, illustrated in Figure 1. Student inputs, comprising entry pathway, study load, remoteness and age, shape opportunities for academic integration and exposure to teaching and social presence, which in turn generate a sense of belonging, proposed as the proximal mediator of persistence. Behavioural traces captured by the LMS operationalise engagement and feed the early-alert system, whose outreach loops back into teaching presence when staff make contact. Institutional context, including the monitoring obligations under the Threshold Standards, conditions the whole system.

Research Design and Methodology

Design

An explanatory sequential mixed methods design will be used (Creswell & Plano Clark, 2018). A quantitative phase will model withdrawal using institutional records and a mid-semester survey; a qualitative phase will then explain the patterns through interviews with purposively selected respondents. Integration will occur through purposive selection of interviewees from identified risk profiles and a joint display setting statistical predictors against interview themes.

Setting and sampling

The study will be conducted at a large Australian public university with substantial fully online undergraduate enrolments. The analytics dataset will comprise approximately 4,000 unit enrolments across 16 fully online units in two consecutive teaching periods, spanning business, education, health and information technology so that instructional design varies across the sample. All students in those units will be invited to the survey, targeting approximately 500 usable responses, a rate of 500 / 4,000 = 12.5% consistent with comparable institutional surveys. Twenty interviewees will be selected for maximum variation across remoteness, age band, alert status and eventual outcome.

Variables and measures

Table 1 sets out the constructs, their operational measures and their analytic role. Behavioural variables will be aggregated weekly rather than semester-wide so that the timing of disengagement, not only its occurrence, can be captured.

Table 1: Constructs, measures, data sources and analytic role

Note. Remoteness is classified under the Australian Statistical Geography Standard and socio-economic status under the Socio-Economic Indexes for Areas, both maintained by the Australian Bureau of Statistics.

Analysis

Quantitative analysis will proceed in three stages. Descriptive and bivariate analyses will profile withdrawal by subgroup. Binary logistic regression will then model withdrawal from cumulative engagement, background characteristics and survey constructs, with mediation of the presence variables through belonging tested using bootstrapped indirect effects. Finally, discrete-time survival analysis will model the hazard of withdrawal by teaching week, entering time-varying engagement as it accrues and identifying the weeks in which risk peaks. Statistical power is adequate: with an expected withdrawal rate of 12%, the events available are 0.12 x 4,000 = 480, giving 480 / 10 = 48 events per predictor for a ten-variable model, well above the conventional minimum of ten. Model discrimination will be reported as the area under the receiver operating characteristic curve at the end of weeks 2, 4, 6 and 8, to establish how early a usable signal emerges, and disaggregated by remoteness and age band to test for differential accuracy. Interviews will be analysed using reflexive thematic analysis (Braun & Clarke, 2022).

Ethical Considerations

Approval will be sought from the university’s Human Research Ethics Committee under the National Statement on Ethical Conduct in Human Research (National Health and Medical Research Council [NHMRC], 2023), with separate approval from the institutional data custodian for release of the records. Four issues require particular attention.

• Consent for analytics data. Students rarely appreciate the granularity of the traces their study generates. Rather than relying on enrolment-time terms of use, a plain-language participant information statement will describe which behavioural fields are extracted, with an opt-out for the analytics component alongside opt-in consent for the survey and interviews.
• Privacy and de-identification. The dataset constitutes personal information under the Privacy Act 1988 (Cth), and handling will follow the Australian Privacy Principles (Office of the Australian Information Commissioner, 2019). Records will be pseudonymised at extraction, held on university-managed infrastructure, and reported only in aggregate, with cell sizes below five suppressed to prevent re-identification.
• Algorithmic fairness. Because predictive models can encode structural disadvantage as individual risk, subgroup performance will be reported rather than only overall accuracy, and no model developed here will be deployed to contact students during the research period.
• Beneficence and dual role. The researcher holds no assessing role in the participating units. Should an interview disclose distress, participants will be referred to counselling services, and data will not be returned to teaching staff in identifiable form (Prinsloo & Slade, 2017).

Project Timeline

The project is scheduled over 18 months, with the sequencing constrained by the academic calendar: survey administration must fall in week 6 of a teaching period, and interviews must follow the release of end-of-period outcomes. Table 2 sets out the phases.

Table 2: Project timeline over 18 months

Significance

The study offers three contributions. Theoretically, it tests whether belonging mediates the relationship between community of inquiry presences and persistence in a fully online Australian cohort, extending an integration tradition into a mode that now carries much of the sector’s equity load. Methodologically, it joins behavioural trace data to validated psychological measures and student accounts within a single design, addressing a separation that has limited both literatures. Practically, the weekly hazard estimates will tell retention teams when outreach is worth resourcing, and the subgroup analysis will show whether current alert thresholds detect regional and mature-age students as reliably as their metropolitan, school-leaver peers. Given the monitoring obligations under the Threshold Standards and the participation targets set by the Accord Panel, evidence of this kind is directly usable by institutional planning units.

Limitations

Three limitations are acknowledged. The design is observational, so associations between early-alert contact and persistence cannot establish causation; propensity-score adjustment will reduce but not remove the resulting selection bias. The single-institution setting limits generalisability, although sampling across four disciplines and two teaching periods provides internal variation in instructional design. Finally, survey respondents are likely more engaged than non-respondents, which may attenuate observed belonging-withdrawal relationships; comparing the two groups on administrative variables will allow the direction of that bias to be estimated.

Conclusion

Online delivery is now the principal vehicle for widening participation in Australian higher education, yet its attrition record remains substantially worse than that of on-campus study, and the analytics systems built to address it have been evaluated more on their outputs than their outcomes. This proposal sets out a mixed methods study that models withdrawal week by week across approximately 4,000 enrolments, tests belonging as the mediating mechanism through a survey of approximately 500 students, and explains the patterns through interviews with regional and mature-age students. The intended result is a defensible account of when disengagement becomes detectable, for whom current systems work, and what teaching presence must supply if online cohorts are to complete at rates approaching those of their on-campus peers.

References

Australian Bureau of Statistics. (2023). Education and work, Australia. ABS.

Australian Government Department of Education. (2024). Completion rates of higher education students: Cohort analysis. Department of Education.

Australian Universities Accord Panel. (2024). Australian Universities Accord: Final report. Commonwealth of Australia.

Braun, V., & Clarke, V. (2022). Thematic analysis: A practical guide. Sage.

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Garrison, D. R. (2017). E-learning in the 21st century: A community of inquiry framework for research and practice (3rd ed.). Routledge.

Gašević, D., Dawson, S., & Siemens, G. (2015). Let us not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

Kahu, E. R., & Nelson, K. (2018). Student engagement in the educational interface: Understanding the mechanisms of student success. Higher Education Research and Development, 37(1), 58-71.

National Health and Medical Research Council. (2023). National statement on ethical conduct in human research. NHMRC.

Office of the Australian Information Commissioner. (2019). Australian privacy principles guidelines. OAIC.

Prinsloo, P., & Slade, S. (2017). Ethics and learning analytics: Charting the (un)charted. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (pp. 49-57). Society for Learning Analytics Research.

Stone, C., & O’Shea, S. (2019). Older, online and first: Recommendations for retention and success. Australasian Journal of Educational Technology, 35(1), 57-69.

Tertiary Education Quality and Standards Agency. (2022). Guidance note: Diversity and equity. TEQSA.

Tinto, V. (2017). Through the eyes of students. Journal of College Student Retention: Research, Theory and Practice, 19(3), 254-269.