Introduction and Background
Artificial intelligence has moved out of the research laboratory and into the everyday infrastructure of Australian health services. Deterioration prediction models now sit inside electronic medical records in tertiary hospitals, imaging triage algorithms reorder radiology worklists, and natural language tools draft clinical documentation for review. Nurses are the largest group of end users of these systems; workforce reporting shows that registered nurses constitute the single biggest registered health profession in Australia and are present at almost every point where an algorithmic recommendation meets a patient (Australian Institute of Health and Welfare [AIHW], 2024). Whatever an algorithm produces, it is usually a nurse who notices it first, escalates it, or quietly disregards it.
The policy environment assumes that this workforce is ready. The National Nursing and Midwifery Digital Health Capability Framework sets out expectations for digital practice across five domains, from digital professionalism to the use of data and information (Australian Digital Health Agency [ADHA], 2020). The Nursing and Midwifery Board of Australia (NMBA, 2016) requires registered nurses to think critically, to analyse the evidence informing their practice, and to accept accountability for their decisions, obligations that apply with equal force when part of the reasoning has been performed by a machine. The Tertiary Education Quality and Standards Agency (TEQSA, 2023), meanwhile, has asked higher education providers to respond deliberately to generative artificial intelligence rather than to absorb it by default.
Sitting beneath these expectations is a capability that has only recently been named. Artificial intelligence literacy refers to the ability to recognise where artificial intelligence is operating, to evaluate its outputs critically, to use such tools appropriately, and to reason about their ethical and safety implications (Long & Magerko, 2020; Ng et al., 2021). It is not a proxy for computer science skill, nor is it the same as general digital literacy, which concerns the competent use of devices and records. A nursing student may be entirely fluent in an electronic medication record and still have no framework for deciding whether a sepsis alert deserves escalation or scepticism.
Problem Statement
Three problems converge. First, the capability frameworks that govern Australian nursing practice were written before generative artificial intelligence entered clinical and academic settings, and they contain no dimension that specifically addresses algorithmic reasoning, model limitations or automation bias (ADHA, 2020). Second, the international evidence on nursing students consistently reports enthusiasm paired with low confidence and weak ethical reasoning, but the studies use heterogeneous and often unvalidated instruments, making pooling unreliable (Buchanan et al., 2020). Third, and most importantly for curriculum design, there is no baseline measurement of artificial intelligence literacy among students enrolled in Australian Bachelor of Nursing programs. Schools of nursing are therefore being asked to prepare graduates for algorithmically mediated practice without knowing what those graduates currently understand.
The consequences of leaving this unmeasured are clinical rather than merely academic. Students who defer uncritically to a model risk automation bias, in which an incorrect recommendation is accepted because it carries the authority of the system, while students who dismiss algorithmic output entirely lose the safety benefit the tool was purchased to deliver. Both errors express the same underlying deficit, and both are addressable through curriculum once that deficit is described with precision.
Research Aim and Questions
The aim of this study is to measure artificial intelligence literacy among students in Australian pre-registration nursing programs, to identify the individual, curricular and placement factors associated with it, and to explain the resulting patterns through the accounts of students themselves. Three research questions follow.
- What is the level and internal structure of artificial intelligence literacy among students enrolled in Australian Bachelor of Nursing programs?
- Which individual, curricular and clinical placement factors predict artificial intelligence literacy and perceived readiness for artificial intelligence enabled practice?
- How do nursing students describe learning about, using and trusting artificial intelligence enabled tools during clinical placement, and what curriculum gaps do they identify?
Condensed Literature Review
Artificial intelligence in clinical decision support
Clinical decision support has a long history, but the current generation differs in kind rather than degree. Rule based alerts encoded human logic that a clinician could inspect; contemporary models learn associations from large data sets and produce recommendations whose reasoning is frequently opaque to the user (Sutton et al., 2020). The recognised benefits, earlier detection of deterioration, reduced variation and relief from documentation burden, are accompanied by recognised harms, including alert fatigue, degraded performance when a model is deployed on an unfamiliar population, and automation bias (Sutton et al., 2020). An international think tank convened by the Nursing and Artificial Intelligence Leadership Collaborative concluded that nurses are largely absent from the governance of these systems and named nursing education as a priority for intervention (Ronquillo et al., 2021).
Digital health capability frameworks
Australia has invested substantially in defining digital capability for the nursing and midwifery workforce. The national framework describes expected capability across digital professionalism, leadership and advocacy, data and information quality, information enabled care, and technology (ADHA, 2020). Its strength is that it is written in the language of practice rather than of information technology; its limitation, for present purposes, is that it treats technology as a tool to be used competently rather than as a source of inference to be appraised. Nothing in the framework asks a nurse to consider whether a model was validated on a comparable population, or what should happen when an algorithmic recommendation conflicts with clinical judgement. That appraisal function is precisely what the NMBA (2016) standards for practice already require of registered nurses, which leaves an unresolved gap between two authoritative documents.
Student readiness and artificial intelligence literacy
A scoping review of the predicted influence of artificial intelligence on nursing found that most published work is speculative, that empirical studies of student capability are scarce, and that education is repeatedly nominated as the mechanism of preparation without being specified (Buchanan et al., 2020). Where student surveys exist, they report a recurring shape: awareness of artificial intelligence is moderate to high, confidence in using it is low, and ethical reasoning about privacy, bias and accountability is the weakest domain. Conceptual work has begun to give this structure, with Long and Magerko (2020) proposing competencies spanning recognition, evaluation and ethical reasoning, and Ng et al. (2021) synthesising the literature into four domains covering knowing, using, evaluating and ethical engagement. Higher education regulation has moved in parallel, with TEQSA (2023) advising Australian providers that assessment must be redesigned on the assumption that generative tools are available to students. Student use is therefore already running ahead of curriculum, which makes measurement urgent rather than merely interesting.
The synthesis of this literature identifies a specific and addressable gap. Conceptual models of artificial intelligence literacy exist but have not been operationalised for nursing; Australian capability frameworks exist but do not extend to algorithmic appraisal; and no Australian study has measured the construct in the pre-registration student population. This proposal addresses that gap directly.
Theoretical Framework
The study is guided by a conceptual model integrating three established bodies of theory. Artificial intelligence literacy is operationalised as a four dimensional construct following Long and Magerko (2020) and Ng et al. (2021). Its translation into behaviour is theorised through self-efficacy, on the basis that capability is expressed only when a person believes they can act on it (Bandura, 1997). Acceptance of the technology itself is modelled using the unified theory of acceptance and use of technology, which identifies performance expectancy, effort expectancy, social influence and facilitating conditions as determinants of intention and use (Venkatesh et al., 2003). Figure 1 illustrates the resulting model.
The model proposes that antecedent digital capability, curricular and placement exposure, and attitudinal beliefs about benefit and risk jointly shape artificial intelligence literacy; that literacy influences readiness for practice principally through self-efficacy; and that year level, program type and prior clinical information technology use moderate these relationships. Framing self-efficacy as a mediator rather than an outcome carries a direct curricular implication: teaching content alone is unlikely to change practice unless students are also given supervised opportunities to act on what they know.
Methodology
Design
An explanatory sequential mixed methods design will be used, in which a quantitative phase is followed by a qualitative phase that explains and elaborates the quantitative results (Creswell & Plano Clark, 2018). The design suits the aim because the first research question requires population level measurement while the third requires the interpretive depth that only student accounts can provide. Priority is given to the quantitative phase, and integration occurs at two points: the quantitative results determine who is invited to interview, and a joint display draws meta-inferences at the analysis stage.
Setting, sampling and sample size
Participants will be students enrolled in Bachelor of Nursing programs at six Australian universities across four states and one territory, selected to include metropolitan, regional and dual sector providers. Stratified sampling by year of enrolment will ensure comparable representation across the three years of the program. The target sample of approximately 300 is justified by power analysis for the planned hierarchical regression. For a model with 10 predictors, a medium effect size of f squared = 0.15, alpha = 0.05 and power = 0.80, the minimum required sample is 172. Adjusting for an anticipated 20 per cent rate of incomplete responses gives 172 / 0.80 = 215. The target is raised to 300 to permit stable subgroup comparison across year levels, yielding approximately 100 respondents per stratum.
Instrumentation
The survey instrument will combine adapted published scales with a small number of study specific items, requiring approximately 15 minutes to complete. Table 1 summarises the constructs, their operational definitions, source instruments, item counts, response formats and the research question each addresses.
Table 1: Constructs, source instruments and measures in the Phase 1 survey
| Construct | Operational definition | Source instrument (adapted) | Items | Response scale | Question |
|---|---|---|---|---|---|
| AI literacy: recognition | Identifying where artificial intelligence is operating in clinical systems | Ng et al. (2021) domain 1 | 6 | 5-point Likert | RQ1 |
| AI literacy: evaluation | Critical appraisal of model outputs, limitations and error risk | Long and Magerko (2020) | 6 | 5-point Likert | RQ1 |
| AI literacy: application | Appropriate use of AI enabled tools in clinical and academic tasks | ADHA (2020) domain 4 | 5 | 5-point Likert | RQ1 |
| AI literacy: ethics and safety | Reasoning about privacy, bias, consent and accountability | Ng et al. (2021); NMBA (2016) standards 1 and 6 | 6 | 5-point Likert | RQ1, RQ3 |
| Digital health capability | Self-rated capability across the national framework domains | ADHA (2020) | 10 | 5-point Likert | RQ2 |
| AI self-efficacy | Confidence to perform specified AI related clinical tasks | Bandura (1997) scale construction guide | 8 | 0-100 confidence | RQ2 |
| Performance and effort expectancy | Perceived usefulness and perceived ease of use | Venkatesh et al. (2003) | 8 | 7-point Likert | RQ2 |
| Perceived readiness | Preparedness for AI enabled practice on graduation | Study developed, expert panel reviewed | 5 | 5-point Likert | RQ2, RQ3 |
| Curriculum and placement exposure | Reported teaching and observed clinical use of AI tools | Study developed | 6 | Categorical and frequency | RQ2, RQ3 |
Content validity will be established through review by an expert panel of six academics and clinicians, using an item level content validity index threshold of 0.78 for retention. A pilot with 30 students will test comprehension and internal consistency, with items reducing scale reliability below an alpha of 0.70 revised or removed.
Phase 2 interviews
Fifteen semi-structured interviews of 40-60 minutes will be conducted with survey respondents who consent to further contact. Selection will follow an extreme case strategy, drawing from the highest and lowest quartiles of the composite literacy score, with maximum variation across year level, institution and placement type. The interview guide will probe how students first encountered artificial intelligence enabled tools, how they decided whether to trust an output, what supervisors modelled, and where they judge the curriculum to have failed them. Interviews will be conducted by video, recorded and transcribed verbatim.
Analysis plan
Quantitative analysis will proceed in three stages. Descriptive statistics will characterise literacy across the four dimensions and by year level. Confirmatory factor analysis will test the hypothesised four factor structure, with model fit assessed against conventional thresholds for the comparative fit index and the root mean square error of approximation, and reliability reported using McDonald omega. Hierarchical multiple regression will then test predictors of perceived readiness, entering demographic variables at step one, curricular and placement exposure at step two, and literacy dimensions and self-efficacy at step three, with mediation tested using bootstrapped indirect effects. Qualitative data will be analysed using reflexive thematic analysis (Braun & Clarke, 2021). Integration will use a joint display in which each quantitative finding is placed alongside the qualitative evidence that confirms, explains or contradicts it.
Ethical Considerations
Approval will be sought from the lead institution’s Human Research Ethics Committee, with reciprocal approval or site authorisation obtained from each participating university before recruitment commences. The study will be conducted in accordance with the National Statement on Ethical Conduct in Human Research, with particular attention to the values of research merit, justice, beneficence and respect (National Health and Medical Research Council [NHMRC], 2023). Nursing students are not a vulnerable population in the technical sense, but they are in a dependent relationship with the academics who teach and assess them. Recruitment will therefore be conducted through student communication channels rather than by unit coordinators, participation will be unrelated to assessment, and no member of the research team will interview a student they currently teach or assess.
Written informed consent will be obtained electronically before the survey and again before interview, supported by a plain language participant information statement setting out the purpose, the voluntary nature of participation, the right to withdraw before data analysis, and the absence of any consequence for declining. Survey responses will be collected without direct identifiers, with a separate linkage file held for those consenting to interview. Recordings and transcripts will be de-identified, participants assigned codes, and data stored on the university’s secure research storage for five years after publication. No patient data will be collected at any stage, and students will be reminded not to disclose identifiable patient information during interview.
Project Timeline
The project is scheduled across 12 months, with overlapping phases to allow ethics processes and instrument development to proceed in parallel. Table 2 sets out the phases, activities, week ranges and deliverables.
Table 2: Project timeline across 12 months (52 weeks)
| Phase | Activity | Weeks | Deliverable |
|---|---|---|---|
| 1. Establishment | Protocol finalisation, instrument adaptation, expert panel content validity review | 1-6 | Final protocol and reviewed item pool |
| 1. Governance | HREC application, response to conditions, site authorisation at six universities | 5-12 | Ethics approval and site agreements |
| 2. Pilot | Pilot survey with 30 students, reliability and comprehension testing | 13-16 | Revised instrument and reliability report |
| 2. Survey | Recruitment and Phase 1 data collection across participating programs | 17-28 | Cleaned data set, n approximately 300 |
| 3. Quantitative analysis | Descriptives, confirmatory factor analysis, hierarchical regression, mediation testing | 27-34 | Quantitative results and interview sampling frame |
| 4. Interviews | Fifteen semi-structured interviews, transcription and de-identification | 33-42 | Transcript corpus |
| 4. Qualitative analysis | Reflexive thematic analysis, member reflection, audit trail | 41-46 | Final theme structure |
| 5. Integration | Joint display construction, meta-inference, curriculum implications | 45-48 | Integrated findings chapter |
| 5. Dissemination | Thesis completion, manuscript preparation, reports to participating schools | 47-52 | Submitted thesis and one manuscript |
Significance and Expected Contribution
The study offers three contributions. Empirically, it will produce the first Australian baseline measurement of artificial intelligence literacy among pre-registration nursing students, disaggregated by year level and dimension, allowing schools of nursing to target teaching at the weakest domain rather than adding generic technology content. Theoretically, it operationalises a construct that is currently conceptual and tests whether self-efficacy mediates the relationship between literacy and readiness, a proposition that bears directly on whether curriculum reform should emphasise content or supervised practice. Practically, it produces evidence relevant to the next revision of the national digital capability framework, which currently lacks an explicit algorithmic appraisal dimension (ADHA, 2020), and to providers responding to regulatory expectations concerning generative artificial intelligence (TEQSA, 2023).
The contribution is also anticipatory. Graduates who can evaluate an algorithmic recommendation, and who can articulate why a model may be unreliable for the patient in front of them, are more likely to participate meaningfully in the governance of systems from which nurses have so far been largely absent (Ronquillo et al., 2021). The professional standard of accountability already assumes such judgement (NMBA, 2016); this study asks whether the education system is producing it.
Limitations
Several limitations are acknowledged at the design stage. Self-report measures of literacy are vulnerable to social desirability and to the well documented tendency for those with least knowledge to overestimate their capability; scenario based evaluation items partially offset this, but objective performance testing would be stronger and is proposed as future work. Recruitment from six universities that agree to participate introduces selection bias, since institutions with existing digital health teaching are more likely to consent, which may inflate estimates. The cross-sectional design establishes association rather than causation, and cannot determine whether curriculum exposure produces literacy or whether interested students seek out such units. Finally, because the pace of technological change dates specific tool references quickly, the instrument measures appraisal capability rather than familiarity with named products.
Conclusion
Australian nursing students are entering a workforce in which algorithmic recommendations are already embedded in the systems they will use on their first shift. The regulatory architecture assumes a capability that has never been measured in this population, and the national framework does not yet name it. This proposal sets out a mixed methods study that will measure artificial intelligence literacy across approximately 300 students in six Australian programs, explain the resulting patterns through 15 interviews, and translate the findings into curriculum recommendations. The study is feasible within 12 months, ethically defensible under the National Statement, and addresses a gap that will only widen while it remains undescribed.
References
Australian Digital Health Agency. (2020). National nursing and midwifery digital health capability framework.
Australian Institute of Health and Welfare. (2024). Health workforce.
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
Braun, V., & Clarke, V. (2021). Thematic analysis: A practical guide. SAGE.
Buchanan, C., Howitt, M. L., Wilson, R., Booth, R. G., Risling, T., & Bamford, M. (2020). Predicted influences of artificial intelligence on the nursing profession: Scoping review. JMIR Nursing, 3(1), Article e23939.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE.
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-16). Association for Computing Machinery.
National Health and Medical Research Council. (2023). National statement on ethical conduct in human research.
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, Article 100041.
Nursing and Midwifery Board of Australia. (2016). Registered nurse standards for practice.
Ronquillo, C. E., Peltonen, L.-M., Pruinelli, L., Chu, C. H., Bakken, S., Beduschi, A., Cato, K., Hardiker, N., Junger, A., Michalowski, M., Nyrup, R., Rahimi, S., Reed, D. N., Salakoski, T., Salanterä, S., Walton, N., Weber, P., Wiegand, T., & Topaz, M. (2021). Artificial intelligence in nursing: Priorities and opportunities from an international invitational think-tank of the Nursing and Artificial Intelligence Leadership Collaborative. Journal of Advanced Nursing, 77(9), 3707-3717.
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. npj Digital Medicine, 3, Article 17.
Tertiary Education Quality and Standards Agency. (2023). Assessment reform for the age of artificial intelligence.
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.