Webinar Recording: Beyond AI Adoption: What University Leaders Should Measure Before They Scale
University leaders should track AI’s impact on learning outcomes, academic integrity, and equitable access to AI tools and support.
Tina Austin has spent four years working on AI adoption across the University of California system. In this conversation, she breaks down what the research supports and what implementation looks like in practice. This conversation offers higher education leaders an examined overview to help them make decisions about choosing the best AI for college students before they commit.
AI decisions in higher education are moving faster than most institutions can carefully evaluate them.
A new study makes a sweeping claim about learning. A vendor arrives with evidence of its own. Faculty need guidance. Leadership needs a policy. Whatever gets put in place has to survive another semester of rapidly changing technology.
In this conversation, Tina Austin, creator of the UnBlooms™ framework and Director for Public Interest Technology Pathways and DataX at UCLA, joins Edvisor's Director of Product & Learning, GP LeBourdais, PhD, to discuss what university leaders should actually be paying attention to as AI adoption moves from experimentation toward implementation.
Tina launched a UC-wide AI community of practice in 2022, which has reached nearly 1,800 faculty. Her work offers a view into what happens after an institution moves beyond the question of whether faculty should use AI and starts asking how adoption can actually work at scale.
Key Takeaways From the Conversation
“What we’re trying to focus on is not so much the models, but really the human in the middle. At the end of the day, we’re here to have better, more discerning humans graduate.”
-Tina Austin, Director of AI Education & Learning Systems for Public Interest Technology at UCLA
Key takeaways
1. AI literacy is about judgment, not proficiency with one product
Students should learn to question AI outputs, identify assumptions, verify claims, and notice what the model may have omitted. These capabilities remain relevant even as institutions switch between ChatGPT, Claude, Gemini, NotebookLM, and other tools (~14:25-17:45 and 14:30-15:05)
2. Knowing when not to use AI is an essential AI skill
Tina's UnBlooms™ framework asks learners to establish their own reasoning before using AI, interrogate rather than simply accept its output, and demonstrate what knowledge and judgment remain independently transferable afterward.
The desired outcome is not compulsory adoption. It is the ability to delegate appropriately, remain transparent, transfer knowledge independently, or resist AI when it does not serve the task (~15:00-18:00)
3. Assessment redesign must be discipline-specific
No universal AI policy or assessment format will work across nursing, physics, humanities, mathematics, and other disciplines. Institutions can provide principles and guardrails, but faculty must retain the freedom to determine what valid learning and measurement look like in their courses. (~21:00-27:00). Tina argues that institutions should measure what learners can explain, defend, transfer, and reproduce independently after AI support is removed-not simply the quality of what they produce while using it.
4. Faculty skepticism should be treated as professional judgment
Faculty are being pressured by conflicting messages: adopt AI immediately or risk falling behind. They have an additional obligation to preserve academic integrity. Tina argues that institutions should validate reasonable skepticism, provide paid professional-development time, and then allow selective use based on whether AI genuinely serves the learning outcome. (~27:20-32:30 and 28:00-29:35)
5. Universities should move faculty away from policing and toward redesign
AI detection and scrutiny of student writing can push instructors into the role of police officers instead of educators. Tina recommends redesigning activities so students critique AI, identify its errors and biases, and engage with it from a position of intellectual authority. (~22:00-24:50 and 27:45-29:15)
- Link to Unblooms website: https://unblooms.gainable.ai/
- Link to Amazon Unblooms workbook
- Link to Tina’s resources.
6. Viral AI research claims need methodological scrutiny
Headlines often flatten complicated findings into claims that AI either definitively improves or harms learning. Tina recommends examining the following:
- Who funded the study
- What was measured
- Whether there was a meaningful control or baseline
- The incentives of participants
- The assessment conditions
- Whether the research captures durable learning rather than a snapshot. (~33:00-40:20)
7. Institutions should judge AI vendors by learning outcomes, not hype or price
Schools face an overwhelming number of vendors promising effortless solutions. Tina’s standard is straightforward: if a tool does not improve the learning outcome or solve the educational problem, institutions should not adopt it, regardless of whether it is expensive, inexpensive, or easy to deploy (~46:20-48:50)
8. Equity will not be solved by giving everyone the same tool
Open or lower-cost models may improve access, privacy, language support, and institutional control. But an “open” model is not automatically free or equitable. Equity also requires infrastructure, faculty support, independent standards, institutional collaboration, and the ability to judge when AI is appropriate. (~43:00-52:00)
9. The AI products that endure will strengthen human agency
Tina predicts that the successful tools and labs will be those designed around learning, student agency, human judgment, and improved cognitive outcomes—not products that simply enable users to “press a button.” (~54:20–55:35)
Try Our Free Research-Backed Assessment Redesigner
About Tina Austin
Tina R. Austin helps educators rethink what counts as evidence of learning in the age of AI. She serves as Director for Public Interest Technology Pathways at UCLA and is the creator of UnBlooms™, a framework for redesigning assessment and measuring metacognitive awareness. Her research examines the difference between AI-assisted performance and durable, independently transferable learning.
She also designs longitudinal studies with schools evaluating learning with AI and an assessment subgroup within Oxford University’s AIEOU examining assessment across the Global North and Global South. In 2022, she launched a UC-wide AI community of practice that has reached nearly 1,800 faculty. She has also been invited to speak at OpenAI headquarters and Cambridge University on critical thinking with AI across disciplines and how to redesign assessment.
Austin is a selected member of the California Public Schools: Artificial Intelligence (AI) Working Group and contributes to statewide AI-literacy language connected to California Senate Bill 1288.
About GP LeBourdais
GP LeBourdais, PhD, is Director of Product & Learning at Edvisor, where he leads research and product development focused on AI-resilient teaching and learning.
Frequently Asked Questions
What does AI literacy actually mean in higher education?
AI literacy should go beyond knowing how to use a specific university AI tool. Tina emphasizes discernment: questioning outputs, identifying assumptions, verifying claims, recognizing uncertainty, and knowing when AI should or should not be used.
Should universities use AI detection tools?
Pushing faculty into a policing role through AI detection is not recommended. A stronger approach is to redesign assessments so that student thinking, judgment, and learning are more visible throughout the process.
How can universities evaluate AI tools and vendors?
Start with the educational problem, not the product. If a higher education AI tool does not improve learning outcomes or solve a meaningful instructional need, institutions should question whether it belongs in the classroom, regardless of cost or ease of implementation.
Is AI helping students with learning or is it doing more harm?
The webinar discussion cautions against simple claims in either direction. Tina recommends looking closely at study methodology, participant incentives, controls, assessment design, funding, and whether that research about AI in higher education measures durable learning rather than short-term performance.
