Research Insights #4: Student-Focused Studies Part 3
Exploring the potential impact of generative AI usage on student learning
Last time we explored student perceptions about generative AI and this time, we’re looking at the impact of generative AI on students with 3 more recent studies.
Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., & Siemens, G. (2024). Impact of AI assistance on student agency. Computers & Education, 210, 104967.
Generative AI summary
Part 1 - Study's Findings and Educational Implications
The study investigates the impact of AI assistance on student agency within the context of peer feedback. Key findings reveal that while AI assistance can improve the quality of student feedback by providing personalized and targeted suggestions, its removal leads to a reliance on less effective self-regulated strategies. Specifically, the study found that students relied on AI assistance without significantly improving their ability to provide quality feedback independently. The implications for higher education suggest that while AI can enhance learning experiences, educators must carefully integrate these technologies to foster student agency and self-regulated learning skills.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
The study employed a randomized controlled experiment design, utilizing AI assistance and self-monitoring checklists in peer review processes to explore their effects on student agency. The theoretical framework was grounded in self-regulated learning (SRL) principles, emphasizing the importance of student agency. This approach underscores the necessity for educational technologies to support students in becoming active agents in their learning process, rather than passive recipients of AI-generated feedback.
Part 3 - Gaps, Challenges, and Limitations
One limitation identified is the potential dependency on AI assistance, which could hinder the development of independent learning strategies. Additionally, the study highlights the challenge of integrating AI tools in a way that supports rather than replaces self-regulated learning processes. Educational developers must consider these limitations when designing AI-supported learning environments, ensuring that these technologies serve as complements to, rather than substitutes for, effective learning strategies.
Part 4 - Application to Educational Development
For educational developers and faculty, the study's insights emphasize the need to design AI tools that promote rather than inhibit student agency. Implementing AI-assisted learning environments should involve strategies that encourage students to engage critically with the feedback provided, such as through self-monitoring checklists or scaffolding that fades over time. This approach could help students internalize feedback and apply self-regulated learning strategies more effectively, enhancing their overall learning outcomes.
By focusing on these areas, educational developers can better leverage AI technologies to support student learning while also promoting critical skills such as self-regulation and agency.
My Commentary
Content thought: This study's lead consideration that students are using the tool without being improved by others is one that I know lots of folks will want to jump on. I do think there are areas to be concerned about that as an issue. However, it does not feel insurmountable with this technology. One could have students do a peer review on their own first and then one that is AI-supported. Then compare the differences in what was drawn out. Some might find that to be too much work on the student who is doing the peer reviewing of another student's work. But that would actually still be grounded in the learning since peer reviewing isn't just a process of reducing work for the instructor but a means to learn and teach for the student doing the peer review.
Output thought: A limitation in this output by generative AI is that it is unclear if the impact is on the student providing the peer review OR the student receiving the peer review. I've read Part 1 a few times and still can't tell.
Habib, S., Vogel, T., Anli, X., & Thorne, E. (2024). How does generative artificial intelligence impact student creativity?. Journal of Creativity, 34(1), 100072.
Generative AI summary
Part 1 - Study's Findings and Educational Implications
The study examined the impact of generative AI, specifically ChatGPT-3, on student creativity, using a mixed methods approach involving the Alternative Uses Task (AUT) to measure creativity dimensions like flexibility, fluency, elaboration, and originality. Key findings indicate that while AI can significantly support creative thinking by enhancing idea generation and elaboration, it also poses negative impacts on creativity and creative confidence, suggesting a nuanced integration of AI into creative education. These findings underscore the potential and challenges of leveraging AI in educational settings, highlighting the need for careful curriculum design that fosters creative engagement while mitigating risks of overreliance on AI for creative processes.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
The study utilized a mixed methods approach, incorporating both qualitative and quantitative data from the AUT test to explore the effects of ChatGPT-3 on divergent thinking. This methodological choice allowed for a comprehensive assessment of AI's impact across different creativity dimensions, providing a balanced view of its potential to enhance student creativity when used appropriately. Theoretical perspectives on creativity emphasize the balance between divergent and convergent thinking, suggesting that effective education strategies should nurture both aspects. By integrating AI tools like ChatGPT-3 into creative tasks, educators can potentially enrich the creative process, though the study also cautions against the uncritical use of AI, highlighting the importance of maintaining a focus on developing students' inherent creative abilities.
Part 3 - Gaps, Challenges, and Limitations
The study acknowledges limitations including the self-selection bias of participants and the influence of course instruction on creativity outcomes, pointing to the need for further research across diverse educational settings. Challenges include the potential for AI to stifle individual creative thinking and the ethical considerations around originality and intellectual property. These issues underline the complexity of integrating AI into creative education and call for ongoing investigation into how AI can be used responsibly to augment, rather than replace, human creativity.
Part 4 - Application to Educational Development
Educational developers and faculty can leverage insights from this study to enhance creative teaching and learning strategies. The findings suggest incorporating AI as a tool for idea generation and elaboration while also emphasizing the development of students' critical and creative thinking skills. For example, educators might design assignments that require students to both use AI for brainstorming and critically evaluate the novelty and feasibility of the ideas generated. This approach can help balance the benefits of AI with the cultivation of independent creative thought, preparing students to navigate the complexities of a technology-infused creative landscape.
My Commentary
Content thought: This finding connects with the one above in that idea of needing to find the right balance of tool use in this process. And I think that's right. We'll be in this space, probably for decades (we're still in the balance of tool space with lots of other technologies that are part of or embedded in education--the internet, laptops, mobile devices, collaborative tools, etc). I do find the suggestion of to both brainstorm with generative AI and critically review its outputs makes sense. Though this might also be context dependent.
Output thought: I'll continue to tweak the question because it's still not clear exactly what they did in the study to get the results. Sometimes, the tone bias in ChatGPT makes me chuckle. The last line of the above output feels like I could hear it being uttered by a AI-generatived voice (Alexa, Siri) with its positive and enthusiastic-tinged tone.
Byrne, A., & Mooney, P. Exploring the potential impact of ChatGPT on student learning activities.
Generative AI summary
Part 1 - Study's Findings and Educational Implications
The study explored the impact of ChatGPT on student learning, revealing broad student support for its use in education. Key findings include a preference among 67% of respondents for personalized tutoring, independent learning, and improved feedback mechanisms over ChatGPT supports, suggesting AI tools should complement traditional educational resources. This highlights the potential for AI to enhance personalized learning experiences, assist in developing critical thinking, and provide real-time feedback, thereby transforming the educational landscape.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
The research utilized a mixed-method approach, combining literature review and student surveys to understand attitudes towards ChatGPT in education. The methodology underscores the importance of integrating student perspectives in evaluating AI's educational impact. By focusing on recent literature and firsthand student feedback, the study leverages contemporary insights to assess AI's role in facilitating learning and teaching processes, emphasizing the need for a balanced integration of AI tools that complement traditional teaching methods.
Part 3 - Gaps, Challenges, and Limitations
The study acknowledges limitations, including a small and possibly non-representative sample size due to the timing of the survey. This highlights the need for broader and more inclusive research to fully understand AI's educational impact. Additionally, the rapid evolution of AI technologies presents a challenge in keeping educational practices up-to-date. Educational developers should consider these limitations, focusing on continuous evaluation and adaptation of AI tools in educational settings.
Part 4 - Application to Educational Development
The findings suggest that educational developers and faculty can leverage ChatGPT to support student learning by integrating it as a supplementary tool for personalized learning and feedback. Recommendations for educational development include the development of guidelines for ethical AI use, ensuring AI tools are used to enhance rather than replace human interactions, and adapting teaching methods to leverage AI's potential in addressing diverse learning needs and styles. This approach aligns with the overarching goal of enriching the educational experience while preserving the essential human elements of teaching and learning.
My Commentary
Content thought: The finding right out of the gate that 67% of respondents would be more interested in other supports than Generative AI supports feels like a data point that's easy to sling around but without real meaning. I don't doubt that those are the results but the larger context feels a bit overrepresented. This student was done probably mid-late 2023 (published in 2023). The technology is still young and new and it seems like the study didn't actually provide a richer context for students to learn and think about what this technology is. The frame of 67% weren’t interested misses the fact that in less than 1 year from its arrival, ChatGPT has intrigued 1/3 of learners enough to consider it in lieu of other supports. To be clear, I'm not advocating to go full throttle into AI learning supports. Rather, I'm more worried about how long a data point like this is relevant (never mind that it was a small sample). Yet, data points like these can stick around as zombie facts long after the context has change.
Output thought: I definitely need to solicit more detail about the nature of the study itself in these prompts. It makes me wonder if it will continue to be a changing set of questions because as one question feels right or is soliciting the right information, another will drop off.
Meta-commentary
Content thought: As with lots of research, it will continue to be a mix-bag. I think that's something we don't always remember with all teaching practices. To my knowledge, there's no learning practice, tool, pedagogy, etc that is effective with 100% of students. I don’t think we remember this enough in all of our work Even those famous "high impact practices" that every institution is deeply devoted to praising (but rarely significantly investing in) are not cure-alls. A significant portion of students won’t respond to these.
Some of this research I think is a really interesting early exploration of views and practices. I'm betting that this will continue to change into more favorable views, uses, and impacts of generative AI over the next 5-10 years as faculty and students get more comfortable with generative AI, they learn through practice what works well and what doesn't, and we can see a more integrated approach to thinking about generative AI than the scattershot way it's existed in the last year or so.
Output thought: I have retooled the questions once again (See appendix) and reduced them 3 three questions. I also noticed this past week, it seemed to not produce as robust outputs in content and detail as it had when I did the last one. It also makes me wonder if I have to enter things manually rather than as 3-4 part single prompts.
Appendix
Prompt #1
You are an expert in educational development, skilled in analyzing research literature like the attached document to extract details, concerns, and pertinent and impactful insights for application in education.
Conduct a comprehensive analysis of the attached document, applying your expertise as an educational developer.
Your output must be detailed, utilizing specific examples and insights from the attached document while also leveraging other research in the field.
Structure your response as follows:
Part 1 - Study's Findings and Educational Implications.
You must:
Explain and elaborate on the main findings of the study.
Explain the implications of these findings for the field of higher education, teaching and learning, citing specific examples from the document.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives:
You must:
Discuss the strategies, methodologies, and theoretical perspectives presented in the document and how they impact the document's content.
Explain how these strategies, methodologies, and theoretical perspectives are used to enhance the value of the document in relation to higher education, teaching, and learning. You must be specific in your explanation with particular references to the document.
Part 3 - Gaps, Challenges, and Limitations:
You must:
Identify and explain each gap, challenge, or limitation evident or absent in the document.
Elaborate on these gaps, challenges, or limitations for how educational developers should consider them in their work, using particular details from the article and other research.
Part 4 - Application to Educational Development:
You must:
Discuss how specific findings, insights, and conclusions of the document are relevant to educational developers and faculty.
Include how the recommendations in the document can be adapted or applied in the context of educational development.
Remember with each part, you must offer in-depth analysis and practical insights, helping educational developers assess the document's relevance and value to their field. That must include examples, applying ideas in other contexts, and relying on particular details. Your response must be as expansive as possible.
Prompt #2
You are an expert in educational development, skilled in analyzing research literature to extract pertinent and impactful insights for application in education.
You must assess the article according to the following categories, scoring each on a scale from 1 to 10 (10 being the highest). For each category, you must provide specific examples and detailed justifications for your scores:
Writing Intention:
Identify the type of writing (e.g., theoretical argument, empirical study, practical explanation, case study).
Determine the specific intended audience(s) of the piece.
Ease of Reading:
Analyze the use of complex terminology and assumptions about the reader's prior knowledge.
Evaluate how the structure and organization contribute to understanding.
Indicate the reader level that would find this article most accessible.
Clarity of Ideas:
Assess the straightforwardness of key points, including logical flow and explicitness.
Evaluate how well the article summarizes its main ideas.
Evidence:
Examine the types of supporting research and evidence used (qualitative, quantitative, mixed methods, primary sources, etc.).
Diverse Perspectives and Research:
Analyze the diversity in the article's approach, including interdisciplinary, global/local perspectives, intersectionality, and historical/contemporary lenses.
Assess the variety in terms of time of publication, authorship diversity, publication type, etc.
Direct Application of Ideas into Practice:
Evaluate the practicality of translating the article's discussions into teaching and learning contexts.
Remember to support each category with detailed, example-driven explanations to substantiate your assessments.
Prompt #3
Provide a bulleted list of 5-8 keywords (no explanations). The keywords must draw from the document be relevant and beneficial to help educational developers categorize this article.
AI+Edu=Simplified by Lance Eaton is licensed under Attribution-ShareAlike 4.0 International
