Research Insights #9: Student-Focused Studies Part 6

More research about AI and students...

May 29, 2024

So it’s been a while since I used ChatGPT to explore the contents of research articles. The last one I did was on open educational resources and this current post is a continuation of a series I did on student-focused research on generative AI. Since then, there was been the release of ChatGPT-4o and I can see both from the speed of the results as well as the details that the outputs have changed.

A charcoal sketching of a black college student staring at a chalkboard. On the chalkboard to the left is a person at a desk with strang objects around them. The middle of the chalkboard is just a white square. On the right of the challkboard is a clunky robot with other symbols around it. — Image source: ChatGPT 4-o

Holland, A., & Ciachir, C. (2024). A qualitative study of students’ lived experience and perceptions of using ChatGPT: immediacy, equity and integrity. Interactive Learning Environments, 1-12.

Generative AI summary

Chatlog

Part 1 - Study's Findings and Educational Implications
Main Findings:
Immediacy:
Efficiency: ChatGPT provides immediate answers, saving students' time compared to traditional search engines like Google.
Idea Generation: ChatGPT aids in overcoming "brain freeze" by providing inspiration for starting assignments.
Reassurance and Validation: ChatGPT acts as a personal tutor, offering instant feedback and validation, which is particularly beneficial when academic staff are unavailable.
Equity:
Access: There are concerns about the fairness of access, particularly between the free version and the more powerful paid subscription (ChatGPT Plus).
Support for Neuro-diverse Students: ChatGPT is seen as beneficial for students with neurodivergent conditions, offering them an advantage in managing tasks.
Degree Value: Concerns were raised about ChatGPT potentially devaluing degrees if perceived as enabling academic dishonesty.
Integrity:
Academic Misconduct: The use of ChatGPT raises issues of academic integrity, particularly in group work where inappropriate use by one member can affect the entire group.
Trust and Accuracy: There are mixed feelings about the reliability and bias of information provided by ChatGPT.
Policy Clarity: There is a need for clear institutional policies regarding the ethical use of ChatGPT.
Educational Implications:
Learning and Teaching:
Enhanced Learning Efficiency: ChatGPT can enhance the learning process by providing quick access to information and facilitating idea generation, which can help students to engage more deeply with academic materials.
Support for Diverse Learners: By offering tailored support, ChatGPT can assist neurodivergent students and those studying in a second language, thereby promoting inclusive education practices.
Shift in Learning Dynamics: The reliance on ChatGPT for immediate feedback may reduce traditional academic interactions, necessitating a balance between digital and personal academic support.
Assessment and Academic Integrity:
Reevaluation of Assessment Methods: Institutions might need to reconsider traditional assessment methods to mitigate the risk of academic misconduct facilitated by ChatGPT. This could include more oral examinations, in-class assessments, and practical evaluations.
Clear Guidelines and Policies: Establishing clear policies on the acceptable use of ChatGPT in academic work is crucial. This includes defining the boundaries between legitimate use for learning enhancement and misconduct.
Educating Students on Ethical Use: Incorporating training on ethical use and critical evaluation of AI-generated content into the curriculum can help students understand the implications of using such technologies responsibly.
Equity and Access:
Ensuring Fair Access: Institutions should consider providing equitable access to advanced versions of AI tools like ChatGPT to all students, possibly through institutional subscriptions.
Addressing Digital Divide: There should be strategies to address the digital divide, ensuring all students have the necessary resources and skills to use AI tools effectively.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
Strategies and Methodologies:
Qualitative Approach:
Semi-Structured Interviews: The study utilized semi-structured interviews with recent graduates to gather in-depth insights into their experiences and perceptions of using ChatGPT.
Thematic Analysis: Data from the interviews were coded and analyzed using NVivo to identify key themes (Immediacy, Equity, Integrity).
Iterative Methodology:
Inductive Approach: An iterative, qualitative methodology allowed for a cyclical process of data collection and analysis, refining the understanding of students' experiences with each iteration.
Social Constructionist Tradition:
Sense-Making: The study is grounded in a social constructionist perspective, exploring how students construct and share meanings about their experiences with ChatGPT.
Theoretical Perspectives:
Technology Acceptance Model (TAM):
Perceived Ease of Use and Usefulness: The study builds on the TAM framework, highlighting that students find ChatGPT easy to use and useful for their academic tasks.
Perceived Control: The study introduces the concept of perceived control as an additional determinant in the acceptance and use of ChatGPT, emphasizing the importance of students' control over the use process.
Impact on Higher Education:
Enhancing Educational Value:
Immediate Feedback and Validation: ChatGPT's ability to provide instant feedback enhances students' learning experiences, making it a valuable educational tool.
Support for Diverse Learning Needs: By accommodating diverse learning needs, ChatGPT can contribute to a more inclusive educational environment.
Addressing Ethical and Integrity Concerns:
Policy Development: Clear and comprehensive policies on the use of AI in education are necessary to address ethical concerns and maintain academic integrity.
Ethical Training: Educating students and staff on the ethical use of AI tools is essential to ensure responsible usage and mitigate risks associated with academic misconduct.
Balancing Digital and Personal Interaction:
Maintaining Human Interaction: While ChatGPT can support learning, it is important to balance its use with personal interactions with academic staff to maintain the social aspect of learning.
Digital Co-Dependence: Institutions need to be aware of the potential for digital co-dependence and develop strategies to integrate AI tools in a way that complements rather than replaces human interaction.
Part 3 - Gaps, Challenges, and Limitations
Gaps:
Sample Size and Demographics:
Gap: The study's sample size is limited to 15 recently graduated management students from a single university in the UK.
Elaboration: This narrow sample limits the generalizability of the findings. Educational developers should consider the diverse contexts in which ChatGPT is used. Broader studies incorporating various disciplines, universities, and geographical locations would provide a more comprehensive understanding of ChatGPT's impact across different educational settings.
Longitudinal Perspective:
Gap: The study lacks a longitudinal perspective, focusing only on recent graduates' experiences.
Elaboration: A longitudinal approach could offer insights into how perceptions and usage of ChatGPT evolve over time. Educational developers should advocate for extended studies to track changes in attitudes, skills development, and academic integrity concerns as students progress through their academic careers.
Comparison with Other AI Tools:
Gap: The study focuses exclusively on ChatGPT without comparing it to other AI tools or traditional educational resources.
Elaboration: Comparative studies could highlight the unique benefits and drawbacks of ChatGPT relative to other technologies. This would help educational developers make informed decisions about integrating multiple AI tools to support diverse learning needs.
Challenges:
Equity in Access:
Challenge: Disparities in access to advanced versions of ChatGPT (e.g., ChatGPT Plus) create an uneven playing field.
Elaboration: Educational developers should consider strategies to provide equitable access to AI tools, possibly through institutional subscriptions or financial support for students. Addressing these disparities is crucial to ensure all students benefit from technological advancements.
Academic Integrity:
Challenge: The potential for academic misconduct with ChatGPT use is a significant concern.
Elaboration: Clear guidelines and robust policies are needed to define acceptable use. Educational developers must collaborate with faculty to create assessments that minimize opportunities for misconduct, such as in-person exams or assignments that require critical thinking and originality.
Limitations:
Focus on Qualitative Data:
Limitation: The study relies solely on qualitative data, which, while rich in detail, may lack the breadth provided by quantitative analysis.
Elaboration: Combining qualitative insights with quantitative data could offer a more balanced view of ChatGPT's impact. Educational developers should encourage mixed-methods research to capture both the depth and breadth of student experiences.
Potential Biases in Self-Reporting:
Limitation: Self-reported data from interviews may be subject to biases, such as social desirability bias.
Elaboration: Triangulating self-reported data with other data sources, like academic performance metrics or usage logs, could mitigate these biases. Educational developers should support studies that incorporate multiple data sources for a more accurate assessment of AI tools in education.
Part 4 - Application to Educational Development
Relevance to Educational Developers and Faculty:
Enhancing Learning Efficiency:
Insight: The study highlights ChatGPT's role in improving learning efficiency by providing immediate answers and aiding in idea generation.
Application: Educational developers can integrate ChatGPT into learning management systems to offer instant support for students. Faculty can use ChatGPT to create interactive learning modules that simulate tutor-student interactions, helping students grasp complex concepts more quickly.
Supporting Diverse Learners:
Insight: ChatGPT's ability to support neuro-diverse students and those studying in a second language is a significant finding.
Application: Institutions should provide training on using ChatGPT to accommodate diverse learning needs. Educational developers can design workshops and resources to help faculty understand how to leverage ChatGPT's capabilities to support inclusive teaching practices.
Promoting Academic Integrity:
Insight: Concerns about academic integrity necessitate clear guidelines and policies.
Application: Educational developers should work with academic integrity offices to draft comprehensive policies on AI use. Faculty training sessions can focus on designing assessments that reduce the likelihood of misconduct, such as collaborative projects, presentations, and reflective essays that require personal engagement and critical analysis.
Adapting Recommendations for Educational Development:
Institutional Policies:
Recommendation: Develop clear, institution-wide policies on the ethical use of ChatGPT.
Application: Educational developers should facilitate discussions with stakeholders (faculty, students, administrators) to co-create these policies. Regular updates and training sessions can ensure all parties understand and adhere to the guidelines.
Training and Support:
Recommendation: Provide training for both students and staff on effective and ethical use of ChatGPT.
Application: Develop comprehensive training programs that include workshops, online tutorials, and user guides. These resources should cover practical skills (e.g., creating effective prompts) and ethical considerations (e.g., recognizing and avoiding plagiarism).
Equity and Access:
Recommendation: Ensure equitable access to advanced AI tools for all students.
Application: Advocate for institutional subscriptions to premium AI services and explore funding options for students who cannot afford these tools. Educational developers can also collaborate with technology providers to negotiate discounts or institutional licenses.
Assessment Design:
Recommendation: Redesign assessments to incorporate AI tools without compromising academic integrity.
Application: Encourage faculty to create assignments that require higher-order thinking skills and personalized responses. Examples include project-based learning, where students document their process and reflections, and in-class assessments that combine traditional and AI-supported elements.
Practical Insights:
Integrating AI into Curriculum:
Incorporate modules on AI literacy into existing courses, ensuring students understand both the capabilities and limitations of tools like ChatGPT.
Use ChatGPT to facilitate peer-to-peer learning, where students use the tool to generate discussion topics or study questions, promoting collaborative learning.
Monitoring and Evaluation:
Implement systems to monitor the use of ChatGPT in coursework, ensuring adherence to ethical guidelines.
Conduct regular evaluations of AI integration to assess its impact on learning outcomes and make necessary adjustments.
By addressing these gaps, challenges, and limitations, and applying the study's insights, educational developers can enhance the integration of AI tools like ChatGPT in higher education, promoting both effective learning and academic integrity.

My Commentary

There's a lot of repitition in this study that has come up in other studies that I've explored in this series and as importantly, in the hundreds of conversations that I've had to with facutly and leadership over the last 18 months about concerns and challenges around AI. What's interesting is that these are coming from students. In particular, the emphasis on needing guidance, ethical use, and equitable access. The applications for educational development are both somewhat obvious and are clearly what's already happening such as a race to find the best way to integrate generative AI into LMSs for better learning or finding best ways to support neuro-diverse learners. I was intriguied by the emphasis on introducing AI literacy into the curriculum as well as leveraging generative AI for peer-to-peer learning. That makes me wonder what that might look like to deploy.

Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Huamaní Jordan, O., López Roca, C., & Saavedra Tirado, K. (2024). Analysis of college students' attitudes toward the use of ChatGPT in their academic activities: effect of intent to use, verification of information and responsible use. BMC psychology, 12(1), 1-18.

Generative AI summary

Chat log

Part 1 - Study's Findings and Educational Implications
Main Findings of the Study:
Positive Attitudes towards ChatGPT:
A significant positive attitude towards ChatGPT is influenced by responsible use (β=0.806***), intention to use frequently (β=0.509***), acceptance (β=0.441***), and positive emotions (β=0.418***).
Negative factors like perceived risk (β=‑0.104**) and boredom (β=‑0.145**) have a less decisive influence.
Ease of Use and Frequent Use:
Ease of use and frequent use intention are strong predictors of positive attitudes towards ChatGPT.
Students perceive ChatGPT as a valuable tool for learning, enhancing their intention to use it frequently.
Verification of Information:
Intention to verify information from ChatGPT is significantly related to responsible use, highlighting the importance of critical evaluation skills among students.
Demographics:
The study included 499 students from public and private universities in Peru, with a higher proportion of women (52%) and a significant representation of young adults aged 17-22 years.
Educational Implications:
Curriculum Development:
Higher Education Institutions (HEIs) should integrate AI literacy into their curricula, emphasizing the responsible and ethical use of AI tools like ChatGPT.
Courses should include modules on the verification of information and critical thinking skills to ensure students can assess AI-generated content accurately.
Teaching Strategies:
Educators should encourage the use of ChatGPT for enhancing learning experiences while maintaining a focus on verifying and evaluating the information provided by the AI.
Incorporating AI tools in assignments and projects can make learning more interactive and personalized.
Addressing Concerns:
HEIs need to address students' concerns regarding data privacy and security to foster a safe learning environment.
Developing clear guidelines and policies for the use of AI in academic settings can help mitigate risks associated with its use.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
Strategies and Methodologies:
Quantitative Approach:
The study utilized a quantitative approach with a non-experimental design.
An online survey was administered to 499 participants to gather data on their attitudes towards ChatGPT.
Models and Constructs:
The study employed the Unified Theory of Acceptance and Use of Technology (UTAUT2) and the short version of the Pupils’ Attitudes Toward Technology model (PATT-SQ-SE).
Constructs like ease of use, perceived risk, intention to use frequently, and verification of information were adapted to assess students' attitudes.
Theoretical Perspectives:
Unified Theory of Acceptance and Use of Technology (UTAUT2):
UTAUT2 provides a robust framework for understanding the factors influencing the acceptance and use of technology.
The model includes constructs such as performance expectancy, effort expectancy, social influence, facilitating conditions, hedonic motivation, and habit.
Pupils’ Attitudes Toward Technology (PATT-SQ-SE):
This model focuses on measuring the affective component of attitudes towards technology, including interest, boredom, and emotions.
It helps in understanding how students' feelings and interests impact their behavior towards using technology.
Impact on Document Content:
Enhanced Understanding:
The integration of UTAUT2 and PATT-SQ-SE models enhances the understanding of students' attitudes by combining cognitive, affective, and behavioral components.
This comprehensive approach allows for a deeper analysis of the factors influencing the acceptance and responsible use of ChatGPT.
Predictive Validity:
The use of structural equation modeling (SEM) and partial least squares (PLS) techniques ensured the reliability and validity of the constructs.
High reliability and validity indicators support the robustness of the findings, making the study's conclusions more impactful.
Practical Application:
The theoretical frameworks guide the practical applications of the findings, such as curriculum development and teaching strategies.
By aligning educational practices with the theoretical insights, HEIs can better prepare students for the digital era and the ethical use of AI tools.
Part 3 - Gaps, Challenges, and Limitations
Gaps Identified:
Limited Demographic Scope:
The study focuses on university students in Peru, potentially limiting the generalizability of the findings to other regions or countries. The sample might not represent the diversity of attitudes toward ChatGPT globally.
Lack of Longitudinal Data:
The research is cross-sectional, capturing attitudes at a single point in time. This approach does not account for how attitudes and behaviors might change over time with increased exposure to and familiarity with ChatGPT.
Focus on Student Attitudes Only:
The study examines only students' perspectives, neglecting the views of other stakeholders such as faculty, administrators, and policymakers, who play crucial roles in the integration of AI tools in education.
Challenges Highlighted:
Data Privacy and Security Concerns:
Students' concerns about data privacy and security negatively impact their intention to use ChatGPT frequently. This challenge necessitates robust data protection measures and clear communication about how data is managed.
Ensuring Responsible Use:
Promoting the responsible use of ChatGPT is critical, especially to prevent academic dishonesty and misinformation. This requires the development of guidelines and educational strategies to foster ethical AI usage.
Limitations Explained:
Non-Experimental Design:
The study's non-experimental design limits the ability to establish causality between variables. While associations are identified, causal relationships remain speculative.
Survey Methodology Bias:
The reliance on self-reported data can introduce bias, as participants may provide socially desirable responses rather than accurate reflections of their attitudes and behaviors.
Technological Familiarity:
The study assumes a baseline familiarity with ChatGPT among participants, which might not hold true in less technologically advanced settings. This could skew the findings towards more tech-savvy student populations.
Implications for Educational Developers:
Addressing Generalizability:
Educational developers should consider conducting similar studies in diverse geographical and cultural contexts to validate and extend the findings.
Longitudinal studies could provide insights into how attitudes towards ChatGPT evolve over time, informing the development of adaptive educational strategies.
Inclusive Stakeholder Engagement:
Future research should include perspectives from faculty, administrators, and policymakers to create a holistic understanding of the challenges and opportunities associated with AI tools in education.
Enhancing Data Privacy Measures:
Institutions must prioritize data privacy and security to build trust among users. Clear guidelines and transparent practices can mitigate concerns and encourage broader adoption.
Promoting Ethical Use:
Developing comprehensive training programs on the responsible use of AI tools is essential. Educational developers can create resources and workshops to guide students in ethical AI practices.
Part 4 - Application to Educational Development
Relevance to Educational Developers and Faculty:
Curriculum Integration:
The study's findings highlight the need for integrating AI literacy into the curriculum. Educational developers can design courses that include modules on AI ethics, data verification, and critical thinking.
Training and Workshops:
Educational developers should organize training sessions for both students and faculty to enhance their understanding and responsible use of ChatGPT. These sessions can cover practical applications, ethical considerations, and data privacy.
Policy Development:
Based on the study's insights, institutions should develop clear policies regarding the use of AI tools in academic settings. These policies should address acceptable use, data privacy, and measures to prevent academic dishonesty.
Adaptation and Application of Recommendations:
Developing AI Literacy Programs:
Implementing AI literacy programs that focus on understanding how AI tools like ChatGPT function, their benefits, and their limitations. These programs should also cover ethical considerations and the importance of verifying information.
Creating Support Systems:
Establishing support systems, such as AI help desks or mentorship programs, where students can seek guidance on using AI tools effectively and responsibly.
Enhancing Digital Resources:
Educational developers should collaborate with IT departments to ensure that digital resources and platforms are secure and user-friendly, addressing concerns about ease of use and data security.
Examples of Application in Other Contexts:
Multidisciplinary Approaches:
Integrating AI tools in various disciplines beyond computer science, such as humanities and social sciences, to demonstrate their versatile applications and encourage cross-disciplinary learning.
Community Engagement:
Engaging with the broader community, including parents and industry partners, to create awareness about the responsible use of AI in education. This can foster a supportive environment for students.
Continuous Improvement:
Establishing feedback mechanisms where students and faculty can provide input on their experiences with AI tools. This feedback can inform continuous improvements and updates to educational practices and policies.
Leveraging Insights from Other Research:
Comparative Studies:
Drawing on research from other regions and institutions to compare findings and identify best practices. For example, studies from regions with high technology adoption rates can provide insights into effective integration strategies.
Innovative Teaching Practices:
Exploring innovative teaching practices that leverage AI tools to enhance student engagement and learning outcomes. For instance, using ChatGPT for personalized tutoring or as a tool for brainstorming and creative writing exercises.
By applying the insights and recommendations from the document, educational developers can foster a more informed, ethical, and effective use of AI tools in higher education, ultimately enhancing the learning experience for students and preparing them for the future.

My Commentary

So the previous study was qualitative, this is quantitative, looking to some degree at a similar topic about student usage. Again here, we get the mention of AI literacy in the curriculum. It makes me wonder if the updated model of ChatGPT 4o has a more recent data source (say, through 2023). That would explain to some degree the reoccurence of this point beyond just the research (Currently, "AI literacy" returns over 425,000 results on Google). It's interesting that with this study (quantitative), the AI didn't acknowledge the gap that it was quantitative and missed the nuance that comes with qualitative. That is, this might be an example of a bias towards quantatitive research over qualitative. I was also intrigued to see the emphasis on data privacy measures that wasn't as relevant in the previous study. It's final recommendations are interesting in the areas it covers. I think I might like to see it dig down further into 1-2 of those areas and provide something more comprehensive.

Share AI + Education = Simplified

Yan, W., Nakajima, T., & Sawada, R. (2024). Benefits and Challenges of Collaboration between Students and Conversational Generative Artificial Intelligence in Programming Learning: An Empirical Case Study. Education Sciences, 14(4), 433.

Generative AI summary

Chat log

Part 1 - Study's Findings and Educational Implications
Main Findings
The study conducted by Yan, Nakajima, and Sawada (2024) explores the benefits and challenges of using conversational generative artificial intelligence (Gen AI) as a learning partner in programming education. The main findings are:
Enhanced Meta-cognitive and Self-regulated Learning Skills: Effective collaboration with Gen AI was found to improve students' meta-cognitive and self-regulated learning skills. Students became better at managing their learning processes, setting goals, and evaluating their progress.
Positive Impact on Human-to-Human Communication: Interaction with Gen AI also positively influenced students' human-to-human communication skills. The need for clear, structured communication with AI helped students develop better communication strategies that translated into their interactions with peers and instructors.
Challenges in Initial Phases and Complex Tasks: Students faced significant difficulties initially due to the lack of experience in communicating with AI and understanding how to use AI effectively in complex tasks like app development. Individual differences in students' abilities also played a significant role in their success.
Sustainable and Independent Learning: The study found that students who developed a collaborative relationship with Gen AI could engage in sustainable and independent learning. This collaboration helped students extend their learning beyond the classroom.
Educational Implications
The findings of this study have several implications for higher education, particularly in the field of teaching and learning:
Development of Communication Skills: Incorporating Gen AI into the curriculum can enhance students' communication skills. As students learn to articulate their queries and understand AI responses, they develop skills that are transferable to human interactions.
Promotion of Self-regulated Learning: Gen AI can be a powerful tool to promote self-regulated learning. By providing immediate feedback and facilitating reflective learning practices, AI helps students take ownership of their learning process.
Addressing Individual Differences: Educators need to consider individual differences in students' abilities to interact with AI. Tailored support and instruction in using Gen AI effectively can help bridge the gap and ensure all students benefit from AI-assisted learning.
Integration into Complex Learning Tasks: To fully leverage the potential of Gen AI, its integration into more complex and higher-order learning tasks should be carefully planned. This includes providing adequate training and resources for both students and educators.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
Strategies and Methodologies
The study utilized a comprehensive set of strategies and methodologies to explore the impact of Gen AI on student learning:
Class Design: The course was designed with progressively challenging tasks, starting from basic communication with Gen AI to more complex app development. This gradual increase in complexity helped students build their skills incrementally.
Teaching-Learning Activities: Activities focused on teaching students effective communication strategies with Gen AI, such as prompt management and memory management. These strategies were essential for maximizing the benefits of AI collaboration.
Data Collection: A variety of data collection methods were used, including class observations, surveys, final reports, dialogues between students and Gen AI, and semi-structured interviews. This comprehensive approach provided a deep understanding of students' experiences and learning outcomes.
Theoretical Perspectives
The study draws on several theoretical perspectives to enhance its findings and implications:
Constructivism: The approach is rooted in Vygotsky and Piaget’s constructivism theory, emphasizing active learning and knowledge construction through interaction and collaboration. Gen AI serves as a partner in this learning process, enabling students to construct knowledge through dialogue and feedback.
Collaborative Learning: The principles of collaborative learning, such as maximum peer interaction, equal partnerships, and shared learning goals, are applied to student-Gen AI interactions. This framework supports the idea that learning is a social process enhanced through effective communication and collaboration.
Self-regulated Learning: The study highlights the role of Gen AI in promoting self-regulated learning. By setting goals, monitoring progress, and reflecting on outcomes, students enhance their ability to manage their learning independently, which is critical for lifelong learning.
Impact on Higher Education, Teaching, and Learning
The strategies, methodologies, and theoretical perspectives used in the study have significant implications for higher education:
Enhanced Learning Strategies: The structured approach to learning with Gen AI helps students develop effective learning strategies that are applicable across various subjects and contexts.
Critical Thinking and Reflection: By engaging in continuous dialogue with Gen AI, students enhance their critical thinking and reflective skills, which are crucial for academic success and personal development.
Broader Application of AI in Education: The study provides a model for integrating AI into educational practices, demonstrating its potential to support diverse learning needs and promote equitable access to quality education.
In summary, this study provides valuable insights into the benefits and challenges of integrating Gen AI in programming education. The findings emphasize the importance of effective communication, self-regulated learning, and tailored support to maximize the potential of AI-assisted learning. Theoretical perspectives from constructivism and collaborative learning further enrich the understanding of how AI can transform teaching and learning in higher education.
Part 3 - Gaps, Challenges, and Limitations
Identified Gaps, Challenges, and Limitations
Sample Size and Generalizability: The study involved only nine third-year undergraduate students from a single department, limiting the generalizability of the findings. The small sample size and homogeneity restrict the applicability of the results to a broader student population.
Initial Phase Difficulties: Students faced significant challenges in the initial phases of learning to communicate effectively with Gen AI. This initial learning curve highlights a potential barrier to the successful integration of AI in educational contexts.
Individual Differences in Student Abilities: The study revealed notable individual differences in students' abilities to interact with and benefit from Gen AI. Some students struggled to formulate effective prompts or fully utilize Gen AI’s capabilities, impacting their learning outcomes.
Complexity of Tasks: Students found it particularly challenging to collaborate with Gen AI on complex tasks such as app development. This difficulty underscores the need for additional support and training for students when integrating AI into more advanced learning activities.
Dependence on Instructor Expertise: The success of the study was partly due to the instructors’ extensive experience with programming and Gen AI. This dependence on instructor expertise poses a challenge for wider implementation, especially in institutions where such expertise may be limited.
Elaboration on Gaps, Challenges, and Limitations
Sample Size and Generalizability: Educational developers must recognize the limitations posed by small, homogeneous samples. To address this, future studies should aim for larger, more diverse participant groups to ensure findings are broadly applicable. Other research supports this need for diversity to capture a wide range of learning experiences and outcomes (Hattie, 2009).
Initial Phase Difficulties: The initial learning curve in effective Gen AI communication suggests a need for structured support during the early stages of AI integration. Educational developers should design scaffolding strategies and introductory modules to ease students into using AI tools. Research on technology adoption in education highlights the importance of such support in overcoming initial resistance and fostering effective use (Rogers, 2003).
Individual Differences in Student Abilities: Addressing individual differences requires personalized learning approaches. Educational developers should consider adaptive learning technologies that tailor AI interactions to individual student needs and abilities. Research in adaptive learning technologies suggests that personalized support can significantly enhance learning outcomes (Chen et al., 2020).
Complexity of Tasks: The challenge of using Gen AI for complex tasks like app development highlights the need for incremental task complexity and ongoing support. Educational developers should integrate a phased approach, where simpler tasks gradually build towards more complex ones. This aligns with the principles of scaffolding in education, which emphasize the importance of providing appropriate support at different learning stages (Wood, Bruner, & Ross, 1976).
Dependence on Instructor Expertise: The reliance on instructor expertise indicates a need for professional development in Gen AI technologies. Educational developers should create training programs to equip faculty with the necessary skills and knowledge to effectively integrate AI into their teaching. Professional development literature underscores the importance of ongoing training and support for faculty to keep pace with technological advancements (Guskey, 2002).
Part 4 - Application to Educational Development
Relevance to Educational Developers and Faculty
Enhancement of Communication Skills: The study's findings on the improvement of students’ communication skills through Gen AI interactions are highly relevant. Educational developers can incorporate AI tools to enhance communication-focused activities across various disciplines, not just programming. For example, AI can be used in language learning to facilitate conversational practice, helping students develop their language proficiency.
Promotion of Self-regulated Learning: The positive impact of Gen AI on self-regulated learning skills provides a strong case for its inclusion in educational strategies. Faculty can integrate AI tools to help students set learning goals, monitor their progress, and reflect on their learning. This approach can be applied in project-based learning environments where students take ownership of their projects, using AI as a supportive tool.
Addressing Individual Differences: The study's identification of individual differences in student abilities to interact with Gen AI suggests the need for personalized learning approaches. Educational developers can leverage AI’s adaptive capabilities to provide customized learning experiences. For instance, AI-driven platforms can offer differentiated instruction based on real-time assessment of student performance, catering to diverse learning needs.
Adaptation and Application of Recommendations
Structured Integration of AI Tools: To address initial phase difficulties, educational developers should design structured AI integration modules. These modules can include tutorials, practice sessions, and feedback mechanisms to help students become proficient in using AI tools. For instance, introductory workshops on AI literacy can be implemented at the beginning of the semester.
Incremental Complexity in Learning Tasks: To manage the complexity of tasks, developers should design curricula that gradually increase in difficulty. This can involve starting with basic tasks that build foundational skills before moving on to more complex projects. For example, in a programming course, initial assignments could involve simple coding exercises, gradually progressing to full-scale app development projects.
Professional Development for Faculty: Addressing the dependence on instructor expertise, educational developers should offer comprehensive professional development programs. These programs can cover the pedagogical use of AI, troubleshooting common issues, and staying updated with the latest AI advancements. Workshops, webinars, and collaborative learning communities can be effective formats for this professional development.
Practical Insights and Applications
Cross-disciplinary AI Integration: The insights from the study can be applied across various disciplines beyond programming. For instance, in medical education, AI can be used to simulate patient interactions, helping students develop diagnostic and communication skills. In business education, AI can facilitate scenario-based learning, allowing students to explore different business strategies and outcomes.
Continuous Feedback and Reflection: Educational developers should emphasize the importance of continuous feedback and reflection in AI-assisted learning. AI tools can provide real-time feedback on student performance, helping them adjust their learning strategies promptly. Incorporating reflective practices, such as maintaining learning journals, can further enhance this process.
Scalability of AI Tools: To ensure the scalability of AI tools, educational developers should focus on creating scalable models that can be adapted to different contexts and student populations. This includes developing robust support systems, clear usage guidelines, and ensuring accessibility for all students. Research on educational technology implementation highlights the need for scalability to achieve widespread impact (Wang, 2009).
In conclusion, the document provides valuable insights into the integration of Gen AI in educational contexts. By addressing the identified gaps and leveraging the specific findings, educational developers can create more effective, personalized, and scalable learning environments. The recommendations and practical applications outlined here can guide the adaptation of Gen AI tools to enhance teaching and learning across various disciplines.

My Commentary

Now, this study is intriguing and makes me want to revisit the article to full flesh out what occurred. It feels too good to be true and some lines make me raise an eyebrow ("By engaging in continuous dialogue with Gen AI, students enhance their critical thinking and reflective skills, which are crucial for academic success and personal development."--without saying how, it's unclear how GenAI did or supported this). Of course, when we see it is a course of 9 students, that feels like it gives away the game. With 9 students, the flexibility and intensity of dialogue and learning can be much more enhanced. It would be interested to see this with a larger class of 15-25 to see if it can be maintained. I also know that it feels a bit different because it was a programming course which can feel different than other courses folks teach. This result also provided citations, which when I asked it, hours later to produce a Works Cited, it did and many of them are accurate, though the DOIs it provides is not.

Meta-commentary

Of the three, the first two felt like compliments to one another that emphasized some larger and consistent concerns and practices that higher education should be thinking about. However, the third one feels like it has some interesting considerations and possibilities. It could be useful to dig more into it and use it as a case study to further share out and explore. In fact, I think that’s one of the next things I want to do is to try to use ChatGPT to build out a case study to examine with questions and a guide for other faculty to build and learn from. Stay tuned for more!

AI + Education = Simplified

Research Insights #9: Student-Focused Studies Part 6

More research about AI and students...

Discussion about this post