Research Insights #8: Open Educational Resources & Generative AI Part 2
Continue the exploration and OER research...
Following up on yesterday’s post with 2 more articles focusing on AI and OER.
***Another shoutout that if you are around today (Wednesday, April 10, 2024, 3pm EDT/ 12pm PDT), you can check out this panel that I’m moderating on AI & OER: Redefining Education? for the Community College Consortium for Open Educational Resources (CCCOER) with panelists, Anna Mills and Peter Shea.
Deus, W., & Barbosa, E. (2024). OER-Chat: An Open Chatbot to Support the Reuse of Open Educational Resources of Introductory Programming.
Generative AI summary
Part 1 - Study's Findings and Educational Implications
Main Findings
The study presents OER-Chat, a chatbot tailored for introductory programming, which utilizes Open Educational Resources (OER) to provide educational materials like slides, courses, or open textbooks to answer users' queries. This approach promotes the reuse of these materials and supports Open Education. The qualitative evaluation demonstrated OER-Chat's effectiveness in user experience, usability, and result adherence.
Educational Implications
For higher education, OER-Chat signifies a shift towards leveraging AI to enhance resource accessibility and reuse. It addresses ethical concerns by using openly licensed materials. In teaching and learning, it offers a model for integrating AI to support both students and educators, providing relevant and tailored resources that enhance learning experiences and promote self-directed learning.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
Strategies and Methodologies
OER-Chat employs a robust architecture involving modules like Pre-processing, Vocabulary, Introductory Programming, Tags, Weights, and Probability Calculator. These modules work in tandem to understand user queries, identify relevant OER, and present them in an organized manner, ensuring the chatbot's responses are pertinent and educational.
The chatbot's design is purposefully open, contrasting with proprietary AI systems, and focuses on academic resources' reuse. This design choice underscores a commitment to open education and transparency, allowing for continuous improvement and adaptability in educational contexts.
Theoretical Perspectives
Theoretical perspectives in the document revolve around the use of AI in education, focusing on enhancing access to and reuse of educational materials. The chatbot's approach aligns with constructivist learning theories, emphasizing resource accessibility and learner autonomy. By providing resources that users can adapt and integrate into their learning processes, OER-Chat embodies principles of open education and constructivist learning, fostering an environment where learners construct knowledge through engagement with curated resources.
In-depth Analysis and Practical Insights
OER-Chat offers a novel approach to integrating AI in education, focusing on resource accessibility and ethical use of materials. Its open architecture and focus on OER reuse present a model that educational developers can adopt or adapt to create more inclusive, resource-rich learning environments. As educational institutions increasingly embrace digital and AI-driven tools, OER-Chat's methodologies and theoretical underpinnings offer valuable insights into creating tools that are not only technologically advanced but also pedagogically sound and ethically responsible.
Part 3 - Gaps, Challenges, and Limitations
Gaps and Challenges
Limited Scope of Evaluation: The study's evaluation involved only five participants, which may not provide a comprehensive understanding of OER-Chat's effectiveness across diverse educational settings. This limitation raises questions about the generalizability of the findings.
Technical Limitations: While OER-Chat is designed to address ethical concerns by using open resources, the document does not deeply explore the chatbot's limitations in understanding complex queries or its adaptability to various learning contexts.
Content Relevance and Quality: The study assumes that OERs provided are of high quality and relevance. However, there's a lack of discussion on how OER-Chat evaluates the pedagogical value of the resources it recommends.
User Experience and Interaction: The sentiment analysis highlights positive user experiences, but there's minimal exploration of how interactive features or user engagement mechanisms could enhance the learning experience.
Considerations for Educational Developers
Educational developers should critically assess the scope and depth of tools like OER-Chat, considering how they can be scaled and adapted for diverse educational contexts. They should also consider the importance of content curation mechanisms to ensure that the resources provided are not only relevant but also pedagogically valuable. Additionally, developers should explore ways to enhance user interaction and engagement with AI-driven educational tools to improve learning outcomes.
Part 4 - Application to Educational Development
Relevance to Educational Developers and Faculty
Enhancing Resource Accessibility: OER-Chat demonstrates a practical application of AI in making educational resources more accessible. Developers can consider similar models to create tools that help educators and students find and utilize OER more effectively.
Supporting Educator Roles: By providing a tool that supplements educators' efforts in resource provision, OER-Chat can be seen as an ally in the educational process, allowing faculty to focus on pedagogy while the chatbot aids in resource discovery.
Fostering Open Education Practices: The chatbot's emphasis on OER promotes the principles of open education. This approach can inspire educational developers to create or adapt tools that support the sharing and reuse of educational materials, aligning with broader goals of democratizing education.
Adapting Recommendations in Educational Development
Customization for Contextual Relevance: Developers can adapt the OER-Chat model to cater to specific disciplinary needs or educational levels, ensuring that the resources provided are contextually relevant.
Integrating Feedback Mechanisms: To address the challenge of content relevance and quality, developers can incorporate mechanisms that allow users to provide feedback on the usefulness of the resources, enabling continuous improvement of the tool.
Enhancing Interactivity: Building on the chatbot's foundation, educational developers can explore ways to make AI-driven tools more interactive, such as integrating adaptive learning pathways or gamification elements to engage learners actively.
In conclusion, while OER-Chat presents an innovative approach to integrating AI in education, its application and effectiveness in broader contexts require careful consideration. Educational developers can draw valuable insights from this study, applying its findings and recommendations to develop tools that are not only technologically advanced but also pedagogically sound and aligned with the goals of open and inclusive education.
My Commentary
What's cool about this article is that it appears to actually apply a concept that I know folks have been thinking about--creating AI Chatbots out of OER content in order for students to have additional help within a given course. By the summary, it sounds like it was a very small sample and there were little addressing of the inaccuracy factor. The limited amount of students and the particular domain (programming), which seems to have higher popularity and ability for helping students makes me leary about what this could look like for other disciplines, but it is a step in the direction we're hoping for. I dug into the article and it doesn't really idenitfy the tool they used which makes me think it was an internally created tool. I have hopes for things like this but also know that other folks have had trouble building these with the custom GPTs from ChatGPT and other similar tools.
Li, Z., Pardos, Z. A., & Rena, C. (2024). Aligning open educational resources to new taxonomies: How AI technologies can help and in which scenarios. Computers & Education, 105027.
Generative AI summary
Based on the detailed analysis of the document "Aligning open educational resources to new taxonomies: How AI technologies can help and in which scenarios," I've structured the response according to the requested format:
Part 1 - Study's Findings and Educational Implications
Main Findings
The study explores the alignment of open educational resources (OER) to skill taxonomies, emphasizing the role of AI and machine learning in automating this process.
It presents a novel machine learning approach to update OER classification to reflect new or evolving taxonomies, significantly addressing the labor-intensive process of manual tagging.
The research demonstrates that while full automation of OER tagging to new taxonomies is not feasible, their approach can achieve near-expert accuracy with a significant amount of labeled examples (around 5,000), showing promising results even with as few as 100 examples.
Educational Implications
For educators and institutions, the findings highlight the potential for AI-assisted systems to rapidly align educational content with evolving standards, facilitating adaptive learning environments and personalized education.
Policymakers and educational technology developers might leverage these insights to foster the development and integration of AI-driven tools in educational platforms, promoting more dynamic and responsive educational resources.
The study underscores the importance of continuous improvement and adaptation of educational materials, aligning with the fast-paced evolution of educational standards and methodologies.
Part 2 - Strategies, Methodologies, and Theoretical Perspectives
Strategies and Methodologies
The study employs a combination of machine learning methods, utilizing text, image figures, and videos from educational resources to train models that can automate the tagging process of OER to various taxonomies.
The research methodology is grounded in practical scenarios, examining the effectiveness of the proposed models in different contexts, such as aligning existing resources to new taxonomies or updating classifications across different platforms.
Theoretical Perspectives
The research is framed within the context of artificial intelligence and machine learning's growing role in education, especially in the realm of educational resource management and taxonomy alignment.
It touches upon the theory of zero-shot learning, showcasing its application in aligning resources to unseen taxonomies, which could provide a new perspective on how AI can contribute to educational resource alignment without extensive retraining.
Enhancing Value and Relevance
The methodologies and strategies detailed in the study can serve as a blueprint for educational developers to implement AI-driven systems for OER alignment, enhancing the accessibility and relevance of educational materials.
The integration of such AI methodologies could significantly reduce the time and effort required to align educational resources with current standards, enabling a more agile response to changes in educational requirements and improving the overall quality of education.
This comprehensive analysis highlights the potential of AI in transforming the alignment of educational resources, offering valuable insights for educators, developers, and policymakers in the field of education.
Part 3 - Gaps, Challenges, and Limitations:
Identified Gaps and Challenges
Scope of Taxonomies: The document primarily focuses on the Common Core State Standards and does not delve deeply into other global or subject-specific taxonomies. This limits the applicability of the findings across diverse educational systems and subjects.
Data Diversity: While the study includes text, images, and videos, the diversity within these data types (e.g., different educational levels, subjects) and the representation of various learning styles or educational contexts is not deeply explored.
Model Generalization: The effectiveness of AI models in adapting to completely new taxonomies without prior examples is promising but not fully proven, indicating a potential challenge in model generalization across varied educational standards.
Considerations for Educational Developers
Adaptation to Various Taxonomies: Educational developers should consider how the methodologies and findings can be adapted or extended to other taxonomies, possibly through additional training or customization of the AI models.
Inclusivity in Data: Developers should ensure that the data used to train AI models are inclusive, covering a broad spectrum of subjects, educational levels, and learning modalities to enhance the models' applicability and effectiveness.
Continuous Monitoring: There is a need for continuous monitoring and evaluation of the AI models' performance in real-world scenarios, especially as they adapt to new taxonomies or educational contexts, ensuring they remain effective and accurate.
Part 4 - Application to Educational Development:
Relevance to Educational Developers and Faculty
Enhanced Resource Alignment: The findings highlight the potential for AI to streamline the alignment of educational resources with curricular standards, aiding developers and faculty in maintaining current and relevant educational content.
Adaptive Learning: The study’s insights can inform the development of adaptive learning systems where AI models can help in dynamically aligning resources to individual learner paths based on evolving taxonomies, enhancing personalized learning experiences.
Policy and Curriculum Development: The research provides a foundation for policymakers and curriculum developers to consider AI-assisted tools for more agile and responsive curriculum updates and standards alignment.
Adaptation and Application
Training and Professional Development: Educational developers can leverage the insights to design training programs for faculty, focusing on how AI tools can assist in resource alignment and curriculum development, fostering a tech-enhanced teaching environment.
Collaboration with AI Experts: Developers are encouraged to collaborate with AI experts to tailor the AI models to their specific educational contexts, ensuring the models' effectiveness in aligning resources with various taxonomies or standards.
Research and Evaluation: Incorporating the study's methodologies into ongoing research, educational developers can evaluate the effectiveness of AI-assisted resource alignment in their institutions, using findings to advocate for or refine the adoption of AI in educational development.
By addressing these gaps and leveraging the study’s insights, educational developers and faculty can enhance their strategies for curriculum design, resource alignment, and the integration of AI technologies, ultimately contributing to more adaptive and responsive educational environments.
My Commentary
This has an interesting proposition if I'm reading it right--allowing AI to create the tagging system for OER repositories. That would be a rather solid use case. Most of the taxonomies are either folksonomies or too rigid and make it tedious for people submitting OERs to figure it all out. I will be curious of the first OER platform that deploys AI in this way. In a bigger way, it makes me think about AIs capacity to do that with any large collection and re-present it to the user base in a more coherent and even contextualized manner. For instance, could it be customized to the person engaging in the database? That is, could the AI have or create/continually recreate organically that tagging based upon how a user engages in the platform?
Meta-commentary
It's not entirely surprising that all of this research still feels a bit abstract or having limited application. As I said in the first part of this series, we're in a holding pattern. And I think a lot of that is boiled down to a few issues:
1. Legal issues: It's unclear what is legal or will be legal in terms of using the platforms.
2. The Weird Tension: AI and OER have weird tensions in that OER is structured to be shared with the licensing often requiring acknowledgment; which AI doesn't do or doesn't do well. And OER's goal is accessibility while AI is largely a tool of exclusivity.
3. OER Still Needs Traction: OER has made huge gains and also, many folks are unfamiliar with it or hesitant to use it. So there's also a deficit in its support financially which make it hard for some institutions to invest in OER and AI.
4. AI X OER=Overload: Facutly are often challenged by taking on one big concept for their teaching at a time (rightfully so--it's a lot of work to integrate and change one's approach through a new lens and practice). So there is a small pool of folks interested and excited to discuss or think about pouring engergy into both.
5. The Unsteady Future: It's not even clear if AI will be tenable for the foreseeable future, so there's also some draft with that.
I think it's an exciting space but also a space that is hard to figure out. I guess we'll just have to keep thinking and talking about it as we figure it out.
Along those lines, check out today's session (Wednesday, April 10, 2024, 3pm EDT/ 12pm PDT - AI & OER: Redefining Education?) to maybe get some insights. If you can’t make it, I believe it will be recorded and put on that page.
AI+Edu=Simplified by Lance Eaton is licensed under Attribution-ShareAlike 4.0 International
