Research Insights #4: Student-Focused Studies Part 2
4 studies about student views of generative artificial intelligence.
In the last Research Insights, I decided to focus on a mini-series on student-focused studies. In this post, I particularly look at studies about students’ perceptions of generative AI. I also decided to revise the prompts once more. This time, I used my Prompt Enhancer GPT to change them up. You can see the new prompts in the appendix
Chan, C. K. Y., & Zhou, W. (2023). An expectancy value theory (EVT) based instrument for measuring student perceptions of generative AI. Smart Learning Environments, 10(1), 64.
Generative AI summary
Strategies, Methodologies, and Theoretical Perspectives
The article employs the Expectancy-Value Theory (EVT) as its central theoretical framework to examine student perceptions of generative AI (GenAI) technologies like ChatGPT in educational settings. EVT, which traditionally assesses the relationship between students' motivations and their learning outcomes, is applied here to understand how students perceive the value and costs associated with using GenAI technologies in higher education. This innovative approach extends beyond conventional usability-focused models (like TAM or UTAUT) to explore deeper motivational aspects that influence students' acceptance and use of technology. For instance, the study's findings reveal a strong positive correlation between students' perceived value of GenAI and their intention to use it, alongside a weak negative correlation with perceived costs. This methodology can inform teaching and learning practices by highlighting the importance of aligning GenAI integration with students' perceived values and addressing their concerns about potential costs or drawbacks. It suggests that faculty should not only focus on the functional aspects of GenAI but also consider how these technologies align with students' goals and learning objectives.
Gaps, Challenges, and Limitations
The study acknowledges limitations in its methodology, particularly in the sampling strategy. It used convenience sampling from universities in Hong Kong, which might not be representative of broader student populations. This limitation indicates a potential gap in understanding how different cultural or educational contexts might influence perceptions of GenAI. Additionally, the rapidly evolving nature of GenAI technologies means that students' perceptions could quickly change, suggesting a need for ongoing research in this area. Recognizing these limitations is beneficial for educational developers as it underscores the importance of contextual and cultural factors in technology adoption. It also highlights the need for continuous, adaptable research methodologies to keep pace with technological advancements. Educational developers can use these insights to design more inclusive and representative research studies and to develop adaptive curricula that are responsive to the changing technological landscape.
Applicability to Educational Developers and Faculty
The insights and conclusions from the article have direct implications for educational developers and faculty. The study's application of EVT to understand student perceptions provides a new lens for considering how GenAI can be integrated into teaching and learning. For example, the positive correlation between perceived value and intention to use GenAI suggests that if faculty can effectively demonstrate the value of these technologies in enhancing learning outcomes, students might be more inclined to use them. This could involve integrating GenAI tools in ways that clearly align with learning objectives, such as using them to provide personalized feedback or to facilitate peer interaction and collaboration. Additionally, by being aware of the potential costs or downsides perceived by students, educators can proactively address these concerns, possibly by providing guidance on ethical and effective use of GenAI tools. In practical terms, this means that educational developers need to focus not only on training faculty in the technical use of GenAI but also on strategies to integrate these tools in a way that resonates with student values and learning needs.
My Commentary
I haven't looked at or played with Expectancy-Value Theory--at least not consciously. It feels like something I might have absorbed at some point but didn't necessarily connect it to the actual theory. The main finding that there is a strong positive correlation between student's perceived value of Generative AI and their intention to use it, makes sense. I'm not surprised that there is a weak negative correlation with the perceived costs. That is, the perceived costs only have a weak impact on usage. Here's where I'd want to dig into the article and see if there was a point of comparison with the actual perceived costs (time, ability, access, ego) of doing the actual work without generative AI. Basically, is it a weaker correlation because students feel it costs less to use generative AI than to put in the time, effort, emotion, etc into an assignment that they might not do well on? After all, if GAI gets you a C, that feels less personally costly than if your work scores a C or worse. Still, weak as it might be, I think that's where I see faculty trying to lean more on is the negative costs of using it--which seems like it won't have the intended effect. Is this another instance of a deficit lense working against the goal?
Habibi, A., Muhaimin, M., Danibao, B. K., Wibowo, Y. G., Wahyuni, S., & Octavia, A. (2023). ChatGPT in higher education learning: Acceptance and use. Computers and Education: Artificial Intelligence, 100190.
Generative AI summary
Strategies, Methodologies, and Theoretical Perspectives:
The study employs the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) to investigate factors influencing the adoption of ChatGPT in Indonesian higher education. This framework considers variables like Performance Expectancy (PE), Effort Expectancy (EE), Social Influence (SI), and Facilitating Conditions (FC) to predict technology acceptance. For educational developers and faculty, this theoretical model offers a comprehensive lens to understand the dynamics of AI tool adoption in teaching and learning. For instance, the emphasis on FC underscores the importance of institutional support and resources in adopting new technologies. This could guide faculty in advocating for necessary infrastructure and training to effectively integrate tools like ChatGPT in their teaching. Similarly, understanding the role of PE and Hedonic Motivation (HM) can assist educators in highlighting the practical benefits and enjoyment of using AI tools in their instructional strategies.
Gaps, Challenges, and Limitations:
The study's focus on a specific geographical context (Indonesia) and the exclusion of demographic moderators are notable limitations. This geographical focus may limit the generalizability of the findings to other educational settings. Also, by not considering factors like age, gender, and experience, the study might overlook nuanced differences in technology acceptance across diverse student populations. For educational developers, these limitations highlight the importance of contextualizing research findings. It suggests a need for localized studies to understand how different regions and demographics interact with educational technologies. Additionally, it emphasizes the importance of inclusive research practices that consider a wide range of demographic factors to ensure broader applicability and relevance of the findings.
Applicability to Educational Developers and Faculty:
The insights from the study are directly relevant to educational developers and faculty in several ways. The finding that FC is a key determinant in technology acceptance suggests that educational developers should prioritize creating an enabling environment for technology integration. This involves not just providing the necessary technological infrastructure but also ensuring ongoing support and training for faculty and students. The recognition of the limited role of EE in the adoption of ChatGPT indicates a shift in focus towards the quality and relevance of technology, rather than just its ease of use. This can encourage faculty to explore more complex or sophisticated technologies, knowing that ease of use may not be a significant barrier. Lastly, the study's findings can guide faculty in designing curriculum and instructional strategies that align with the identified determinants of technology acceptance, such as enhancing the perceived performance and enjoyment of using AI tools in the learning process.
My Commentary
This article has some interesting tidbits to consider. Again, it's a theory I'm not as familiar with which is great to discover and appreciate this process of exploring articles as a further exploration into things I didn't know. It doesn't surprise me that Facilitating Conditions are important. I think so much of teaching and learning is about context and we don't often recognize all the different contextual elements at play. I mean, in some ways, that sounds like education; faculty and institutions are often trying to leverage institutional support and resources (facilitating conditions) to make the classroom a space (physical or virtual) conducive to learning. I'm not too surprised that Effort Expectancy is low in this study or in consideration of its application to other technologies. Video games continue to remind me that in the right conditions, expending a lot of effort on constant failure or even confusion doesn't really matter. Still, this article does highlight and explain a significant portion of the work that I've been getting in supporting faculty with generative AI and that's creating conditions to make faculty comfortable and supported in learning and trying out this set of tools.
Tu, Y. F., & Hwang, G. J. (2023). University students’ conceptions of ChatGPT-supported learning: a drawing and epistemic network analysis. Interactive Learning Environments, 1-25.
Generative AI summary
Paragraph 1 - Strategies, Methodologies, and Theoretical Perspectives:
The article employs a unique combination of the draw-a-picture technique and epistemic network analysis (ENA) to explore university students' conceptions of ChatGPT-supported learning. This innovative methodology allows for a multifaceted exploration of student perceptions, capturing both visual and textual elements. This method, which deviates from standard quantitative and qualitative approaches, provides richer insights into students' conceptual structures regarding AI in education.
These strategies are valuable for informing teaching and learning practices. For example, the draw-a-picture technique could be utilized by educators to understand students' perceptions and misconceptions about new technologies or concepts in a more engaging and comprehensive way. Similarly, ENA can help educators and educational developers analyze complex relationships between different learning components, leading to more effective curriculum designs that resonate with students' perspectives.
Paragraph 2 - Gaps, Challenges, and Limitations:
The study highlights several limitations, including its focus on a specific demographic (Taiwanese university students), which may restrict the generalizability of its findings. Additionally, while the study provides insights into students' perceptions, it does not deeply explore the impact of ChatGPT on critical thinking or problem-solving skills, nor does it address potential over-reliance on AI tools.
Recognizing these limitations is beneficial for educational developers. It suggests the need for broader research across diverse educational contexts and a more in-depth exploration of AI's impact on various cognitive skills. Understanding these limitations can guide the development of AI-integrated curricula that balance technological support with the development of independent thinking skills.
Paragraph 3 - Applicability to Educational Developers and Faculty:
The insights and conclusions from this study are directly relevant to educational developers and faculty. Understanding the diverse perceptions of ChatGPT based on students' learning attitudes can guide the development of more tailored AI-integrated learning strategies. For instance, the study's findings can inform the creation of training programs for educators, enabling them to incorporate AI tools effectively while considering students' diverse learning attitudes.
Moreover, the study underscores the importance of a balanced approach to AI integration in education. It suggests that while AI tools like ChatGPT can significantly enhance learning, they should be used in ways that complement traditional teaching methods and foster a range of skills, including critical thinking and problem-solving. This balanced approach is crucial for developing comprehensive, effective educational experiences that leverage the benefits of AI while maintaining the core objectives of education.
My Commentary
The methodology is fascinating and one that I appreciate for its reliance on different methods of communication. However, I do wonder about folks like me who are quite limited in their drawing abilities. How do you account for someone who feels so very inadequate about their drawing abilities (e.g. can't even do stick figures well). But this summary seems more focused on the method than the results. There's not much of the findings here but the fact that it does mention a diversity of opinion about AI usage would have me going back to the article to learn more.
Obenza, B. N., Salvahan, A., Rios, A. N., Solo, A., Alburo, R. A., & Gabila, R. J. (2023). University Students' Perception and Use of ChatGPT: Generative Artificial Intelligence (AI) in Higher Education.
Generative AI summary
Paragraph 1 - Strategies, Methodologies, and Theoretical Perspectives:
The study employs a quantitative research design, utilizing a 5-point Likert scale questionnaire to assess university students' perceptions and use of ChatGPT in higher education. This approach is grounded in the Technology Acceptance Model (TAM), which emphasizes perceived usefulness and ease of use as key determinants of technology adoption. The use of TAM is particularly relevant for educational developers, as it provides a framework to understand and predict how new technologies like ChatGPT might be received in educational settings. For instance, the study's focus on students' perceptions of ChatGPT's advantages and disadvantages can inform educators about the aspects of AI technology that are most valued by learners. This insight can guide the development of AI-based educational tools that are not only technologically advanced but also align with students' expectations and needs.
Paragraph 2 - Gaps, Challenges, and Limitations:
The study highlights several challenges, particularly concerning the potential misuse of AI in educational settings, such as academic dishonesty and the dissemination of inaccurate information. These concerns are crucial for educational developers, as they underscore the need for developing ethical guidelines and integrity measures when integrating AI technologies like ChatGPT in education. Acknowledging these challenges allows educational developers to foresee potential risks and proactively develop strategies to mitigate them. For example, integrating AI literacy and ethics into the curriculum can help students understand and navigate the complexities of using AI tools responsibly.
Paragraph 3 - Applicability to Educational Developers and Faculty:
The study's findings and conclusions are highly relevant to educational developers and faculty members. The positive perception of ChatGPT among students for learning enhancement, especially in language skills, indicates a growing acceptance of AI tools in academic environments. This trend presents an opportunity for educational developers to explore innovative ways to integrate AI technologies into the curriculum. For instance, ChatGPT could be used as a supplementary tool for language learning, offering personalized practice and feedback. Additionally, the study's emphasis on the potential drawbacks of AI in education serves as a reminder for faculty to be vigilant about academic integrity and to educate students about the ethical use of AI technologies. Overall, the study provides actionable insights for faculty and educational developers to harness the potential of AI in enriching the learning experience while being mindful of its challenges.
My Commentary
If I'm following the bouncing ball, it seems that TAM has the second article's theoretical approach (Unified Theory of Acceptance and Use of Technology 2) has a bit of TAM subsumed into it. The finding that students find AI useful, particularly in language acquisition is interesting and leaves me asking what is it about AI that enhances the learning that other available tools and technologies don't do. Is it the speed, the (potential) clarity, or the absence of challenge? That last one is concerning only because some tension in learning we know is part of what helps its stickiness. That is, does it feel easier and therefore "enhance" but students don't necessarily learn better?
Meta-commentary
The new questions ('I’ve updated the appendix with them) seem to be working better; these results feel much more comprehensive and helpful in exploring the content of the article. They draw out significant enough details that it makes me want to go back to the previous articles that I've done and now apply these questions to see what else comes out.
The majority of these articles (and all the articles I've collected thus far) were from outside the United States. It's something I've noticed in a lot of the research that's come out in the last year. There can be lots of reasons for this and so I'm curious if there are substantial differences in results from these studies or where there are similarities.
I appreciated there was a mixture of quantitative and qualitative methods to draw out from students their views and approaches to AI. I gravitate toward the deeper understanding we can draw from qualitative studies about how students make sense of their experiences, yet, the quantative approaches are helpful in understanding larger trends emerging.
Across these studies, it seems like students are finding AI favorable. Though not surprising, it does create more questions for me about what are the favorable elements they are gravitating towards and how they understand the impact of their use in the short and long term.
I also encountered several new and familiar theories--to which I asked ChatGPT to provide the following summaries for so that I (and readers) could keep track of them:
Expectancy-Value Theory: This theory, often used in psychology and education, suggests that an individual's motivation to engage in a behavior is determined by two key factors: expectancy and value. Expectancy refers to a person's belief about how well they will do on an activity, while value pertains to the perceived importance or benefit of the activity to the individual. In practical terms, this theory is applied to understand and predict how people make choices, particularly in educational settings, such as a student's decision to pursue a particular course of study based on their expectations of success and the value they place on the subject.
Unified Theory of Acceptance and Use of Technology 2 (UTAUT2): This model builds upon the original UTAUT and is specifically tailored to consumer contexts. It integrates elements from various theories to predict the acceptance and use of technology. UTAUT2 includes factors such as performance expectancy, effort expectancy, social influence, facilitating conditions, hedonic motivation, price value, and habit. These factors are thought to influence a user's intention to use a technology and their actual use of it. In practice, UTAUT2 is particularly useful for businesses and developers in understanding consumer adoption and sustained use of new technologies, apps, or services.
Draw-a-Picture Technique: This is a qualitative research method used primarily in psychological and educational research. Participants are asked to draw a picture about a specific topic or experience. The resulting drawings are then analyzed to gain insights into the participant’s thoughts, feelings, and perceptions. This technique is especially useful when working with children or with populations that may have difficulty expressing themselves verbally. It is applied in various settings, from classrooms to therapeutic environments, offering a creative and non-verbal way of exploring individual perspectives and experiences.
Epistemic Network Analysis (ENA): ENA is a quantitative method used to analyze and represent the structure of interconnected ideas or concepts (epistemic elements) within a set of data, such as textual data from interviews, observations, or written texts. This analysis helps in understanding how different ideas or concepts relate to each other within a knowledge domain. ENA is often used in educational research to assess the development of complex thinking and learning in students, as well as in organizational studies to understand knowledge structures in professional environments.
Technology Acceptance Model (TAM): The Technology Acceptance Model is a framework that describes how users come to accept and use a technology. The model suggests that there are two primary factors that influence this acceptance: perceived usefulness and perceived ease of use. Perceived usefulness refers to the degree to which a person believes that a particular system would enhance their job performance, while perceived ease of use refers to the degree to which a person believes that using a system would be free of effort. TAM has been widely used in business and information systems research to predict and explain user behavior in a variety of technology-related contexts, including software adoption and e-commerce.
Appendix
Here are the updated prompts. I found that they gave much more robust answers as a whole. More importantly, I found that even when generating a response, it would go back to review the document a few times rather than just once at the initial entering of the prompt.
Prompt #1
You are an expert in educational development, skilled in analyzing research literature to extract pertinent and impactful insights for application in education.
You must assess the article according to the following categories, scoring each on a scale from 1 to 10 (10 being the highest). For each category, you must provide specific examples and detailed justifications for your scores:
Writing Intention:
Identify the type of writing (e.g., theoretical argument, empirical study, practical explanation, case study).
Determine the specific intended audience(s) of the piece.
Ease of Reading:
Analyze the use of complex terminology and assumptions about the reader's prior knowledge.
Evaluate how the structure and organization contribute to understanding.
Indicate the reader level that would find this article most accessible.
Clarity of Ideas:
Assess the straightforwardness of key points, including logical flow and explicitness.
Evaluate how well the article summarizes its main ideas.
Evidence:
Examine the types of supporting research and evidence used (qualitative, quantitative, mixed methods, primary sources, etc.).
Diverse Perspectives and Research:
Analyze the diversity in the article's approach, including interdisciplinary, global/local perspectives, intersectionality, and historical/contemporary lenses.
Assess the variety in terms of time of publication, authorship diversity, publication type, etc.
Direct Application of Ideas into Practice:
Evaluate the practicality of translating the article's discussions into teaching and learning contexts.
Remember to support each category with detailed, example-driven explanations to substantiate your assessments.
Prompt #2
Conduct a comprehensive review of the attached document, applying your expertise as an educational developer. Your analysis must be detailed, utilizing specific examples and insights from the document. Structure your response as follows:
Paragraph 1 - Study's Findings and Educational Implications:
Elaborate on the main findings of the study.
Explain the implications of these findings for the field of education, citing specific examples from the document.
Paragraph 2 - Strategies, Methodologies, and Theoretical Perspectives:
Discuss the strategies, methodologies, and theoretical perspectives presented in the article.
Describe how they could inform or enhance teaching and learning practices, with references to the document.
Paragraph 3 - Gaps, Challenges, and Limitations:
Identify any gaps, challenges, or limitations highlighted in the article.
Explain how recognizing these elements can aid educational developers in their work, using details from the article.
Paragraph 4 - Application to Educational Development:
Discuss how the insights and conclusions of the article are relevant to educational developers and faculty. Include how the recommendations in the article can be adapted or applied in the context of educational development.
Remember, your response should offer in-depth analysis and practical insights, helping educational developers assess the article's relevance and value to their field.
Prompt #3
You must conduct an in-depth review of the attached document, focusing specifically on identifying any findings or approaches that are surprising or unconventional for educational developers or educators. Your analysis should include the following elements:
Identification of Unconventional Elements:
Highlight specific findings or approaches in the document that deviate from conventional practices or expectations in educational development.
Clearly mark these elements as either 'Findings' or 'Approaches' for clarity.
Detailed Explanations:
Provide an in-depth explanation for each identified element, detailing why it is considered surprising or unconventional.
Compare these elements with standard practices or commonly accepted theories in educational development.
Contextual Analysis:
Analyze how these surprising or unconventional elements contrast with previous literature on the topic.
Discuss any potential reasons or theories behind why these elements have not been previously discussed or widely recognized in the educational field.
Your response must offer a nuanced understanding of the document, drawing attention to novel insights and potentially transformative ideas in the realm of educational development.
Prompt #4
You must review the attached document with a focus on its key points, findings, and recommendations, specifically in relation to their applicability to educational development and faculty work. Structure your detailed response into three distinct paragraphs, ensuring each contains specific details and examples from the document.
Paragraph 1 - Strategies, Methodologies, and Theoretical Perspectives:
Discuss the strategies, methodologies, or theoretical frameworks presented in the article.
Explain how these can inform or enhance teaching and learning practices, with concrete examples from the document.
Paragraph 2 - Gaps, Challenges, and Limitations:
Identify and elaborate on any gaps, challenges, or limitations in the study or its approaches as mentioned in the article.
Describe how recognizing these elements can be beneficial for educational developers in their work.
Paragraph 3 - Applicability to Educational Developers and Faculty:
Analyze how the insights and conclusions from the article are directly relevant to the roles and responsibilities of educational developers and faculty.
Illustrate the practical application of the article's recommendations in the context of educational development.
Your response must provide significant depth and insight, aiding educational developers in evaluating the article's relevance and value to their professional practices.
AI+Edu=Simplified by Lance Eaton is licensed under Attribution-ShareAlike 4.0 International
