ChatGPT for Teaching and Learning: An Experience from Data Science Education

Linjun Zhang James Y. Zou Xinming Tu + 1 lainnya

6 Juli 2023

The rapid advances of large language models (LLMs), such as ChatGPT, are revolutionizing data science and statistics. These state-of-the-art tools can streamline complex processes. As a result, it reshapes the role of data scientists. We argue that LLMs are transforming the responsibilities of data scientists, shifting their focus from hands-on coding, data-wrangling and conducting standard analyses to assessing and managing analyses performed by these automated AIs. This evolution of roles is reminiscent of the transition from a software engineer to a product manager. We illustrate this transition with concrete data science case studies using LLMs in this paper. These developments necessitate a meaningful evolution in data science education. Pedagogy must now place greater emphasis on cultivating diverse skillsets among students, such as LLM-informed creativity, critical thinking, AI-guided programming. LLMs can also play a significant role in the classroom as interactive teaching and learning tools, contributing to personalized education. This paper discusses the opportunities, resources and open challenges for each of these directions. As with any transformative technology, integrating LLMs into education calls for careful consideration. While LLMs can perform repetitive tasks efficiently, it's crucial to remember that their role is to supplement human intelligence and creativity, not to replace it. Therefore, the new era of data science education should balance the benefits of LLMs while fostering complementary human expertise and innovations. In conclusion, the rise of LLMs heralds a transformative period for data science and its education. This paper seeks to shed light on the emerging trends, potential opportunities, and challenges accompanying this paradigm shift, hoping to spark further discourse and investigation into this exciting, uncharted territory.

Natalie Kiesler D. Schiffner

15 Agustus 2023

This paper investigates the performance of the Large Language Models (LLMs) ChatGPT-3.5 and GPT-4 in solving introductory programming tasks. Based on the performance, implications for didactic scenarios and assessment formats utilizing LLMs are derived. For the analysis, 72 Python tasks for novice programmers were selected from the free site CodingBat. Full task descriptions were used as input to the LLMs, while the generated replies were evaluated using CodingBat's unit tests. In addition, the general availability of textual explanations and program code was analyzed. The results show high scores of 94.4 to 95.8% correct responses and reliable availability of textual explanations and program code, which opens new ways to incorporate LLMs into programming education and assessment.

Cemal Yilmaz Orçun Çetin Khadija Hanifi

22 Oktober 2023

ChatGPT, an increasingly popular Large Language Model (LLM), has found widespread acceptance, especially among the younger generation, who rely on it for various tasks, such as comprehending complex course materials and tackling homework assignments. This surge in interest has drawn the attention of researchers, leading to numerous studies that delve into the advantages and disadvantages of the upcoming LLM dominant era. In our research, we explore the influence of ChatGPT and similar models on the field of software engineering, specifically from the perspective of software engineering students. Our main objective is to gain valuable insights into their usage habits and opinions through a comprehensive survey. The survey encompassed diverse questions, addressing the specific areas where ChatGPT was utilized for assistance and gathering students’ reflections on each aspect. We found that ChatGPT has garnered widespread acceptance among software engineering students, with 93% of them utilizing it for their projects. These students expressed satisfaction with the level of assistance provided, and most intend to continue using it as a valuable tool in their work. During our investigation, we also assessed the students’ awareness of the underlying technologies behind ChatGPT. Approximately half of the students demonstrated awareness of these technologies, while 38.7% had made extra efforts to explore prompt engineering to enhance ChatGPT’s productivity. However, an important finding was that 90.6% of the students reported experiencing hallucinations during their interactions with ChatGPT. These hallucinations were shared as examples, raising significant concerns that warrant further exploration and mitigation. Moreover, we delved into potential improvements and gathered valuable recommendations, which could help ChatGPT to become even more effective and dependable in its applications.

Adalbert Gerald Soosai Raj Xinyi Ai Meenakshi Syamkumar + 2 lainnya

7 Maret 2024

ChatGPT is a conversational AI platform that can produce code to solve problems when provided with a natural language prompt. Prior work on similar AI models has shown that they perform well on typical intro-level Computer Science problems. However, little is known about the performance of such tools on Data Science (DS) problems. In this work, we assess the performance of ChatGPT on assignments from three DS courses with varying difficulty levels. First, we apply the raw assignment prompts provided to the students and find that ChatGPT performs well on assignments with dataset(s) descriptions and progressive question prompts, which divide the programming requirements into sub-problems. Then, we perform prompt engineering on the assignments for which ChatGPT had low performance. We find that the following prompt engineering techniques significantly increased ChatGPT's performance: breaking down abstract questions into steps, breaking down steps into multiple prompts, providing descriptions of the dataset(s), including algorithmic details, adding specific instructions to entice specific actions, and removing extraneous information. Finally, we discuss how our findings suggest potential changes to curriculum design of DS courses.

Dominic Lohr H. Keuning Natalie Kiesler

31 Agustus 2023

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices.