Compiler Error Messages: Their Content and Accessibility in Novice Programming Environments

Arto Hellas Sami Sarsa Juho Leinonen + 4 lainnya

20 Oktober 2022

A key part of learning to program is learning to understand programming error messages. They can be hard to interpret and identifying the cause of errors can be time-consuming. One factor in this challenge is that the messages are typically intended for an audience that already knows how to program, or even for programming environments that then use the information to highlight areas in code. Researchers have been working on making these errors more novice friendly since the 1960s, however progress has been slow. The present work contributes to this stream of research by using large language models to enhance programming error messages with explanations of the errors and suggestions on how to fix them. Large language models can be used to create useful and novice-friendly enhancements to programming error messages that sometimes surpass the original programming error messages in interpretability and actionability. These results provide further evidence of the benefits of large language models for computing educators, highlighting their use in areas known to be challenging for students. We further discuss the benefits and downsides of large language models and highlight future streams of research for enhancing programming error messages.

J. Prather Brett A. Becker Nigel Bosch + 3 lainnya

22 Februari 2022

As computing students progress through their studies they become proficient with multiple programming languages. Prior work investigating language transitions for novices has tended to analyze program artifacts rather than explore the benefits and difficulties as perceived by students in their own words, and has often overlooked problems that may arise in switching paradigms or where familiar syntax has a different meaning in the new language. In this paper, we ask students to reflect on the transition from an interpreted language and environment (MATLAB) to a compiled language (C), prompting comments on the aspects of learning the new language that they found both easier and harder. Analysis of over 70,000 words written by 771 students revealed that the highest-performing students expressed more negative sentiments towards the language transition -- a surprising result that we hypothesize is explained by their generally stronger metacognitive skills. We also report the most common difficulties described by students, which include challenges with syntax, error messages, and the process of compilation, and suggest teaching practices that might help students as they transition to a new programming language.

F. Hermans

1 Agustus 2020

One of the aspects of programming that learners often struggle with is the syntax of programming languages: remembering the right commands to use and combining those into a working program. Prior research demonstrated that students submit source code with syntax errors in 73% of cases and even the best students do so in 50% of cases. An analysis of 37 million compilations by 250.000 students found that the most common error was a syntax error, which occurred in almost 800.000 compilations. It was also found that Java and Perl are not easier to understand than a programming language with randomly generated keywords, stressing the difficulties that novices face in understanding syntax. This paper presents Hedy: a new way of teaching the syntax of a programming language to novices, inspired by educational methods by which punctuation is taught to children. Hedy starts as a simple programming language without any syntactic elements such as brackets, colons or indentation. The rules slowly and gradually change until the novices are programming in Python. Hedy is evaluated on 9714 programs.

Dominic Lohr H. Keuning Natalie Kiesler

31 Agustus 2023

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices.

Sverrir Thorgeirsson Z. Su

10 Oktober 2021

Empirical research suggests that programming language syntax is a common impediment for beginners, a concern that is mitigated to a varying degree by visual programming. In this paper, we introduce a novel visual programming language that is founded on program synthesis and the programming-by-demonstration paradigm. By using an intuitive visual syntax, we show how we can meet our primary goal of providing support to computer programming novices in exploring foundational programming concepts. We present the language's current and planned use in computer science education, provide preliminary evidence for its effectiveness, and discuss its future possibilities.