How Graduate Computing Students Search When Using an Unfamiliar Programming Language

André C. Hora

1 Mei 2021

Developers often search for software resources on the web. In practice, instead of going directly to websites (e.g., Stack Overflow), they rely on search engines (e.g., Google). Despite this being a common activity, we are not yet aware of what developers search from the perspective of popular software development websites and what search results are returned. With this knowledge, we can understand real-world queries, developers’ needs, and the query impact on the search results. In this paper, we provide an empirical study to understand what developers search on the web and what they find. We assess 1.3M queries to popular programming websites and we perform thousands of queries on Google to explore search results. We find that (i) developers’ queries typically start with keywords (e.g., Python, Android, etc.), are short (3 words), tend to omit functional words, and are similar among each other; (ii) minor changes to queries do not largely affect the Google search results, however, some cosmetic changes may have a non-negligible impact; and (iii) search results are dominated by Stack Overflow, but YouTube is also a relevant source nowadays. We conclude by presenting detailed implications for researchers and developers.

Chris Parnin Titus Barik Colton Botta + 1 lainnya

27 Juni 2020

Once a programmer knows one language, they can leverage concepts and knowledge already learned, and easily pick up another programming language. But is that always the case? To understand if programmers have difficulty learning additional programming languages, we conducted an empirical study of Stack Overflow questions across 18 different programming languages. We hypothesized that previous knowledge could potentially interfere with learning a new programming language. From our inspection of 450 Stack Overflow questions, we found 276 instances of interference that occurred due to faulty assumptions originating from knowledge about a different language. To understand why these difficulties occurred, we conducted semi-structured interviews with 16 professional programmers. The interviews revealed that programmers make failed attempts to relate a new programming language with what they already know. Our findings inform design implications for technical authors, toolsmiths, and language designers, such as designing documentation and automated tools that reduce interference, anticipating uncommon language transitions during language design, and welcoming programmers not just into a language, but its entire ecosystem.

Pierre Weill-Tessier A. Costache Maksymilian Sekula + 2 lainnya

1 Agustus 2022

Objectives. Java is a popular programming language for use in computing education, but it is difficult to get a wide picture of the issues that it presents for novices; most studies look only at the types or frequency of errors. In this observational study, we aim to learn how novices use different features of the Java language. Participants. Users of the BlueJ development environment have been invited to opt in to anonymously record their activity data for the past 8 years. This dataset is called Blackbox, which was used as the basis for this study. BlueJ users are mostly novice programmers, predominantly male, with a median age of 16 years. Our data subset featured approximately 225,000 participants from around the world. Study Methods. We performed a secondary data analysis that used data from the Blackbox dataset. We examined over 320,000 Java projects collected over the course of 8 years and used source code analysis to investigate the prevalence of various specifically selected Java programming usage patterns. As this was an observational study without specific hypotheses, we did not use significance tests. Instead, we present the results themselves with commentary, having applied seasonal trend decomposition to the data. Findings. We found many long-term trends in the data over the course of the 8 years, most of which were monotonic. There was a notable reduction in the use of the main method (common in Java but unnecessary in BlueJ) and a general reduction in the complexity of the projects. We find that there is only a small number of frequently used types: int, String, double, and Boolean, but also a wide range of other infrequently used types. Conclusions. We find that programming usage patterns gradually change over a long period of time (a period in which the Java language was not seeing major changes) once seasonal patterns are accounted for. Any changes are likely driven by instructors and the changing demographics of programming novices. The novices use a relatively restricted subset of Java, which implies that designers of languages specifically targeted at novices can satisfy their needs with a smaller set of language constructs and features. We provide detailed recommendations for the designers of educational programming languages and supporting development tools.

J. Prather Brett A. Becker Nigel Bosch + 3 lainnya

22 Februari 2022

As computing students progress through their studies they become proficient with multiple programming languages. Prior work investigating language transitions for novices has tended to analyze program artifacts rather than explore the benefits and difficulties as perceived by students in their own words, and has often overlooked problems that may arise in switching paradigms or where familiar syntax has a different meaning in the new language. In this paper, we ask students to reflect on the transition from an interpreted language and environment (MATLAB) to a compiled language (C), prompting comments on the aspects of learning the new language that they found both easier and harder. Analysis of over 70,000 words written by 771 students revealed that the highest-performing students expressed more negative sentiments towards the language transition -- a surprising result that we hypothesize is explained by their generally stronger metacognitive skills. We also report the most common difficulties described by students, which include challenges with syntax, error messages, and the process of compilation, and suggest teaching practices that might help students as they transition to a new programming language.

Guanlin Wang Sanka Rasnayaka Ganesh Neelakanta Iyer + 1 lainnya

29 Januari 2024

Large Language Models (LLMs) represent a leap in artificial intelligence, excelling in tasks using human language(s). Although the main focus of general-purpose LLMs is not code generation, they have shown promising results in the domain. However, the usefulness of LLMs in an academic software engineering project has not been fully explored yet. In this study, we explore the usefulness of LLMs for 214 students working in teams consisting of up to six members. Notably, in the academic course through which this study is conducted, students were encouraged to integrate LLMs into their development tool-chain, in contrast to most other academic courses that explicitly prohibit the use of LLMs.In this paper, we analyze the AI-generated code, prompts used for code generation, and the human intervention levels to integrate the code into the code base. We also conduct a perception study to gain insights into the perceived usefulness, influencing factors, and future outlook of LLM from a computer science student’s perspective. Our findings suggest that LLMs can play a crucial role in the early stages of software development, especially in generating foundational code structures, and helping with syntax and error debugging. These insights provide us with a framework on how to effectively utilize LLMs as a tool to enhance the productivity of software engineering students, and highlight the necessity of shifting the educational focus toward preparing students for successful human-AI collaboration.CCS CONCEPTS• Software and its engineering → Software development techniques; • Applied computing → Education.