Overview

Software bugs are costly. Beyond the developer effort spent resolving them, they can trigger failures with substantial financial consequences (Federal Communications Commission 2024; Business Insider 2024; Zahn et al. 2024). Resolving such bugs is therefore a challenge in software development (Zou et al. 2020), and a time-consuming one. Indeed, developers are estimated to spend roughly half of their time on debugging and bug resolution (Britton et al. 2013). This process is cognitively demanding, as it requires a deep understanding of the codebase (Yoon and Garcia 1998; Hu et al. 2024).

To assist developers with this activity, automated program repair (APR) has been proposed (Gazzola, Micucci, and Mariani 2019). Traditionally, these systems have relied on information retrieval and template-based solutions. However, these approaches are often rigid, making it difficult to extrapolate them to unseen scenarios. Large language models (LLMs) and LLM-based systems offer great promise in this regard. Such overly parameterized models are trained on vast datasets, and possess stronger generalization capabilities. Methods like Agentless (Xia et al. 2025) and SWE-agent (Yang et al. 2024) demonstrate a remarkable aptitude for resolving issues compared to classical APR methods.

However, their massive scale makes them expensive to operate. This cost is multifaceted: computational (requiring extensive arithmetic operations), monetary (demanding high-end computing infrastructure), and environmental (correlating increased compute with higher emissions). It is well established that LLMs are over-parameterized, meaning, not every neuron contributes equally to a given computation, and ablating a subset can incur only minimal performance degradation (Voita et al. 2019; Sandoval-Segura et al. 2026; Dong, Chen, and Chi 2024). Moreover, the set of components that matter is often input-dependent, a phenomenon known as contextual sparsity (Liu et al. 2023), and prior work has localized specific skills or functions to identifiable groups of neurons (Wang et al. 2022).

Separately, automated bug resolution follows a structured workflow. First, the process starts by localizing the bug, reproducing it, issuing a patch, and finally running tests to verify the fix and check for regressions. In light of these two observations, we hypothesize that distinct regions of the network could be activated during the phases of the APR process. A pictorial representation of this hypothesis is illustrated in Figure 1. If this holds, the phase becomes a conditioning signal for sparsity. This is because an APR agent always knows which phase it is currently executing, we could selectively activate (or prune) only the relevant regions per phase, reducing the floating-point operations and latency of each step without sacrificing repair accuracy.

Main hypothesis that states that regions of the network are conditioned on the phase of automated program repair.

Research Questions

We organize the investigation around three research questions that move from existence to exploitation:

Approach

The following is a tentative methodology which can be refined as the project progresses.

Defining a phase.

We take a phase to be an agent step (or prompt role) within an established APR scaffold such as Agentless (Xia et al. 2025) or SWE-agent (Yang et al. 2024): bug localization, reproduction, patch generation, and test/verification. The main goal is to investigate if there exist consistent, reusable sub-circuits within a single model serving all phases. A minimal ReACT-based agent can also be implemented, if needed.

Models.

Neuron-level inspection requires access to model weights, so we focus on open-weights code LLMs (e.g., Qwen2.5-Coder or DeepSeek-Coder).

Data and harness.

SWE-bench (Verified or Lite), Defects4J.

Units and attribution.

The units of analysis are MLP neurons and attention heads, taken per layer. To attribute components to phases, we plan to combine (i) mean/peak activation statistics per phase, (ii) lightweight probing classifiers that predict the phase from activations, (iii) activation-overlap metrics across phases, and (iv) causal ablation to test whether the implicated components are necessary for a given phase. Such methods have been partially covered in the work of (Sandoval-Segura et al. 2026).

Feasibility and Resources

Because the study targets open-weights models and reuses existing scaffolds and benchmarks, it should be tractable. Access to GPUs is also available.

Logistics and Collaboration

Minimum Requirements

An ideal candidate should meet the following minimum requirements:

Nice to Have

The following are qualifications that are nice-to-have, but can definitely be acquired throughout the project:

References