Why Software Factories Fail (or: harness engineering is not enough) (github.com)

Article summary

The article discusses the limitations of software factories that rely heavily on AI coding agents, arguing that they can lead to decreased code quality and increased maintenance issues over time. The author shares their own experience with implementing a 'lights-off' software factory, where human code review was eliminated, and the resulting problems that arose. The article highlights the importance of human oversight and code review in maintaining codebase quality. The author also notes that current benchmarks for evaluating AI models are insufficient for measuring their ability to maintain codebase quality.

Main themes

• Software factories
• AI coding agents
• Code quality
• Human oversight
• Code review
• AI limitations

What commenters say

• Some commenters argue that automated code review is not a replacement for human review, as it lacks the nuance and understanding that a human reviewer can provide.
• Others believe that automated code review can be effective in catching bugs, but may not be sufficient for ensuring code quality and maintainability.
• There is a disagreement among commenters about the role of code review, with some seeing it as a quality control mechanism and others viewing it as a way to assess the 'taste' and overall quality of the code.
• Some argue that the use of AI coding agents will inevitably lead to a decrease in code quality, while others believe that it can be a useful tool when used in conjunction with human oversight and review.
• A few commenters suggest that the focus should be on creating better tools and processes for human reviewers, rather than relying solely on automated solutions.
• There is a concern that the increasing use of AI coding agents will lead to a loss of understanding and ownership of the codebase among human engineers.
• Some commenters believe that the market demand for speed and problem-solving ability will drive the adoption of AI coding agents, regardless of the potential risks to code quality.
• Others argue that human code review is essential for ensuring accountability and responsibility among engineers, and that automated solutions can be used to augment but not replace this process.