LLMs corrupt your documents when you delegate (arxiv.org)

Article summary

A study introduced DELEGATE-52 to evaluate the readiness of Large Language Models (LLMs) in delegated workflows, revealing that current models degrade documents during delegation, with even frontier models corrupting an average of 25% of document content by the end of long workflows. The experiment involved 19 LLMs and simulated long delegated workflows that required in-depth document editing across 52 professional domains. The study found that LLMs introduce sparse but severe errors that silently corrupt documents, compounding over long interaction. The degradation severity is exacerbated by document size, length of interaction, or presence of distractor files.

Main themes

• LLM limitations
• Document degradation
• Delegated workflows
• AI reliability
• Error accumulation
• Human-AI collaboration

What commenters say

• LLMs are prone to making mistakes on every turn, and these mistakes can have little to no apparent connection to the difficulty of the task or the prevalence of the data in the training set.
• The use of LLMs can lead to a loss of nuance and precision in documents, making them less reliable for tasks that require high accuracy.
• Humans are also fallible and can make mistakes, but they have cognitive awareness of which tasks are critical and need more checking and re-checking, whereas LLMs do not.
• The degradation of documents by LLMs is similar to the effect of repeatedly saving a JPEG image, where each pass slightly degrades the quality until it becomes unrecognizable.
• Some argue that LLMs are useful despite their limitations, and that their flaws are a price to pay for their scalability and cost-effectiveness.
• Others believe that the use of LLMs can lead to a lack of accountability and a tendency to overlook errors, and that humans should be more cautious when relying on AI-generated content.
• There is a need for more research on the types of errors that LLMs make and how they can be mitigated, in order to improve the reliability of AI-generated content.
• The comparison between human and LLM performance is not always straightforward, and humans may degrade documents further than LLMs if they use the same technique as the LLMs in the study.