Google refuses to reinstate account after man took medical images of son’s groin (theguardian.com)

Article summary

Google refused to reinstate a man's account after it was flagged for containing child sexual abuse material (CSAM), despite the images being medical in nature and taken with the intention of sending to a doctor. The man had taken pictures of his son's groin to send to a doctor after noticing inflammation, and the doctor used the image to diagnose and prescribe antibiotics. Google's automated detection system flagged the images as CSAM, and the company stood by its decision despite the man being cleared of any wrongdoing by law enforcement. The incident highlights the limitations and potential pitfalls of automated detection systems.

Main themes

• Automated detection systems
• False positives
• Account bans
• Privacy and due process
• Content moderation
• Legal change
• Regulation and oversight
• Balancing protection and privacy

What commenters say

• The incident demonstrates the need for legal change to protect users from false positives and account bans.
• Google's decision to stand by its ban despite evidence of the images being medical in nature is seen as overly cautious and potentially harmful.
• The use of automated detection systems can lead to false positives and have serious consequences for individuals, including damage to their reputation and livelihood.
• Some argue that having a formal process for reinstating banned accounts could make the scanning more acceptable, while others believe that scanning itself is the problem.
• The incident raises questions about the balance between protecting users from harmful content and respecting their privacy and due process.
• There are concerns that the use of AI and automation in content moderation can lead to a lack of human oversight and accountability.
• Some commenters suggest that companies like Google should be subject to a 'universal service obligation' to prevent them from refusing service to users without just cause.
• The incident highlights the potential for conflicts between local and federal laws, and the need for clearer guidelines and regulations around content moderation.