Apple’s Mistake (stratechery.com)

Article summary

Apple has introduced new child safety features that use on-device machine learning to scan for and report child sexual abuse material (CSAM) in iCloud Photos. The system is designed to preserve user privacy by not scanning content in the cloud, but some argue that this approach is still an invasion of privacy. The move is seen as a response to government pressure to proactively look for and report CSAM. Apple's approach differs from Facebook's, which scans all images sent to and over its service.

Main themes

• CSAM scanning
• user privacy
• government pressure
• censorship and control
• liability and regulation
• security vs. privacy
• surveillance and trust

What commenters say

• The introduction of CSAM scanning on Apple devices is a significant invasion of user privacy and sets a dangerous precedent for future surveillance.
• Apple's approach is a necessary measure to protect children from exploitation and abuse, and the company has taken steps to preserve user privacy.
• The capability to scan for CSAM on devices could be expanded to scan for other types of content, such as illegal firearms or copyrighted material, which raises concerns about censorship and control.
• The fact that Apple controls the algorithm and database used for CSAM scanning means that the company has the power to decide what is and is not considered acceptable content.
• Some commenters argue that Apple's move is motivated by a desire to minimize liability and avoid investing in trust-and-safety teams, rather than a genuine concern for user privacy or child safety.
• Others see the move as a response to government pressure to proactively look for and report CSAM, and argue that it is a necessary compromise to avoid more invasive regulations.
• There are concerns that the CSAM scanning system could be used to target and suppress certain groups or individuals, particularly in countries with authoritarian governments.
• The discussion around Apple's CSAM scanning feature highlights the tension between privacy and security, and the need for a nuanced and informed debate about the trade-offs between these competing values.