ImageNet contains naturally occurring Apple NeuralHash collisions (blog.roboflow.com)

Article summary

A search through the ImageNet dataset uncovered two naturally occurring image pairs with identical NeuralHashes, a perceptual hashing model used by Apple for client-side CSAM detection. This raises questions about the real-world false-positive rate and potential attack surfaces. The findings suggest a slightly higher false-positive rate than Apple's reported rate. The discovery of these collisions highlights the limitations and potential vulnerabilities of the NeuralHash system.

Main themes

• NeuralHash collisions
• CSAM detection
• user privacy
• database security
• government abuse
• technical limitations
• precedent for future surveillance

What commenters say

• The introduction of NeuralHash creates a new attack surface that could be exploited by governments or other entities to compromise user privacy.
• The system's reliance on a database of hashes raises concerns about the potential for abuse or manipulation of the database.
• The false-positive rate of the system is a significant concern, and the discovery of natural collisions in the ImageNet dataset suggests that it may be higher than Apple's reported rate.
• The use of NeuralHash for CSAM detection is a violation of user privacy, regardless of the system's technical merits or limitations.
• The system's design and implementation make it difficult to trust that it will not be used for purposes other than CSAM detection, such as political censorship.
• The fact that Apple can access user photos on their servers because they are not end-to-end encrypted is a significant security risk.
• The introduction of NeuralHash sets a precedent for the use of similar systems in the future, which could have far-reaching consequences for user privacy and security.