Web scraping is legal, US appeals court reaffirms (techcrunch.com)

Article summary

A US appeals court has reaffirmed that web scraping of publicly accessible data is legal, citing a Supreme Court decision that narrowed the definition of hacking under the Computer Fraud and Abuse Act. The ruling is a win for archivists, academics, researchers, and journalists who use web scraping tools. LinkedIn had brought the case against a rival company, Hiq Labs, which was scraping personal information from public LinkedIn profiles. The court found that scraping publicly accessible data does not violate the CFAA.

Main themes

• web scraping legality
• Computer Fraud and Abuse Act
• website owner costs and concerns
• APIs and data access
• scraper behavior and ethics
• TLS fingerprinting and data poisoning
• residential proxies and CFAA violations
• interoperability and data sharing

What commenters say

• Some commenters argue that badly-behaved scrapers can cause significant system load and costs for websites, and that website owners should be able to block or limit such traffic.
• Others suggest that implementing APIs for scrapers to use could be a more efficient and cost-effective solution for both parties.
• A few commenters propose using techniques like TLS fingerprinting or poisoning data to deter scrapers, while others express concerns about the potential impact on legitimate users.
• Some argue that web scraping is a legitimate activity and that website owners should not complain about the costs of handling scraper traffic when they benefit from the visibility and reach that the internet provides.
• Others point out that using residential proxies or other methods to hide scraping activity can be seen as a violation of the CFAA.
• A few commenters suggest that building good interoperability tools and providing easy access to data through APIs or other means could reduce the need for web scraping and the associated costs and issues.
• Some argue that the issue is not with web scraping itself, but with the lack of transparency and communication between website owners and scrapers about their needs and intentions.