news.volyx.in

Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search (bitmagnet.io)

497 points by KoftaBob · 1054 days ago · 109 comments on HN

Article summary

Bitmagnet is a self-hosted BitTorrent indexer, DHT crawler, and torrent search engine that allows users to search and index torrents without relying on external trackers or indexers. The project is currently in alpha and features a DHT crawler, content classifier, and GraphQL API. The DHT crawler works by requesting metadata about each info hash it discovers, which is then further enriched by attempting to classify it and associate it with known pieces of content. This approach allows Bitmagnet to build an index of everything shared on BitTorrent over time.

Main themes

  • BitTorrent indexing
  • DHT crawling
  • Decentralized systems
  • Content moderation
  • Spam and fraud
  • Torrent search engines
  • Self-hosting and automation
  • Technical challenges and infrastructure
  • Legal and ethical implications

What commenters say

  • The effectiveness of the DHT crawler approach is questioned, with concerns about its ability to distinguish between legitimate and illegitimate content.
  • Some commenters argue that a decentralized system like Bitmagnet will be more susceptible to spam and fraud, while others believe that the torrent ecosystem is already largely automated and resistant to such issues.
  • The use of DHT crawling may incentivize spammers to target the DHT with junk and malware, potentially undermining the usefulness of the system.
  • Moderation methods, such as gatekeeping and sorting, are discussed as potential solutions to mitigate the risks associated with decentralized systems like Bitmagnet.
  • The project's potential to index illegal content is a concern, with some arguing that it could be used to distribute unlawful material.
  • Others believe that the project's focus on decentralization and automation makes it less vulnerable to abuse, and that existing torrent sites and clients already crawl the DHT.
  • The technical challenges of building a reliable and efficient DHT crawler are highlighted, including the need for uptime, sedate request pacing, and handling buggy or malicious implementations.
  • Some commenters suggest that a simple CLI tool or daemon-based approach could be effective for DHT crawling, while others argue that a more complex infrastructure may be necessary to achieve good results.