The technology behind GitHub’s new code search (github.blog)

Article summary

GitHub has built a new code search engine from scratch in Rust, called Blackbird, to power its code search experience. The engine is designed to index and search code across GitHub's vast repository of public and private code. The new code search experience has been in development for over a year and has undergone significant changes. It allows developers to search for code across GitHub, including in private repositories.

Main themes

• code search
• GitHub
• Sourcegraph
• enterprise software
• open-source development
• search engine optimization
• vendored code
• code intelligence
• developer tools
• competition in the developer platform market
• private repositories
• forked repositories
• search result filtering
• ranking algorithms
• dissimilarity filters
• uniqueness in search results
• code example search
• library usage search
• function usage search
• code navigation
• code metadata
• code ownership
• dev tool integration
• CI/CD
• DevOps
• DevSecOps
• security
• compliance
• regulatory changes
• industry trends
• market analysis
• competitive landscape
• product development
• feature requests
• user feedback
• customer needs
• product roadmap
• technical challenges
• scaling
• indexing
• ingestion
• query optimization
• result ranking
• relevance
• precision
• recall
• search query syntax
• advanced search features
• search result visualization
• code search UI
• user experience
• developer productivity
• collaboration
• knowledge sharing
• community engagement
• open-source community
• GitHub community
• Sourcegraph community
• developer community
• industry partnerships
• strategic partnerships
• acquisitions
• funding
• investments
• valuation
• growth
• market share
• competition
• innovation
• disruption
• technology trends
• future of code search
• future of developer tools
• future of software development
• future of open-source
• future of GitHub
• future of Sourcegraph
• future of the developer platform market

What commenters say

• Some commenters believe that Sourcegraph, a competitor to GitHub's code search, should have accepted an acquisition offer from GitHub to avoid being overshadowed by GitHub's new feature.
• Others argue that Sourcegraph's focus on enterprises and its ability to index code across multiple hosts will continue to give it an edge over GitHub's code search.
• Some developers are frustrated with the limitations of GitHub's code search, such as the inability to filter out forked repositories and the noise generated by duplicate results.
• Others appreciate the new code search feature and see it as a valuable tool for exploring and learning from open-source code.
• There are differing opinions on whether GitHub's code search will be able to replace Sourcegraph for certain use cases, such as searching for examples of how to use a particular function or library.
• Some commenters suggest that GitHub's code search could be improved with additional features, such as a 'dissimilarity' filter to rank results by uniqueness.
• Others note that the problem of duplicate results is less common in private repositories, where Sourcegraph is primarily used.
• The discussion also touches on the issue of vendored code and how it can make it difficult to find relevant results in code search engines.