Reddit Restricts Microsoft’s Access to Platform Search Without Payment

Reddit Inc. has implemented a new policy restricting search engines from accessing its posts and comments without formal commercial agreements. This change, effective last month, requires companies to enter into agreements with Reddit to use its data, including for web searches.

New Policy for Search Engines

Reddit's updated policy reflects a shift from its previous practice of allowing open access to its site for search engines. Chief Executive Officer Steve Huffman explained, “We can no longer be completely open because we have to be very considerate of where our data ends up and what it’s used for.” Under the new policy, any search engine or crawler that does not have a formal agreement with Reddit is now blocked from accessing the site.

Current Access Status

As a result of this policy change, Alphabet Inc.'s Google remains the only major search engine with access to Reddit’s content, thanks to a $60 million agreement signed in February.

While Reddit's partnership with Google is not exclusive, it does ensure that Google can continue to feature Reddit posts and comments in its search results. Reddit’s decision to limit access reflects concerns over the misuse of its data, including its use for training AI models and other purposes beyond mere search indexing.

Impact on Search Engines

Previously, search engines like Google were significant sources of traffic for Reddit, often driving 40% to 50% of Reddit's daily traffic. Huffman noted that while Reddit initially benefited from this traffic, the current use of its data for purposes such as AI training and resale prompted the policy shift.

"When it was used for simple search, to create simple links that would send us traffic from search engines, that was fine," Huffman said. "But now folks are using Reddit data for training, they’re reselling it, they’re doing search summaries instead of linking to us."

Negotiations with Other Companies

Reddit has been negotiating with other search engines, including Microsoft’s Bing, and AI companies like Anthropic and Perplexity, to establish commercial agreements. However, these companies have reportedly been unwilling to comply with Reddit's new content policies.

Microsoft has ceased crawling Reddit as of July 1, following Reddit's updated robots.txt file that blocks such activity. Perplexity and Anthropic have also adhered to Reddit's blocking signals, with Perplexity keeping an open invitation for future partnerships and Anthropic ceasing Reddit URL inclusion since mid-May.

Partnerships and Licensing Deals

Despite the restrictions, Reddit continues to explore opportunities for collaboration. Microsoft-backed OpenAI has signed a partnership allowing Reddit results to be featured in ChatGPT.

Additionally, Reddit has secured data licensing deals worth $203 million over the next two to three years, underscoring its ongoing commitment to monetizing its content while managing its data access policies.