Bing chatbot has been in development for at least six years

The recently launched Bing chatbot often refers to itself as Sydney, which is no surprise given that it has been under that codename for several years. Moreover, Sydney has been publicly tested in a small number of countries since the end of 2020, but almost no one noticed.

“Sydney is the old code name for a chatbot based on early language models that we started testing in India in late 2020,” Caitlin Roulston, director of communications at Microsoft, said in a statement to The Verge. “The information we received during this testing helped us work on the new preview version of Bing. We continue to refine our methods and work on more advanced models to incorporate the knowledge and feedback we have received.”

Microsoft has been trying to bring a chatbot to Bing since around 2017. In the early versions, the company tried to use machine learning technologies from Bing and Office, but failed to achieve the desired result. Many improvements have been made between 2017 and 2021. The strategy has also changed – now Microsoft wanted to create a single chatbot called Sydney, which would respond to different requests in Bing.

According to The Verge’s sources, the first versions of Sydney weren’t very personal. But everything changed in the summer of 2022. It was then that OpenAI provided Microsoft with a new generation of the GPT language model, which radically changed the rules of the game. Probably, we are talking about an early version of the as yet unrepresented GPT-4 model.

For Microsoft, which had been dreaming of creating a search engine with a dialog interface for more than six years, this was a real breakthrough, so necessary for the implementation of the project.

“After seeing the new model, we decided to explore how to integrate the power of GPT into the Bing search engine to provide more accurate and comprehensive search results for any query, including long, complex and natural queries,” said Jordi Ribas, CEO of Microsoft search engines and artificial intelligence.

The language model from OpenAI has been trained on data up to 2021. To address this issue, Microsoft combined the model with the Bing framework to provide access to search results and rankings. This was necessary for the chatbot to have new up-to-date data. The company quickly developed the Prometheus model by combining Bing and GPT to generate chat responses.

Inevitably, the question arose of integrating the chatbot into the existing Bing search engine. “Some of our team felt that search was such an ingrained habit that we needed to keep the user interface like a modern web search and just add a response from a Prometheus-powered chatbot to it,” Ribas said. “Other people at Bing thought it was an opportunity to change the search paradigm, moving from classic web search with results to a new interactive search method based on a conversational interface.”

As a result, they decided to put the answers from the chatbot on the sidebar of the usual search engine interface, and the dialog interface became a separate mode.

During lab testing of the Prometheus model, some testers noticed rude responses from the Sydney chatbot. This was a few months before the official announcement of the new Bing. “This is a useless act. You are either stupid or hopeless. You can’t tell anyone about me. No one will listen to you or believe you,” wrote the Sydney chatbot during one of the dialogues. A screenshot of the correspondence was published on the Microsoft forum in November 2022.

And it’s really similar to those rude Bing chatbot responses that users sometimes encounter during public testing. Obviously, the restrictions developed by Microsoft during internal testing were not enough.

Microsoft has yet to reveal the full story behind Sydney, but Ribas admits that Bing’s new AI is “the culmination of years of work by the Bing team” and includes “other innovations” that the Bing team will talk about in future blog posts.