Nikolai Yakovenko: the stillborn promise of the LLM age

On this episode of Unsupervised Learning Razib talks to Nikolai Yakovenko, a returning guest to the podcast, about his new AI startup, DeepNewz, and the state of the LLM-driven AI landscape circa the summer of 2024, where we are in relation to earlier expectations and where we might be in the next decade. Yakovenko is an AI researcher who has worked at Google, Twitter and Nvidia, and is now a serial entrepreneur. He is also a competitive poker player. He currently lives in Miami, Florida, though he travels frequently to America’s numerous “ideaopolises,” from San Francisco, Austin, Boston to New York City.

Razib and Yakovenko discuss the reality that in the middle of 2024 here they are again, chatting about the world on a podcast, a scenario not everyone anticipated in the heady days of December 2022/January 2023 when the more overheated visiony tech  imaginations swirled with expectations that the advent of artificial general intelligence, the “machine god,” was imminent. Though OpenAI’s GPT 3/3.5 was a leap ahead of GPT 2, GPT 4/4o has been a less spectacular advance. One of the major unforeseen aspects of the LLM-based framework in AI has been the returns to scale in terms of training data, but the last year and a half have not seen any great quantum jumps. The paradigm-shifting revolution that was promised has not arrived. Though AI has increased productivity on the margins, and certain artistic professions and translators have been decimated, overall, it is still a technology with more promise than realized outcomes. Yakovenko points out that AI-driven music creation produces serviceable outputs, but not great masterpieces. To test this, Razib used Suno to create a song “Nikolai’s Dream” within 5 minutesmid-conversation. Though mildly catchy, Suno’s lyrical styling elicited more amusement than awe.

Yakovenko notes the importance of the LMSYS Chatbot Arena Leaderboard to get a sense of the performance of the various LLM projects. Using the feedback of participants, it produced an updated ranking of chatbot performance. The top ten models are from OpenAI, Anthropic, Google and a Chinese vendor. Yakovenko notes the compression, with a very tight distribution of scores at the top. It turns out that OpenAI is not running away from their competition contra  its brand visibility likely being two orders of magnitude greater than Anthropic’s. This brings to the fore the reality that these AI technologies have been viewed as both scientific research projects and potential business and consumer products.

Finally, Yakovenko and Razib talk about DeepNewz. While most LLM-based chatbots tend to exclude very recent data and events, Yakovenko had the idea of creating DeepNewz to aggregate and assemble the breaking news in various categories like science and sports. Instead of a top-down query of news in various categories, the idea behind DeepNewz is to both cater to your preferences in terms of what you might find interesting, but also to surface stories that you might not know you might be interested in, adding more value.

Related: David McKay: AI and the end of the world as we know it and Nick Cassimatis: fear not AI, this too shall pass.