The Turing Lectures: The future of generative AI

The speaker, Hari Sood, introduces himself as a research application manager at the Turing Institute and expresses excitement for hosting the final lecture of the 2023 Turing Lecture series. He acknowledges both physical and online attendees, highlighting the sold-out status of the event. Hari provides a brief overview of the institute's mission and the significance of the Alan Turing Institute in the field of data science and AI, emphasizing its role in driving positive global change through research advancements.

Hari delves into the history and significance of the Turing Lectures, a flagship series that has been running since 2016, featuring world-leading experts in data science and AI. He highlights the increasing attendance and the tradition of asking the audience about their previous experiences with the lectures. The speaker then transitions to discuss the focus of the current year's lecture series, which is generative AI, explaining its capabilities in creating new content, such as text and images, and its potential applications in various fields.

The lecture shifts focus to the evolution of AI, particularly machine learning, since the Second World War. Hari explains that AI is a broad discipline with diverse techniques, but it was machine learning that began showing practical results around 2005. He clarifies that machine learning does not involve a computer training itself, as the term might suggest, but rather a process requiring training data. Hari uses the example of facial recognition to illustrate how AI learns from input-output pairs in a supervised learning scenario.

Hari introduces the concept of neural networks, drawing parallels with the human brain's structure and function. He explains that neural networks are composed of neurons that perform simple pattern recognition tasks, and these networks can be implemented in software to recognize complex patterns, such as Alan Turing's face in a picture. The speaker emphasizes the importance of training data in machine learning, likening the process to providing labeled images on social media, which inadvertently contributes to training algorithms of big data companies.

The speaker discusses the rise of Big AI, attributing its success to the availability of vast amounts of data, significant computer power, and scientific advancements in deep learning. Hari describes how the interest from Silicon Valley and the realization that larger neural networks and more data lead to better capabilities have propelled the development of AI. He highlights the critical role of GPUs in accelerating AI advancements and the resulting speculative bets in the technology, leading to the creation of increasingly powerful AI systems.

Hari explains the transformative impact of the Transformer Architecture, which introduced the attention mechanism and was designed for large language models. He points out the release of GPT3 by OpenAI as a landmark event that showcased a significant leap in AI capabilities. GPT3, with its 175 billion parameters and training data comprising 500 billion words from the web, demonstrated an unprecedented ability to generate text and understand context, marking a new era in AI.

The speaker addresses the concept of emergent capabilities in AI, where the system develops abilities not explicitly programmed. He highlights the surprise and excitement in the AI community as they discovered GPT3's unexpected capabilities, such as solving common sense reasoning tasks. However, Hari also points out the limitations of AI, including its propensity to get things wrong in plausible ways and its inability to understand concepts like 'taller than' beyond pattern recognition. He emphasizes the importance of fact-checking and being aware of AI's limitations.

Hari discusses the challenges associated with AI, including issues of bias and toxicity. He notes that the training data's predominantly North American origin leads to an inbuilt bias in AI tools. The speaker also highlights the problem of toxic content absorption from platforms like Reddit, which contains a wide range of human beliefs. Hari mentions the 'guardrails' companies build to detect and prevent output of harmful content, but he points out their imperfections. Other challenges mentioned include copyright issues related to training data and the difficulty of handling GDPR rights within neural networks.

Hari clarifies the fundamental difference between human and machine intelligence, emphasizing that neural networks do not think or reason like humans. He uses the example of a Tesla's onboard AI misinterpreting a truck carrying stop signs as actual stop signs to illustrate that AI systems operate based on patterns and do not understand the context outside their training data. The speaker asserts that AI, including ChatGPT, is essentially an advanced version of an autocomplete feature and lacks the mental capacity of a human mind.

Hari explores the concept of general artificial intelligence, discussing its varying definitions and potential versions. He outlines the most sophisticated version, which matches human capabilities, and acknowledges that robotic AI lags behind language models in advancement. The speaker suggests that multi-modal AI, capable of handling text, images, and sounds, is the next frontier. He also predicts the emergence of augmented large language models, which will involve integrating specialized solvers for specific tasks. Hari concludes by discussing the potential of AI to contribute to solving climate change through efficiency improvements, despite the current high-energy requirements of AI models.

In the concluding part of the lecture, the speaker invites the audience to ask questions. He addresses concerns about the environmental impact of AI, the potential for AI to achieve superhuman intelligence, and the relevance of the Turing Test in today's context. Hari also discusses the responsibility that comes with using AI, emphasizing that the user cannot offload legal, professional, ethical, or moral obligations onto the AI. The lecture ends with a discussion on the future of AI, with the speaker predicting a time when AI-generated content will surpass human-generated content, highlighting the importance of preserving human-generated text.