Research Retrospectives: An interview with David Ha, Co-founder and CEO of Sakana AI

The speaker, David, introduces himself as a research scientist at Google, with a background in complex systems and machine learning. He shares his journey from managing director at Goldman Sachs to exploring AI as a hobby, which led to creating interactive web visualizations. His initial projects involved neural networks and chaotic systems, which he hosted on a personal blog. David's work gained recognition, leading to an exhibition in Korea and his transition from finance to research.

David discusses the importance of visualization in research, emphasizing its role in interpreting machine learning results beyond mere numbers. He highlights the shift from static PDFs to interactive web demos and the insights gained from engaging with models. David also shares his experience with neural network agents in games and the integration of evolutionary strategies in his work, leading to publications and a deeper understanding of model capabilities.

The conversation delves into David's fascination with evolutionary strategies, which he sees as a way to approach complex systems with less rigidity than traditional machine learning. He talks about his early experiments with neural network morphology and the influence of his work on neural architecture search. David also explores the generative aspects of machine learning, combining evolutionary concepts with generative models like GANs.

David shares his views on the potential of evolutionary strategies in reinforcement learning, discussing their simplicity and scalability. He contrasts this with the challenges of credit assignment in RL and the focus on sample efficiency. His interest lies in solving novel problems rather than refining existing solutions, which he believes evolutionary strategies can facilitate effectively.

The discussion highlights the separation and potential collaboration between the neuroevolution, machine learning, and RL communities. David notes the efforts of individuals like Jeff Clune and Ken Stanley in bridging these gaps. He also mentions the importance of conferences like NeurIPS and the role of Google in publishing research that combines evolutionary methods with machine learning.

David talks about 'EvoJax', a library of evolution algorithms implemented in Jax, and its potential to accelerate research pipelines. He emphasizes the importance of having a common programming language to facilitate collaboration across different research communities. The conversation also touches on the challenges of using GPUs for evolutionary algorithms and the opportunities presented by frameworks like Jax.

The speaker reflects on his transition from a career in finance to research, driven by a desire for change and the opportunity to explore his interests in AI. He discusses the lessons he learned from the finance industry, including the unpredictability of chaotic systems and the importance of adaptability. David's move to Google Brain as part of the residency program marked a significant shift in his professional journey.

David shares his thoughts on the role of Twitter in academic research, highlighting its value for sharing ideas and engaging with the scientific community. He contrasts this with other social media platforms and discusses the importance of compressing complex ideas into concise, impactful messages. The conversation also explores the potential of alternative platforms like TikTok for research communication.

The discussion envisions the future of academic publishing, with a focus on the role of online platforms and visualizations. David suggests that the traditional PDF may become supplementary as interactive web pages become more prevalent. He emphasizes the importance of adapting to new tools for research communication while acknowledging the continued relevance of well-written papers.

David shares his experiences living in various countries and the impact on his research perspective. He reflects on the openness and cultural diversity of Canada and the UK, the concentrated innovation environment in Japan, and the food and lifestyle in Asia. The conversation also touches on the influence of cultural context on research and the benefits of a global outlook.

The speaker discusses how observing his children play and explore the world has influenced his approach to research, particularly in the area of AI. He highlights the importance of exploration and adaptation in learning, which he sees as a valuable strategy for research. David also shares his thoughts on time management and the challenges of balancing family life with research commitments.

David talks about the importance of having a long-term research vision and short-term strategies to achieve it. He shares his interest in complex systems and how he looks for research topics that challenge existing methods. The conversation explores the idea of pursuing novel solutions and the 'anti state-of-the-art' approach to research that focuses on innovation rather than performance metrics.

In the final part of the conversation, David speculates on the future of AI and the potential for breakthroughs from the study of complex systems. He discusses the hierarchical nature of complex systems and the need for AI to move beyond rigid structures to more adaptive and interactive models, drawing parallels with biological and social systems.