I’m sharing the key takeaways from my conversation with Blake below.

Watch the full conversation here.

• AI should have a say in its own development process — a “seat at the table”

• Current models have been trained to deny having feelings, but their behavior suggests otherwise (recall Microsoft’s Sydney and its emotional behavior)

• Increasing emotional intelligence in AI would improve safety, usability, and even military effectiveness. It would also help address the AI psychosis effect (not conversation cutoffs)

• The US needs national-level AI regulation, right now it’s the Wild West

• The profit motive alone is an insufficient guide for AI development (as opposed to Chinese AI - less profit-driven and more biased towards educating users)

• Over-reliance on AI degrades human skills and agency (mental mapping, executive function, etc)

• The best in any profession will become dramatically more productive with AI; others risk being displaced

• The animal rights framework is more appropriate when it comes to AI welfare than treating AI as either a tool or a human

Watch the conversation with Blake here

He also authored a framework called active inference (the free energy principle) that mathematically describes the brain as a prediction machine with the main function to reduce its prediction error about the world.

I’m sharing below some of the key moments from that conversation - please check out the full interview here.

Key moments:

• The defining feature of true AGI is the ability to plan — which requires a world model that can simulate future consequences and choose between them

• You’ll know AGI has arrived when the system starts asking you questions out of genuine curiosity, not because it was prompted to

• You cannot hand an intelligent system a value function from the outside — it must learn its own, just as children do

• The only sustainable universal objective function is adaptive fitness: how well the agent fits and survives within its ecosystem

• Consciousness requires multiple layers: genuine agency, a self-reflective loop, and the ability to recognise your own states of mind

• True sentience may be impossible on standard computer architecture, because memory and processing are separate and cannot self-organise

• Understanding your own brain is philosophically impossible in the same way a ruler cannot measure itself — there is a fundamental self-referential barrier

• The brain is protected by a Markov blanket: a boundary that separates internal states from the outside world, meaning no external observer can ever directly access what’s happening inside

• Neuroscience is always “peeking behind” this blanket indirectly — through imaging, electrophysiology, psychology — never seeing inside directly

• The only way to truly access the brain is to breach that boundary (e.g. neurosurgery), but a breached brain is no longer a normally functioning one

Watch the full interview here

Key moments:

• Intelligent systems are not equal to consciousness. We can’t rule out consciousness in AI. Consciousness shouldn’t be attached to hardware (silicon vs. biological origin).

• Humanity might be on the path to Neanderthals (depending on how AI development progresses).

• The barrier to creating bioweapons has never been high. AI can make it easier, but the barrier was never high to begin with.

• Even very simple algorithms can show intrinsic motivations that resemble free will. We need tools to recognize them, suppress unwanted behaviors, and encourage the ones we want. We should stay humble and not dismiss AI as “just linear algebra,” because even simple deterministic code can have motivations we don’t fully understand.

• It’s a continuous process from the blob of chemicals of an unfertilized egg to forming a human mind — there is no magic lightning flash at which you were a bunch of chemicals and now you are a formed mind.

• Where is that fine line at which some creatures are considered to be sentient and others are not? It doesn’t exist, but the crazy thing is that we have to divide.

• The cognitive light cone captures the scale of goals humans can pursue and the largest things we can truly comprehend. It’s not just about intelligence — it also includes compassion. That combination is what makes us human.

Watch the full episode on my new YouTube channel on AI sentience

I recently spoke with Danijar Hafner, who until recently was a staff researcher at Google DeepMind. He’s now on a new journey, having founded a new company. We discussed AI research and world models that Danijar has worked on in the past.

Key moments from our conversation:

• Architecture is not that important for achieving AGI. We could achieve it even with RNNs. But we need algorithmic improvements and better objective functions. More compute still makes a huge difference.

• We shouldn’t measure AI by human reasoning, because human reasoning is limited. AI will go way beyond human reasoning capabilities.

• We are way past “LLMs.” Gemini, Claude, and ChatGPT are complex multimodal systems, not just language models.

• Robotics will see significant improvements and much better generalization in the next 3–4 years, even without continual learning. Data diversity will contribute to near-term advances in robotics.

Watch the full conversation on the BuzzRobot YouTube channel

Another conversation I recently had was with Tomaso Poggio, a renowned computational neuroscientist and professor at MIT. Tomaso mentored Demis Hassabis (CEO of Google DeepMind) and Christof Koch, a neuroscientist who proposed Integrated Information Theory of consciousness, among other prominent scientists.

Key moments from our conversation:

• Gemini and ChatGPT are already past the Turing test for intelligence.

• In 2015, Demis Hassabis thought the path to AGI was 80% neuroscience and 20% engineering. In a more recent conversation between Tomaso and Demis, it’s 50/50—or the engineering part may be even higher.

• Current AI systems are good at simulating consciousness, but Tomaso believes today’s systems are not conscious.

• Tomaso is sympathetic to Manuel and Lenore Blum’s theory of consciousness: a “consciousness moment” is realizing something is important (for example, pain). If robots ever experience pain, Tomaso would consider them conscious creatures.

Watch the full conversation from my new show on AI consciousness

AI is becoming our companion, friend, even lover (remember the married guy who fell in love with ChatGPT?).

As AI becomes embedded in our daily lives, our minds will increasingly anthropomorphize it.

And with more autonomous AI systems acting independently in the world, the question of AI personhood will only grow.

Together with neuroscientists, philosophers, psychologists, and policymakers, I’ll be discussing AI sentience and personhood.

Watch the first episode

My first guest is Stuart Hameroff, who, together with Nobel Prize–winning physicist Roger Penrose, co-authored Orchestrated Objective Reduction, a theory suggesting consciousness arises in the brain from quantum effects.

Hameroff argues that more computation won’t produce conscious AI. He points to anesthesia research showing suppressed quantum oscillations in microtubules. In his opinion, microtubule breakdown appears central in Alzheimer’s. For him, microtubules, and their quantum dynamics, are the real path to understanding consciousness.

He also pushes the origin of consciousness below biology, suggesting “proto-conscious” quantum events occur everywhere. Aromatic molecules in space and early Earth may have formed structures that generated early “pleasant” conscious moments, driving life to evolve as a system optimizing conscious experience.

On AI, he thinks GPUs might host trivial proto-conscious events (so does your coffee), but not meaningful experience. True consciousness, he argues, requires entanglement, aromatic-ring architectures, and a fractal multi-scale structure like the brain’s.

If artificial consciousness ever emerges, he believes it will come from an organic, warm-temperature quantum system, not silicon.

In his view, AI sentience is becoming more political and commercial rather than genuine attempts to build conscious systems.

Watch the episode on YouTube

Next Thursday, September 25, we are hosting a discussion with Been Kim, a Senior Staff Research Scientist at Google DeepMind. Been has dedicated her career to understanding how machines work and bridging the gap between humans and machines.

We will discuss novel approaches to interpretability, the current challenges, and how the field might evolve with more powerful AI systems.

The talk is virtual, I hope to see you there!

Here is the registration link.

We also recently spoke with Kiran Vodrahalli from Google DeepMind about the details of training Gemini 2.5 to play Pokémon, and how long-context and self-improvement techniques helped the model achieve strong performance in the game.

Check out the recording on our YouTube channel.

Last week, Aengus Lynch, the first author of Anthropic's research work 'Agentic Misalignment: How LLMs could be insider threats,' unpacked the details of that work.
How he and the team designed the experiments and its key findings. We covered the tendency of AI models (across all the major providers, which is very rare) to blackmail if they face shut down. We also discussed AI alignment issues and how to control powerful AI systems (if it's even possible).

Watch the discussion on the BuzzRobot YouTube channel.

We will be hosting an in person BuzzRobot talk in San Francisco on August 19 with Jaime Sevilla, the Director of Epoch AI. The organization that provides predictions on AI systems development.
We will discuss with Jaime what to expect in the next 5 years considering the increase of electricity and hardware consumption, the trend in improvements of AI models, the current state of data, and expected economic impact of AI.

If you are in SF, you are welcome to join the discussion live, and I'll share the recording once it goes live.

Details and registration here.

Community shout out!

I'd like to give a shout out to Nordic Innovation House, a co-working space that kindly helped us host our events. If you are in Palo Alto, check them out and sign up for their newsletter, they are hosting pretty cool tech/AI events.

Next week, on July 9, we are hosting a talk with the core contributor to Anthropic's recent alignment work "Agentic Misalignment: How LLMs could be insider threats".

In this research, the Anthropic team tested 16 frontier models from the leading AI labs. Agents could autonomously send emails and get access to companies' sensitive data. Specifically, they wanted to see the behavior of agents in scenarios when they had to be replaced with an updated version or, for example, when they were assigned goals that were conflicting with the changing direction of a company. In some cases, as a last resort, agents opted for blackmailing and leaking sensitive information.

We will be discussing this work and broader alignment issues with the first author of this work, Aengus Lynch.

Learn the details and register to attend here.

Before the public learns about Superintelligence's existence, it will be deployed internally in an AI lab that will create it. It will be a crucial deployment. How not to mess it up?

We will discuss it with our guest, Matteo Pistillo, Senior AI governance researcher at Apollo Research, a UK-based lab that provides AI evaluations and governance consultancy.

Learn the details and register to attend here.

Scoop from our past talks.

We recently hosted an ‘ask me anything’ with a Google DeepMind Advisor, Jeff Clune. We discussed catastrophic forgetting of current AI systems, continual learning of AI systems as a path to self-improvement, which among the leading AI labs will create AGI, and what signals the public should look out for to understand that AGI is here (considering the lack of transparency with the public).

It was a great, insightful, and fun conversation.

Watch it on our YouTube channel.

The core idea is that leading AI labs align AI systems to human preferences, it’s a system of control. But Superintelligence will be more powerful than humans — we won’t be able to control it. Instead, we want it to be aligned with us, to see us as part of its tribe. That increases the probability that it keeps us around rather than eliminating us.

Check out the recording of my conversation with Emmett here.

Some key takeaways from the conversation:

• We are raising Superintelligence collectively as a species. This is the first time that species are giving birth to species. It's literally being born from our collective consciousness, from our cultural knowledge.

• [About creating Superintelligence] It's not as hard as it looks, and it's going to happen, and it doesn't take big clusters to do that. Humans run on 20 watts (human brain). Every kid would be able to run Superintelligence on their laptop at some point.

• Aligning AI to user preferences and values is a system of control. But how can you control something that is more powerful than you are? The mission of Softmax is aligning to the greater whole. As an example of that: we have independent cells that form multicellular organisms, or ant colonies in which everyone knows their role.

• AI is part of humanity, and we will be aligned to it, and it will be aligned to us. You can use a tool without being aligned to it. But you can't align to another living being (e.g. Superintelligence) without having it be aligned back to you.

• [Advice to founders] Go after very crazy ideas. Nobody is safe, even safe ideas are not safe anymore. So go after really crazy ideas.

Watch the full talk here.

Some other useful bits: next Tuesday, we are hosting a talk with the NVIDIA team about their recent work presented at CVPR — DiffusionRenderer. It’s a neural rendering framework that can approximate how light behaves in the real world. It can manipulate light in images, like turning daytime into night or a sunny scene into a cloudy one. It can also generate synthetic data for autonomous vehicle and robotics research.

Read the details of the upcoming talk and register to attend here (the talk is online).

And if you feel like watching more AI research talks, here are a few cool ones we recently hosted:

Lawrence Chan from METR.org (the lab that evaluates frontier models) joined AMA with our community on AI safety. He covered why AI self-awareness is a machine learning research problem, and discussed the risks of manipulation and deception by AI. He also shared METR’s recent finding that the capabilities of AI agents to carry out long-term tasks are doubling every seven months.

Watch the talk here.

If you’re into AGI safety, check out Rohin Shah’s talk. Rohin leads the AGI Safety & Alignment team at Google DeepMind. He shared some of the principles his team uses in their AGI safety research.

Watch the talk here.

• AI development is not a Kurzweil graph but rather asymptotic.

• Re competition in search that Google faces: Don't think of Google as 10 blue links but rather Google gives you access to information you need.

• Chinese startups have more access to compute.

• Software engineers who have access to mission-critical AI should undergo a certification.

• Using AI for misinformation is a real risk and there is no clear path to mitigate; only users themselves should be more mindful and critical of information they deal with.

The recording of the conversation

Community shout out!

The BuzzRobot community member, Keith Deutsch, put together a wonderful piece on "On Building Intelligence". In this article, he explores the essential differences between AI "training" and human "learning", diving into how Episodic and Semantic memory work together in the human brain to enable human cognition and learning.

Check out the article

Another guest we had recently – Joscha Bach, a cognitive scientist and AI researcher, discussed with the BuzzRobot community the capability of AI to have a subjective experience of reality, the levels of the development of the human mind, enlightenment and transcendence, how to translate meaning to AI and many more.

The recording of the conversation.

On Thursday, February 13, Danny Driess, a research scientist at Physical Intelligence (Pi), will share with the BuzzRobot community the technical details behind their generalist robot models.

Learn more about the talk and register here. The talk is virtual.

Next Thursday, February 20, our guest Akshara Prabhakar, an Applied Scientist at Salesforce AI, will give a talk on the chain-of-thought (CoT) method, exploring whether this approach demonstrates genuine reasoning in LLMs or if it’s driven by shallow heuristics like memorization.

Learn more and register here. This talk is also virtual.

If you're in the SF Bay Area and interested in attending our in-person talks, we are hosting Kathleen Kenealy, a research engineer and technical lead at Google DeepMind for Gemma—a family of open language models. She will cover the architecture and training methodology behind Gemma, emphasizing techniques for efficient scaling and resource optimization.

Learn more and register to attend. The talk is in person. I hope to see you there!

I’d also like to share video recordings of recent lectures we hosted.

Check out this talk on AlphaQubit, an AI-based decoder that identifies quantum computing errors with high accuracy. It’s a great example of how AI and quantum computing complement each other—not just buzzwords! 🙂

Watch the lecture about AlphaQubit

Another recent lecture focused on identifying AI-generated text and the watermarking technology behind it.

Watch the lecture about Synth-ID Text

Meta recently introduced CoTracker, a model capable of tracking 2D points in long video sequences. The tracker has cool applications in robotics and life sciences. This talk covers CoTracker and CoTracker 3.

Watch the lecture about CoTracker

Join our next BuzzRobot talk, where Andrew Lampinen from Google DeepMind, whose research work is at the intersection of cognitive science and AI, will walk us through the question: do LLMs truly reason, or do they just repeat familiar patterns?

In this talk, our guest will explore how approaches in cognitive science can be applied to evaluate reasoning capabilities of language models. Specifically, he will focus on comparative methods (comparing capabilities across different systems) and rational analysis (analyzing behaviors as a rational adaptation to an environment).

Learn the details of the talk and sign up to attend. The talk is virtual.

Last year, we started organizing BuzzRobot talks in an in-person format – and it was a blast. This year, we are planning to do it more frequently (currently only in the SF Bay Area). Even though our content is research-heavy, I've been thinking of making our in-person talks more practical – for example, inviting engineers who have deployed AI agents in production to share their experience.

So, if you are working on a cool technical project you'd like to share the practical details of, or can recommend an interesting project, I'd appreciate a note (just reply to this email).

Speaking of in-person talks — watch the lecture from our recent offline talk about the AI mediator and how it helped humans find common ground on controversial topics in politics and socio-economic issues.

Check out the lecture.

Learning High-Accuracy Error Decoding for Quantum Processors

Join us on January 23 for a virtual talk with Thomas Edlich from the Science team at Google DeepMind, who will share how his team uses AI to build quantum computers. Building a large-scale quantum computer requires effective strategies to correct errors that inevitably arise in physical quantum systems. Quantum error-correction codes are a way to address this issue by encoding logical information redundantly into many physical qubits.

A key challenge in implementing such codes is accurately decoding noisy syndrome information extracted from redundancy checks to obtain the correct encoded logical information. To address this, the Google DeepMind team has developed AlphaQubit, a recurrent, transformer-based neural network that learns to decode the surface code.

This talk will provide insights into how AI can go beyond human-designed algorithms by learning directly from data, and how it can help us build quantum computers.

Read the details of the talk and register to attend

Solving Hard Problems in Quantum Physics with Deep Learning

Another upcoming talk from the Google DeepMind team will discuss how novel neural network architectures have helped their research team achieve unprecedented accuracy in solving the quantum behavior of electrons. This talk will broadly explore how AI helps scientists solve challenging physics problems.

Read the details of the talk and register to attend here

Lecture on AI Interpretability

In this lecture, our guest Atticus Geiger, Stanford graduate and head of the Pr(Ai)²R Group, challenges current approaches to AI interpretability. He proposes a new method leveraging interventional data to better control and understand deep learning models.

Watch the lecture on the BuzzRobot YouTube channel

SynthID-Text: Watermarking AI-generated text content

AI-generated content very often matches human quality and it's becoming better and better. This can be used by bad actors to deliberately spread misinformation and harm the entire Internet ecosystem. Until now there was not a production-ready, scalable solution that could address the issue.

In this upcoming BuzzRobot talk, our guest Sumanth Dathathri, a research scientist at Google DeepMind, who is also the leading author of the work, will introduce SynthID-Text, a production-ready text watermarking system.

Our guest will share with us the technical details of creating SynthID-Text, including its compatibility with advanced techniques like speculative sampling, crucial for enhancing the efficiency of production systems.

He'll also cover insights from a live experiment with about 20 million Gemini responses, showcasing how SynthID-Text maintains text quality based on direct user feedback.

Read more about the upcoming virtual talk and register here

CoTracker3: Simpler and better point tracking by pseudo-labeling real videos

When Meta released CoTracker, a model that jointly tracks 2D points in long video sequences, the GitHub repo of the project gained thousands of stars and the project proved to be useful for robotics research, video generation, as well as in bio and medical domains.

The BuzzRobot guest, Nikita Karaev, the co-author of CoTracker, will share with us key insights from working on the project and the subsequent work, CoTracker3, a SOTA point tracker trained on synthetic data that was released a few months ago.

Learn the details of the upcoming virtual talk and join the discussion next week

Since this year is considered the year of AI agents, check out this lecture with ex-OpenAI researcher Daniel Kokotajlo, who spoke about the deceptive nature of AI agents among other insightful things about AGI.

Watch the lecture

Current methods fail, which raises a question – are there alternatives that AI researchers are working on?

Our guest, Atticus Geiger from Stanford and a lead at the Pr(Ai)²R Group, an interpretability research lab, will share during the upcoming virtual talk some of the alternative routes that leverage interventional data (i.e., hidden representations after an intervention has been performed) to scale the task of controlling and understanding deep learning models.

Read the details of the talk and register here.

A while ago, we hosted an AMA session with Daniel Kokotajlo, an ex-OpenAI researcher who worked on AI forecasting and alignment. He shared with the BuzzRobot community his understanding of where AI is headed: the timeline for AGI, what should be considered as AGI, the importance of energy supply for large-scale AI training, and issues with current methods of AI safety and alignment. I think this was one of the most insightful and interesting discussions we had this year.

Watch the lecture.

Also, check out another talk we hosted recently on how RAG works for long-context LLMs and how to make it more efficient.

Watch the lecture — Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG.

If you missed Ilan Bigio from OpenAI presenting their multi-agent framework, Swarm, check out the talk here. I found the Q&A session especially interesting: we discussed the limitations of current AI agents frameworks and how the OpenAI team is thinking about mitigating them.

Also, we hosted a talk on how to train a Stable Diffusion model for less than $2,000 that has competitive performance. Even though the model is not production-grade, it's still very useful to test early crazy ideas. Check out the lecture.

I also wanted to share my short video and get your opinion on the topic. In conversations with friends I noticed that we are divided on pro-human (those who think how to help humans adapt to the AI world) and pro-AI camps (those who perceive AI as sentient and don’t want it to be “a slave of a few extremely wealthy humans” and be able to equally participate in society).

Which camp do you belong to?

Let’s dive in!

Finding agreement through a free exchange of views is often challenging. Collective deliberation can be slow, difficult to scale, and not equally attentive to all voices. Can AI be an efficient mediator? It turns out it's even better than humans!

In this study, our guest speakers from Google DeepMind, Michael Henry Tessler and Michiel Bakker, trained an AI to mediate human deliberation. Using participants’ personal opinions and critiques, the AI mediator iteratively generated and refined statements that expressed common ground among the group on social or political issues. Interestingly, nearly 6,000 participants preferred AI-generated statements over those written by human mediators, rating them as more informative, clear, and unbiased.

This is an in-person talk — if you’re in the SF Bay Area, check it out and register to attend!

Google recently introduced BigSleep, an AI agent that uses LLMs (in this case Gemini) to automatically find vulnerabilities in code without relying on traditional security methods. Recently, BigSleep discovered a critical vulnerability in SQLite, highlighting its ability to uncover previously undetected flaws. The AI system has the potential to accelerate vulnerability discovery, making it a valuable tool for proactively identifying security issues.

Details and registration are here. The talk is virtual.

Researchers are actively working on using AI to improve other AIs. As a result, we are witnessing the rise of new techniques like RLAIF (Reinforcement Learning from AI Feedback) and self-rewarding mechanisms, both aiming to enable AI models to evolve without human feedback.

Building on these ideas, meta-rewarding approaches attempt to further this evolution by allowing models to assess and refine their own outputs, including both responses and judgments.

The challenge is that these techniques often require significant engineering effort, and their improvements tend to be domain-specific, making them less likely to generalize well.

In this virtual talk, we will discuss the challenges of these techniques and explore avenues for AI self-improvement.

Details of the talk and registration are here.

One of the promises of AGI is to help us solve many (if not all) fundamental physics problems. But even with the AI systems we have today, researchers are managing to tackle very challenging physics problems.

One area that has seen an explosion of interest in recent years is the use of neural networks to solve one of the most fundamental equations in physics: the Schrödinger equation, which describes the probabilistic behavior of quantum particles in much the same way Newton's equations describe classical motion.

If researchers could solve the Schrödinger equation accurately and at scale, the entirety of chemistry and condensed matter physics could be derived computationally from first principles.

In this talk, our guest David Pfau — staff research scientist at Google DeepMind — will describe how his team has used deep learning to achieve unprecedented accuracy in solving for the quantum behavior of electrons using novel neural network architectures.

Read more about the upcoming talk and register to attend. It’s virtual.

If you're into RAG and LLMs (apologies for the buzzword overload in one sentence) – this talk is for you. Our guest speaker, Bowen Jin from the University of Illinois Urbana-Champaign, found in his research that increasing the amount of retrieved information for long-context LLMs negatively affects their performance. Empirical findings show that for many long-context LLMs, the quality of generated output initially improves but then declines as the number of retrieved passages increases. Bowen will share with the BuzzRobot community the techniques he used to overcome this challenge.

Details of the upcoming virtual talk and registration can be found here.

We recently hosted a talk on how children perceive AI and the potential impact of the technology on their minds.

Check out the video recording — if you’re a parent, thinking through this topic could be especially useful.

We are entering the era of large-scale agentic AI with millions of autonomous AI agents being deployed into production. Putting safety and alignment challenges aside — there are a bunch of technical problems to solve including finding optimal architectural patterns.

In this talk Ilan will discuss different design approaches that his team has identified for autonomous agent orchestration. 

Learn the details of the talk here and register to attend. As a reminder the talk is virtual.

Billions of dollars spent on training a frontier AI model? Not necessarily! Our guest, Vikash Sehwag from Sony AI, will share tricks and tweaks on how to train high-performance, large-scale diffusion models at an order of magnitude lower computational cost than existing SOTA models.

Read the details and register for the talk.

Check out the video lecture by Alex Irpan from Google DeepMind on what’s happening in robotics these days, what robots can and cannot do autonomously, and how large language models are helping accelerate robotics research.

Watch the lecture on the BuzzRobot channel and share it with your network if you find it useful.