BOLD Talks: How the flexibility of liquid neural networks resembles the brain’s nature

It’s 2023, and AI has become practically omnipresent, with ChatGPT notably making its mark. In a recent podcast, we had the privilege of interviewing Ramin Hasani, an AI scientist at the Computer Science and Artificial Intelligence Lab (CSAIL), Massachusetts Institute of Technology (MIT).

He is a representative of the „BOLD Minds,“ a part of the BOLD Community, an initiative by the Austrian Federal Economic Chamber (Wirtschaftskammer Österreich or WKO). However, our conversation focused not just on tools like OpenAI, but rather on his captivating journey to becoming a scientist and a pioneer in his field.

His involvement in a novel type of artificial neural network, called liquid neural networks, significantly shaped his career and could probably influence our future on a big scale. Why? Liquid neural networks resemble the brain’s nature. That means they are incredibly flexible. Therefore they hold immense potential as a game-changer in areas like autonomous navigation systems, medicine and finance. In our conversation Ramin Hasani also shares advice for future-scientists and explains why he thinks that the concept of passion is overrated.

From designing rescue robots to becoming an AI scientist

Ramin Hasani started our conversation about his career by expressing his passion for mathematics and physics, describing them as subjects that captivated his interest from an early age. He explained that he was naturally drawn to comprehending the inner workings of things and found these subjects to be his strongest during his school years. Subsequently, he made the deliberate choice to launch his professional journey within the field of electrical engineering.

“This is where I got hands-on with mathematics and physics to develop real-world systems and real-time applications such as chips and robots. I also engaged in competitions. These experiences were incredibly rewarding, as they allow us to create meaningful outcomes that have a significant impact. For instance, one of my notable achievements involved creating a rescue robot capable of navigating various environments effectively. This was particularly valuable in situations like natural disasters, where robots could be deployed to assess the situation”, he says.

Ramin’s involvement in robotics persisted, evolving further after he attained his bachelor’s degree. Progressing in his academic journey, he transitioned into the role of an electronic engineer. This job encompassed the intricate task of designing microscale chips that functioned as control units, applicable in a wide range of contexts, from vehicles to robots to various appliances. Ramin’s focus remained steadfast on these electronic pursuits. He recounted pursuing his master’s studies at Politecnico di Milano in Italy in 2014.

A shift from hardware to software

This is where he first came into contact with what would later become his special focus. “In my master’s thesis, a professor approached me with an experimental project to develop brain-inspired neural networks on a chip. The aim was to design advanced chips that go beyond traditional logical systems found in electronic devices like phones. These neural networks would emulate the behavior of the brain, performing exceptionally well on specific applications that traditional logical systems may struggle with.”

The initial goal was to develop brain-inspired neural networks on a chip, a goal that ended up being achieved. This demonstrated the utilization of noise as a computational resource. While originally focused on chip design and electronics, the interest gradually shifted toward artificial intelligence and the algorithms behind intelligent behavior. This transition led to pursuing a Ph.D. position at Vienna University of Technology under Professor Radu Gruso, exploring the brains of small animals, including the sea elegans worm. This research pathway delved into software development, marking a shift from hardware to software and the intelligent aspects of the field.

Back to drawing inspiration from the brain

Before he finally explained how liquid neural networks work, Hasani discussed how artificial intelligence (AI) systems have evolved, highlighting a shift in approaches over time. Initially, the focus was on emulating brain processes to create algorithms, giving rise to the neural network movement in 1956. However, modern computer science prioritizes efficient algorithms, leading AI to move away from its brain-inspired origins. Emerging from simpler algorithms, transformative technologies like transformers have become the foundation for language models and generative AI. In computer vision, various designs and convolution techniques stemmed from the brain-mimicking concept. Despite their simplicity, these systems have scaled into advanced neural networks.

However, challenges arise as many AI systems lack transparency, functioning as opaque „black boxes.“ Deciphering their complex behaviors poses socio-technical hurdles intertwined with AI’s progress and system construction methodologies.

“My research question during my PhD and subsequent postdoc positions between TU Vienna and MIT was: Can we do better? Can we return to the source, which is the brain, and draw proper inspiration from its mechanisms this time, given the improved hardware?“

Hasani explained that this essentially marked the beginning of his Ph.D. program. He started actively working on liquig neural networks, and over the years, they managed to develop a comprehensive understanding of the entire infrastructure underlying these concepts.

Massive amount of information with small components

He says: “ This understanding can serve as the source and foundation for new artificial intelligence systems that draw inspiration from the brain. What does this brain-inspired approach provide? It offers us significantly greater capacity to learn a massive amount of information with small components. Moreover, it grants us a substantial advantage in terms of explainability. These systems can be explained, especially if you construct what we term liquid neural networks. These systems have the potential to remain flexible even after training. This flexibility resembles the brain’s nature; it’s not a rigid structure. Post-training, it can still adapt. Therefore, adaptable mechanisms are essential. And that’s where we move forward.“

BOLD Talks: Ramin Hasani über die vielen Talente von Liquid Neural Networks

19 neurons instead of millions for autonomous driving

But how can they be used in the real world? For this, Ramin Hasani and his team delved into autonomy applications and real-world scenarios like autonomous driving. They demonstrated that within the realm of autonomous driving, achieving successful car navigation requires just 19 neurons. This stands in contrast to the prevailing trend at the time, around 2020 when the project commenced, where artificial neural network systems relied on millions of neurons or elements.

“This represents a fundamental reason underlying the significant value of this technology. Additionally, the intricate and complex nature of each node within our system enables us to achieve remarkable potency in a compact format. Subsequently, we extended this concept by demonstrating that drone flight can be realized using just 30 neurons“, said Hasani.

To sum it up: Liquid neural networks, inspired by biological neurons, offer adaptability post-training. With only 19 neurons, they outperform larger models. Their attention map resembles human focus. Unlike traditional scaling norms, they prove smaller models can excel, enhancing accountability and generalization. They use nonlinear probabilities for synaptic weights, adapting to environmental changes.

Financial evolution, medical trends or the progression of user engagement on websites

Another question one could ask is, what other fields of application are there? “They are all in the stage where they are being productionalized, making liquid eural networks applicable at the product level too”, Hasani revealed. He went on saying: “ The other thing is that liquid neural networks are universal computers, in other words, you can apply them to any type of data. Take the medical field, for instance. While media highlights prominent use cases like autonomous driving, lesser-discussed applications involve liquid neural networks excelling in modeling time series data.“

According to Hasani, this data pertains to financial evolution, medical trends, or the progression of user engagement on websites. Additionally, these networks are adept at recommendation systems. For instance, Amazon using them to predict customer preferences based on their evolving behavioral patterns over time. liquid neural networks excel in processing any real-world process that involves a temporal element or a time element. Consequently, they can be utilized to identify locations for resources like oil and gas. Hasani also mentioned that they are proficient in processing real-world processes with temporal elements, making them valuable for tasks like identifying oil and gas deposits.

Toyota, Boeing and the U.S. Air Force

When asked about companies showing interest in using liquid neural networks, Ramin Hasani explains that they have indeed formed collaborations with various big players. He mentions having worked with sponsors at MIT, including Toyota, Boeing and the U.S. Air Force. These partnerships have a common goal of achieving explainable autonomy, offering a service through the utilization of this technology.

Founding LiquidAI

But that’s not all. Hasani also founded a company called LiquidAI, which is primarily focused on delivering Liquid AI technologies to customers. It is currently in stealth mode, with plans to emerge backed by strong funding and successful accomplishments. What makes this endeavor exciting is his intention to establish the company as a pioneering foundation model startup. This framework could become a fundamental pillar for both generative AI and specialized AI.

“This foundation is based on liquid neural networks, chosen for their widespread applicability across various fields. My goal is to transition from academic concepts to practical industry solutions, specifically targeting the socio-technical challenges that are intrinsic to artificial intelligence. The vision of LiquidAI is to accomplish this by building upon fundamental principles, encapsulating the core essence of what Liquid AI represents“, he adds.

Joining the BOLD Community

Reflecting on why he got involved in the BOLD community, Hasani says: “Because of the power of people! My initial motivation stems from the individuals I’ve had the privilege to connect with through the BOLD Community. These connections go back to various backgrounds even before the community formally existed. I have immense faith in these individuals. As I mentioned earlier, I strongly believe that bringing together diverse minds from various disciplines is incredibly valuable. It’s a strategic approach for making impactful decisions. Organizing a community in this manner is undoubtedly the right path, resulting in tangible outcomes from conferences and discussions.”

He believes it’s crucial to set up an agenda for documenting and taking action based on the discussions that occur during these conferences. These gatherings assemble influential figures from the industry who wield the power and influence to impact our surroundings. He sees this initiative as indispensable, a truly brilliant step to establish such a community. That’s precisely why he chose to become a part of it, and he’s enthusiastic about contributing by sharing his own insights.

Making a meaningful contribution to the world

When asked what his advice for future-scientist is, he told us: “What I would say is that you should begin by exploring; Don’t be afraid of trying out different options and experiencing various aspects of life. To start, you need to conduct a thorough search to understand what genuinely interests you and what brings you happiness. The concept of passion is, in my opinion, overrated—quite overrated, actually. If you think about passion, dedicating excessive time to defining it is somewhat overblown.”

He concludes that it’s crucial to shift your focus towards leading a life that harmonizes happiness with creating a meaningful impact on the world. Successfully achieving both personal contentment and significant influence demands active involvement in various endeavors during your youth. At the outset, exploration plays a vital role. As you discover something with the potential to spark your interest, delving deeply into that realm becomes imperative. Evolving into an expert within that domain becomes the inherent progression.

The BOLD Community is a global network of visionary pioneers promoting innovation and collaboration. Through meetings of „BOLD Minds“ from all over the world, unconventional ideas and a lively exchange of knowledge emerge here, strengthening Austria as an innovation hub.  

An important opportunityfor members to meet up is coming soon at the BOLD Unconference 2023. From September 7-10, 2023, this event will gather 50 thought leaders from around the world to shape the economy of tomorrow. Ramin Hasani is among the invited personalities. The BOLD Unconference is fundamentally different from traditional events and conference formats. It offers an open platform for interaction and idea generation.

If you’re interested in the BOLD Community, follow us now on Instagram and LinkedIn! Here you’ll find more information about the community.