Solving an Equation from 1907 and Liquid Neural Networks

Last year, MIT researchers announced that they had built “liquid” neural networks, inspired by the brains of small species. This is a class of flexible, robust machine-learning models that learn on the job and can adapt to changing conditions. That is important for safety-critical tasks, like driving and flying.

The flexibility of these “liquid” neural nets are great but they are computationally expensive as their number of neurons and synapses increase and require inefficient computer programs to solve their underlying, complicated math.

Now, the same team of scientists has discovered a way to alleviate this bottleneck by solving the differential equation behind the interaction of two neurons through synapses. This unlocks a new type of fast and efficient artificial intelligence algorithm and is orders of magnitude faster, and scalable.

What I find interesting is that the equation that needed to be solved to do this has not had a known solution since 1907. That was the year that the differential equation of the neuron model was introduced. I recall when I was a student and when I was teaching at a university (in the humanities) hearing the complaints of students battling away in a course on differential equations.

These models are ideal for use in time-sensitive tasks like pacemaker monitoring, weather forecasting, investment forecasting, or autonomous vehicle navigation. On a medical prediction task, for example, the new models were 220 times faster on a sampling of 8,000 patients. 

Specifically, the team solved, “the differential equation behind the interaction of two neurons through synapses… to unlock a new type of fast and efficient artificial intelligence algorithms.” “The new machine learning models we call ‘CfC’s’ [closed-form Continuous-time] replace the differential equation defining the computation of the neuron with a closed form approximation, preserving the beautiful properties of liquid networks without the need for numerical integration,” MIT professor and CSAIL Director Daniela Rus said. By solving this equation at the neuron-level, the team is hopeful that they’ll be able to construct models of the human brain that measure in the millions of neural connections, something not possible today. The team also notes that this CfC model might be able to take the visual training it learned in one environment and apply it to a wholly new situation without additional work, what’s known as out-of-distribution generalization. That’s not something current-gen models can really do and would prove to be a significant step toward the generalized AI systems of tomorrow.



child question
   Image: Gerd Altmann

I read an article by Margot Machol Bisnow, author of  “Raising an Entrepreneur,” who did interviews with parents who raised highly successful people. She was curious about what skills they taught their kids at an early age. A simple takeaway from her research is that one skill they all agreed on was curiosity.

Curiosity can be defined as the desire to know something but that is oversimplified. Every teacher values curiosity of some kind. Sometimes teachers find that student curiosity can be overwhelming (or even annoying) when it doesn't match the path of a lesson. Questions off the topic at hand can hijack a lesson - or they can lead to interesting discussions.

So we might define curiosity as including trying to fix something, asking good questions, wanting to know how something works and wondering how it might be done differently or better.

From the article, here are 3 things parents did with their kids that should also be part of a classroom.

1. They encouraged their kids to fix things.
2. They instilled the confidence to tackle big, real-world problems.
3. They asked hard questions.

Streaming Learning

video playerThis past summer for the first time ever, streaming services captured more viewers than cable or broadcast TV, according to new data from Nielsen. Streaming has outperformed broadcast before, but never broadcast and cable in the same month. It's a close race though.
In the U.S., streaming captured 34.8% of viewership in July, while cable accounted for 34.4% and broadcast came in third at 21.6%.

When I read an article such as "Reasons Why Video Streaming Is The Future Of Education In 2022," the reasons are really the reasons why we should be offering online learning. Streaming is just a newer delivery method.

The history of distance learning goes from correspondence (snail mail) to broadcast and ITV, to videotapes, CDs and DVDs, the Internet (the earlier and slower version), and now streaming. When video first appeared in classrooms as broadcast, ITV and even on tapes it was sometimes considered controversial. Did it have educational value? Was it a lazy way to teach? Didn't students get enough video at home? But that is no longer true in almost every instance. Video is effective for learning. Online video has been shown to enhance comprehension and retention of information, support multi-modal learning, can help develop digital literacies when it is taught rather than just consumed. It can also be a more cost-effective learning solution, and can be repurposed in multiple ways.

Frequently, video is a supplement to make additional material available to students online. Some movements, like the flipped classroom, used online videos to swap lectures and classroom time. And this is true beyond traditional classrooms and schools as video became a training model for employees and customers.

When you use a streaming service (Netflix, Apple+ et al) you are almost always watching recorded videos. But the newer use of streaming is live streaming. Teachers are live streaming lectures and lessons to fully online students and also to students when you can’t meet in person. The real-time nature of live video allows a virtual classroom to be interactive in ways similar to in-person lessons.

Educational live streaming goes beyond lectures. There are also discussion panels, debates, guest speakers, presentations, virtual field trips, laboratory exercises, tutorials on demand and workshops.

Live streaming almost alw and "interact" sometimes ays ends up being recorded video later. Many presentations I register for that are live are later offered as a recording. That's great but it does make me feel less of an obligation sometimes to watch it live. Yes, I can sometimes ask questions in the live sessions, but I've gotten so used to recording TV programs and watching them later that it has carried over to "educational" applications.