The future of AI is large scale coordination
A thought exercise on the journey from frozen time to frozen intelligence
Back in June 2019, one late night I was thinking about how technological advancements unlock large scale changes in the world. I had been fascinated by how software was changing the world so rapidly, and why it had different characteristics from the hardware world. I also pondered what would the advances in AI technology unlock - what came next? While trying to find patterns, I tried to look back into advancements that had altered the course of human history.
To my utter amusement, I ended up reaching all the way for the Big Bang theory itself, which felt as if a moment from the eponymously named show had come to life. Feeling a mix of happiness and embarrassment at what I had done, I just filed my thoughts in a self-addressed email, which is how I usually record such fleeting thoughts.
Recently, with the rapid advancements in AI - especially Generative Models like ChatGPT and StableDiffusion, people in the global tech community and beyond, have been asking - what comes next? This took me back to my earlier thoughts and I spent the next three days pondering over this question. I could finally found an answer I was satisfied with and so I thought of writing this blog (my first ever) to share it with anyone who comes looking.
While I’m quite serious about my conclusion, before coming to it, I’d also like to share a snapshot of my crackpot thought exercise with you.
By “frozen” in the above lines, I’m implying something that is compactly stored or static. Let’s parse this abomination line my line so we can understand the premise of the question we are asking.
In the beginning, time was frozen
frozen time → unlocked scaled up energy
Yes, time began with the initial expansion of the Universe, which is usually referred to as the Big Bang. Even if we look at alternative explanations, they have some form of a new beginning, at which point time comes into play.
So in my opinion, the moment “frozen time” was unfrozen (due to whatever cosmic reason), it unlocked an immense amount of energy. This energy was converted to different forms of potential and kinetic energy - gravitational, magnetic, chemical and nuclear among others.
Then a lot of stuff happened
frozen energy → unlocked scaled up action
All this energy that was unlocked had it’s way throughout the universe as it expanded for about 13.8 billions of years. A lot of stuff happened - stars were born and died, big things broke into smaller things, smaller things joined to become bigger things, galaxies happened, black holes happened.
And one fine day on earth, life happened.
Humans happened. The big brain moment of humanity had arrived and we did what we do best - use big brains to get big wins. One of these wins was to harness the energy of matter to do mechanical work in a scalable way. Thus we created the Steam Engine, Combustion Engine, Nuclear Fission, Hydro Electricity and more. All this enabled humans to perform actions at a never before seen scale. Instead of using the leverage of our own bodies, we were able to utilise these energy converters to drive mechanical action via machines.
Actions speak louder when frozen
frozen action → unlocked scaled up thought
While machines could perform actions at scale, the critical element became controlling them directly via mechanical means - think steering wheels, rail tracks, gear mechanisms etc. Human thought became the limiting factor for operating these machines. Enter the field of computer hardware. It had started with Charles Babbage designing the analytical engine. It could theoretically perform any general purpose calculations via a set of hardware actions that, given some inputs, could compute a certain result.
Over the next century, we kept on miniaturising these actions through a series of technological breakthroughs. Eventually we arrived at hardware so compact that the earliest ones could perform more than 90,000 instructions per second while being smaller than a finger tip - the microcontroller. Thus we were able to have an enormous number of “frozen actions” into miniature machines (silicon circuits) capable of decision making (conditional setting of bits).
For the first time, we could program machines by “freezing” our own decision making actions into microcontrollers circuits and unlock scaled up thought by machines as described above. This resulted in large scale batch processing by machines of all kinds, specialising in various environments.
Thoughts are fleeting, until they are not
frozen thought → unlocked scaled up intelligence
The microcontroller substrate allowed humans to encapsulate their thoughts from rudimentary decision making to very complex computations. This was done in a layered manner by programming each successive layer with higher order abstractions.
Machine code gave way to assembly programming which in turn made possible higher level languages in which humans could encode their thoughts - by a process called - well, coding. Each of these were increasingly complex collection of “frozen thoughts” in a sense that they could, given some input, replay the entire set of calculations and decisions the person writing the code would have thought of performing.
Using these programming languages, increasingly large and complicated “frozen thoughts”, i.e. programs - were written for all kinds of domains and applications. This intelligence of humans- the calculated steps focused on solving specific problems, for the first time in history could be deployed on a world wide scale through machines. Humans were able to further scale up intelligence by developing techniques under the field of Machine Learning and AI, where they could encode how machines could be made to “learn” about the characteristics of the data and do inference on new data.
Storing into the abyss
frozen intelligence -> ?
Yeah that’s right. We have been storing representations of vast amounts of human knowledge, almost the entire archived Internet, questions, answers, communications, images, videos, books, encyclopaedias and even software written by humans - into compact machine learning models - GPT-3, Stable Diffusion, BERT and the like. In other words, the intelligent reasoning that we encoded into machines has now evolved, via pre-training on vast amounts of data, into the form of ready-to-use condensed numerical weights. This is similar to how our own brains store information and intelligence via neuronal connections.
This is what I refer to as “frozen intelligence”.
Alright, now that we have unwrapped this chain of thought, let’s come back to the question we and everyone else for that matter, is asking.
What, in the name of the holy singularity, comes next?
I believe it is coordination at scale.
The future of intelligence is coordinated
frozen intelligence → unlocks scaled up coordination
Coordination at scale would be autonomous coordination via artificial intelligent agents. They would be able to adapt to each others interfaces and figure out how to utilise the expertise of other domain-focused agents, while in turn offering their own expertise to other agents that come looking.
For this to happen, an agent would have to be a part of a network and it would advertise (make available) the subject of its expertise and the interface it understands, similar to how service discovery works in software engineering.
Intelligent coordination at scale is one of the abilities that separates humans from other life forms - whether it is war or work, we can coordinate with hundreds of thousands, even millions of others, while offering our own expertise.
This is also the likeliest way, in my opinion, how AGI would emerge - by having domain focused artificial agents collaborate to achieve higher forms of intelligence.
I’d love to know what think about this stuff. Agree? Disagree? Please let me know your comments here or on this tweet:
