I wanted to quickly share my thoughts on the future of doing frontier research. This is most likely be a very chaotic unordered collection of my thoughts, but bare with me.
What is research?#
I like to formalize research as the acquisition of knowledge. Merriam-Webster defines research as:
investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws
The scientific method is one way to do research - it formalizes the process so that independent parties can not only understand what was done faster by have an easier time verifying that the process itself was correct and the results can be trusted.
I’ve hinted on it, but a big part of research is also the verification - that is the case especially for frontier research. People can make mistakes, people can misinterpret measurements, both intentionally and unintentionally.
Now, while I do have a lot of critique for the current academic verification system, it is inarguably one of the pillars of research - or even science as a whole.
How to do research?#
Doing research is actually remarkably easy.
- Collect prior knowledge - you either already have it, or read books/papers/linkedin slop; it can be literally whatever you what, the main goal is to collect it and name it
- Create a hypothesis - this is the derivation step, the “implication part of modus ponens”; this is, potentially, the new piece of knowledge - now, you have to prove it
- Test the hypothesis - rely on proven formalisms; create a “harness” to either prove or disprove your hypothesis; can be using formal logic, mathematical analysis, or even repeated measurements and statistical significance analysis
- Profit or get back to the drawing board
Bam, you’re a scientist.
What’s the future of research?#
Many people are now talking about auto-research using AI (so far that’s mainly LLMs). They paint a picture of the not-so-distant future where the AI systems will be self-improving by doing the research and making newer, better, versions of themselves.
I think this is very tech bro sci-fi dreaming - just like “crypto will replace all finance in the next few years”.
At least the AI systems right now are basically all LLM based. LLMs are autoregressive - meaning that they are trained to interpolate, not extrapolate. LLMs fundamentally do not understand the patterns and compose them, they are just very sophisticated parrots.
Many will try to prove me wrong in saying this - there’s many financial motives for proving me wrong - but that is the fact of the matter; that is how the technology works.
What I’m essentially trying to say is that LLMs cannot produce anything new per se; they only repeat what they’ve been trained on, but there is one caveat …
You probably have heard about the case of a Google DeepMind team using “AI research” to find a more optimal algorithm for computing matrix multiplication. If you’ve looked under the hood of that, you may have also realized that instead of there being some ultra smart AGI scientist model, it was just a bunch of somewhat normal models that were just lefts looping on the problem until one of them hallucinated the code that was faster than the current SOTA.
“But wait, aren’t hallucinations bad? The thing we want the LLMs to actually not be doing at all?”
For the normal case, yes, but for the auto-research this is something it’s fundamentally built on. The whole point is that we let the model go a bit wild and hope that it somehow spits out a groundbreaking discovery that is also true.
This is the monkey with a typewriter and infinite time thing all over again.
Essentially, we’re just exploring the state space of all possible “solutions” and using the LLM as a heuristic. The temperature is the exploration/exploitation balancing coefficient.
Will this work?#
I, quite obviously, cannot predict the future. What I can say is that the auto-research hype is in my opinion heavily oversold.
This will definitely be a useful tool to do broad analysis of a problem space, optimize approached, cheaply test ideas. What I mean is that it will be an auxilary thing.
I imagine that a researcher will a half baked hypothesis will use this to “cover more surface area” while being the ultimate leader and decision maker.
The fundamental problem with auto-research is that it cannot work without a measurable goal/outcome. It needs well defined scope. Humans can operate in ambiguity, reason about abstract thoughts. When the human gets the problem to a point where there is a clear metric (or a set of metrics) to optimize on, that’s when the auto-research comes in. (We already do this btw, search for optimization problems, but i do see how including an LLM skips the annoying part of formalizing the problem into either math of some programming language from the 90s)