For LLMs, I think every business is currently throwing things at the wall and seeing what sticks. Nothing wrong with a phase of experimentation for a new technology, and I prefer the $200M spent on training GPT-4 to the $500M spent on advertising the mobile app Monopoly Go, but I could see an eventual contraction in this activity and number of start-ups. I feel as though the challenges of intelligence and non-shallow reasoning may be more of a slow chipping away with incremental progress, fighting against diminishing returns, rather than exponential improvement or a "hard takeoff" that some are expecting.
Generally though, deep learning on large data is undeniably a very useful technique. We've seen huge advances in areas like language translation, voice transcription, OCR, visual defect detection, etc. and will doubtlessly continue to see it applied to a wide range of tasks. If the public and investors sour on "AI", it'll just go by another name as it did before.
I'm not making any predictions, but I think we'll have a much better idea of the long-term prospects of AI in late 2024 - mid 2025. Claude Opus 3.5 is supposed to be released later this year, and GPT-5 is expected sometime in 2025 IIRC.
I'm still amazed by the quality of Sonnet 3.5's outputs and OpenAI's realtime voice model. I expected the skepticism from this site's userbase but not to this degree. Anyone who has used Sonnet 3.5 and still thinks generative AI is just a "party trick" is either not thinking hard about AI's potential impact or simply in denial, imo.
The skeptics hang around Ask HN. The optimists and opportunists tend be over in Show HN. Most people can't tell the difference between Sonnet 3.5 and ChatGTP-3.5, or GPT-4o and GPT-4.0.
There's a joke I first heard in the early 90s: "True AI is 10 years in the future and has been for the last 30."
I suspect we'll be in exactly the same place: incremental improvements in LLMs, but the corner case failures will be perceived as significant enough to preclude widespread adoption.
Maybe we'll go back to the "neural network winter" of the 80s and 90s where "anti-hype" seemed to make research into CNNs unfashionable.
Or maybe industry will deploy LMMs widely and just cope with their flaws.
Similar to self-driving cars, future AI will train directly on reality rather than on text documents. The ultra-low-power-very-high-resolution cameras we all will wear will track our eyes and take note of what we pay attention to and ignore (including on computer screens). The cameras will also record audio, and data from a variety of sensors. Like seeing Google cars on roads scanning the roads, everyone will become a Google person, training their personal AI and contributing data to the global Google AI. Small robots will be running around everywhere, both to collect data and to run errands, powered by new battery technologies. A standard will finally arrive for indoor location with sub-centimeter accuracy which will transform the amount and type of data AI's are trained on, and enable many new applications and privacy invasions. AI's will be trained on forests, oceans, lakes, rivers, fields, etc., and start cataloging all of nature and what actually happens in all these places. Some of these mobile sensors will be chemical sensors, to track pheromones, body odors, toxic chemicals, food odors, DNA, RNA, soil conditions, and more, creating a whole new training set and new ways to query and react to the natural world, as well as revolutionizing medical AI's by finally including the environments effect on health of people and animals (including wild animals). Some of this will happen quickly, some of it will take decades.
In the next 2 years, we can expect AI and Large Language Models (LLMs) to become even more sophisticated in natural language understanding and generation, leading to more accurate and context-aware applications in customer service, content creation, and education. (https://studycorgi.com/blog/can-ai-change-the-way-we-study-i...)
In 5 years, key milestones might include significant improvements in AI's ability to understand and generate complex human emotions, better integration with other AI technologies (like computer vision and robotics), and broader use in personalized healthcare, legal research, and financial analysis.
In 10 years, LLMs and AI are likely to transform various industries by becoming integral parts of decision-making processes, automating complex tasks, and providing highly personalized user experiences. Potential new applications could emerge in areas like mental health support, autonomous transportation, and creative industries.
It will remain in the digital realm all this time.
The only major disruption will be to anything content related. You will be able to generate blockbuster films/series for a thousand dollars.
OF, P0rn basically for a few dollars.
Instagram and tiktok profiles of fake people for a few dollars.
Artists will be able to put out thousands of good songs a month and see what sticks, become big with one peraphs.
You’re already able to generate articles/copywriting/books.
Since content will be free, the only value add will be in how much capital you have available to advertise.
AI is to digital content what china did to physical items.
Only advertising money will make or break a product because anyone can go to an AI and have a great product instantly. Just like you can go to china and get a pretty good product instantly.
First off, LLMs, ChatGPT, DALL-E, etc... these are all just neat party tricks. Generative AI is nothing special and - along with the other comments - going to be the butt of many jokes in the years to come (and already is: see South Park). Everyone clamoring to get on that bandwagon either doesn't understand it or is attempting to capitalize on (and sell it) to people who don't understand it.
That said, what _powers_ those models: encoders, attention, and transformers, has been a massive leap in AI model generation. This cannot be overstated!
I don't care about AI generating a token or pixel. What I care about is being able to throw very complex data at a model and have it "learn" relationships between them in ways I couldn't see before and then being able to perturb the data to see what how that changes the "meaning" of it.
For example, in biotech (my particular field), training models on the gene expression profiles of healthy vs. diseased tissues and then perturbing expression levels (in silico) to determine if doing so encodes the gene profile as "healthy" is ground-breaking. As data profiles of are added it has the potential to also reveal potential toxicities of gene inhibition (or expression) without ever needing to test it in the lab saving $100s of millions.
The party tricks are neat, and they are what bring in the $$. But it's the building-block use-case getting very little attention where all the benefits are going to be had in the coming years. And they are coming fast!
P.S. My personal prediction is that the next massive leap in AI is going to be a paradigm shift away from how we train and simulate networks. The current framework of more and bigger GPUs to process larger and larger models is unsustainable. Someone, somewhere in the next 5-10 years will revolutionize how this is done and THEN we'll have our true, AI "revolution".
I'm a programmer and data scientist. Using LLMs (specifically Claude 3.5 Sonnet) made me at least 10x as productive. I don't write boilerplate code anymore.
Sometime in the next year, I finally get some other human interested in the BitGrid architecture[1] as a general purpose compute platform. It's an FPGA without the hassle (sorta). That helps me get out of analysis paralysis, and make a better case for the architecture.
Over the next few years, people figure out how to best impedance match it with the execution of AI workloads, and we're off to the races.... all those high speed transistors no longer have to sit around waiting for their turn to take part in executing code, and things get much, much faster.
But first... I need something more than a Pascal based emulator for the BitGrid, with no meaningful I/O.[2]
Maybe if I implement it as a blueprint in Factorio?
I think this idea of what it means to be a person of vision is interesting, we’re in this place now where we’re trying to make the future happen. Self driving cars, robots, AI, if you’d thought of all this stuff 100 years ago, you’d still be a visionary but you’d be writing science fiction, If you have these ideas 100 years from now, it would be too late.
AI is amazing, but I think it will continue to be a relatively specific product for the near future. So 10 years from now we’ll probably revolutionize the UX of a lot of products on the market right now, but will the world look fundamentally different than it does now? I’m not so sure.
I think it won’t be a great thing anymore. More like commodity. Is anyone impressed by the difference between the lates iPhone and a model that is 5 years old? Not really, sure these phones get better and better but not to the point of surprise anymore.
1. Building blocks. Your core is GPT-4 level intelligence, similar to a transistor. Other things can be built around that. RAG. Functions. Agents. Building blocks lead to bigger ones. Other forms of blocks. Multimodal. Try out Suno and Midjourney if you haven't. Building blocks are composed of smaller blocks, and as engineers, this is where your salaries are made.
2. Amdahl's & the subsequent Gustafson's Law. More resources will have diminishing gains, until they hit a point where they unlock new uses. GPUs and cloud are a cool side effect from the computing revolution, and they solve all kinds of problems they were never invented for. You'll see LLMs slow down in problem solving then suddenly someone figures out how to build self-3D printing machines or something.
3. Old, uncool paradigms. Procedural generation is incredibly good at controlling quality. LLMs are like play doh, and proc gen gives it a skeleton. Evolutionary programming goes the other way. You can give an agent a problem to solve, and now it's able to figure out paths or branch agents on the way there.
4. Combos with existing blocks. LLM agents + crypto wallet leads to agent ecosystems. Multimodal vision/hearing can also be combined with all kinds of things. LLMs will be available to all apps, and it'll be cheaper than TTS was. RAG will make search cheaper, not enough to threaten Google, but you can quickly search interview questions or emergency medical treatments.
I'd say don't spend too much effort predicting. But play with new blocks as they form - several will be unimportant, some will matter. Two blocks together could be amazing. Brush up on the old paradigms that are no longer taught in average universities.
In an attempt to make interesting/bold predictions:
- <10 yrs Google maps will use photogrammetry + Computer vision AI etc. to make 'street view' continuous, and you will be able to go anywhere and look with any perspective. New layers can start to be added, e.g. infrastructure layers that are captured from crowd sourcing from phones.
- Music will be continuously generated < 5yrs. Within 10, can adjust to auto and manual inputs that nudge it in the direction you want.
- <5 yrs - Business role-based email addresses automatically parsed and routed. E.g. most business bills captured and added to accounting platform.
- Proper HUD AR glasses that are on an open-enough OS that allows hacking + easy app dev, so not a DOA product. Auto labelling of objects, live translation etc. will actually be well implemented and used by many people.
- War is going to look even more terrifying, and the Terminator Salvation vision of hunter/patrol drones manning the skies is precisely what front lines will look like (except higher altitude).
- Defensive war: Multi-layer iron domes start to be erected on nation borders, and we start to hand over to AI such that humans are not immediately/directly in charge of decisions. I.e. defence ops handled by AI, but tactics and strategy still directly managed by humans.
- Offensive war: Opposing nations will start to talk re: agreements about a human needing to make final decisions - To what success I have no idea.
- Low altitude airspace mostly managed by AI, drone swarms + coordination start to be seen as commonplace. >10 yrs Propellor noise to somewhat improve after massive push from citizens, and we eat some efficiency in the interest of reducing noise pollution.
- 20 yrs Maybe... First AI enemy, i.e. a nation state 'battles' an AI that seems to be acting alone or cannot be pinned on a specific human group actor. Will be possible due to the 'scale' that AI affords, not the 'smarts'.
- A Japanese citizen will be more likely to receive 'care' from a robot than from a human... This ratio mostly due to heavy use in eldercare.
- We will hear of edge-cases (similar but smaller situation to tang ping + incel choices) where some people explicitly announce / decide never to interact with humans again, and opt to just interact with AIs only.
- EU legislation that demands open/reviewable algorithms before products be legally available to EU citizens
- 20 yrs: AI 'Oracles' that some people revere (just very insightful, accurate, consistent AIs that appear to exceed human wisdom)
- 10 yrs: MSPs/IT providers will handle ~5x as many endpoints per employee compared to 2024
- Some very very crude prototype of 'inter-animal language translation' that people will want to talk to their dogs with, but it won't really add value for a long time and will mostly just be used for zoology research.
For LLMs, I think every business is currently throwing things at the wall and seeing what sticks. Nothing wrong with a phase of experimentation for a new technology, and I prefer the $200M spent on training GPT-4 to the $500M spent on advertising the mobile app Monopoly Go, but I could see an eventual contraction in this activity and number of start-ups. I feel as though the challenges of intelligence and non-shallow reasoning may be more of a slow chipping away with incremental progress, fighting against diminishing returns, rather than exponential improvement or a "hard takeoff" that some are expecting.
Generally though, deep learning on large data is undeniably a very useful technique. We've seen huge advances in areas like language translation, voice transcription, OCR, visual defect detection, etc. and will doubtlessly continue to see it applied to a wide range of tasks. If the public and investors sour on "AI", it'll just go by another name as it did before.
I'm not making any predictions, but I think we'll have a much better idea of the long-term prospects of AI in late 2024 - mid 2025. Claude Opus 3.5 is supposed to be released later this year, and GPT-5 is expected sometime in 2025 IIRC.
I'm still amazed by the quality of Sonnet 3.5's outputs and OpenAI's realtime voice model. I expected the skepticism from this site's userbase but not to this degree. Anyone who has used Sonnet 3.5 and still thinks generative AI is just a "party trick" is either not thinking hard about AI's potential impact or simply in denial, imo.
The skeptics hang around Ask HN. The optimists and opportunists tend be over in Show HN. Most people can't tell the difference between Sonnet 3.5 and ChatGTP-3.5, or GPT-4o and GPT-4.0.
There's a joke I first heard in the early 90s: "True AI is 10 years in the future and has been for the last 30."
I suspect we'll be in exactly the same place: incremental improvements in LLMs, but the corner case failures will be perceived as significant enough to preclude widespread adoption.
Maybe we'll go back to the "neural network winter" of the 80s and 90s where "anti-hype" seemed to make research into CNNs unfashionable.
Or maybe industry will deploy LMMs widely and just cope with their flaws.
Similar to self-driving cars, future AI will train directly on reality rather than on text documents. The ultra-low-power-very-high-resolution cameras we all will wear will track our eyes and take note of what we pay attention to and ignore (including on computer screens). The cameras will also record audio, and data from a variety of sensors. Like seeing Google cars on roads scanning the roads, everyone will become a Google person, training their personal AI and contributing data to the global Google AI. Small robots will be running around everywhere, both to collect data and to run errands, powered by new battery technologies. A standard will finally arrive for indoor location with sub-centimeter accuracy which will transform the amount and type of data AI's are trained on, and enable many new applications and privacy invasions. AI's will be trained on forests, oceans, lakes, rivers, fields, etc., and start cataloging all of nature and what actually happens in all these places. Some of these mobile sensors will be chemical sensors, to track pheromones, body odors, toxic chemicals, food odors, DNA, RNA, soil conditions, and more, creating a whole new training set and new ways to query and react to the natural world, as well as revolutionizing medical AI's by finally including the environments effect on health of people and animals (including wild animals). Some of this will happen quickly, some of it will take decades.
In the next 2 years, we can expect AI and Large Language Models (LLMs) to become even more sophisticated in natural language understanding and generation, leading to more accurate and context-aware applications in customer service, content creation, and education. (https://studycorgi.com/blog/can-ai-change-the-way-we-study-i...)
In 5 years, key milestones might include significant improvements in AI's ability to understand and generate complex human emotions, better integration with other AI technologies (like computer vision and robotics), and broader use in personalized healthcare, legal research, and financial analysis.
In 10 years, LLMs and AI are likely to transform various industries by becoming integral parts of decision-making processes, automating complex tasks, and providing highly personalized user experiences. Potential new applications could emerge in areas like mental health support, autonomous transportation, and creative industries.
It will remain in the digital realm all this time.
The only major disruption will be to anything content related. You will be able to generate blockbuster films/series for a thousand dollars. OF, P0rn basically for a few dollars. Instagram and tiktok profiles of fake people for a few dollars. Artists will be able to put out thousands of good songs a month and see what sticks, become big with one peraphs. You’re already able to generate articles/copywriting/books.
Since content will be free, the only value add will be in how much capital you have available to advertise.
AI is to digital content what china did to physical items. Only advertising money will make or break a product because anyone can go to an AI and have a great product instantly. Just like you can go to china and get a pretty good product instantly.
First off, LLMs, ChatGPT, DALL-E, etc... these are all just neat party tricks. Generative AI is nothing special and - along with the other comments - going to be the butt of many jokes in the years to come (and already is: see South Park). Everyone clamoring to get on that bandwagon either doesn't understand it or is attempting to capitalize on (and sell it) to people who don't understand it.
That said, what _powers_ those models: encoders, attention, and transformers, has been a massive leap in AI model generation. This cannot be overstated!
I don't care about AI generating a token or pixel. What I care about is being able to throw very complex data at a model and have it "learn" relationships between them in ways I couldn't see before and then being able to perturb the data to see what how that changes the "meaning" of it.
For example, in biotech (my particular field), training models on the gene expression profiles of healthy vs. diseased tissues and then perturbing expression levels (in silico) to determine if doing so encodes the gene profile as "healthy" is ground-breaking. As data profiles of are added it has the potential to also reveal potential toxicities of gene inhibition (or expression) without ever needing to test it in the lab saving $100s of millions.
The party tricks are neat, and they are what bring in the $$. But it's the building-block use-case getting very little attention where all the benefits are going to be had in the coming years. And they are coming fast!
P.S. My personal prediction is that the next massive leap in AI is going to be a paradigm shift away from how we train and simulate networks. The current framework of more and bigger GPUs to process larger and larger models is unsustainable. Someone, somewhere in the next 5-10 years will revolutionize how this is done and THEN we'll have our true, AI "revolution".
I'm a programmer and data scientist. Using LLMs (specifically Claude 3.5 Sonnet) made me at least 10x as productive. I don't write boilerplate code anymore.
Sometime in the next year, I finally get some other human interested in the BitGrid architecture[1] as a general purpose compute platform. It's an FPGA without the hassle (sorta). That helps me get out of analysis paralysis, and make a better case for the architecture.
Over the next few years, people figure out how to best impedance match it with the execution of AI workloads, and we're off to the races.... all those high speed transistors no longer have to sit around waiting for their turn to take part in executing code, and things get much, much faster.
But first... I need something more than a Pascal based emulator for the BitGrid, with no meaningful I/O.[2]
Maybe if I implement it as a blueprint in Factorio?
[1] https://esolangs.org/wiki/Bitgrid
[2] https://github.com/mikewarot/Bitgrid
In 10 years, AI and LLMs will be a joke on The Simpsons in the same way they made fun of the Palm Pilot and cryptocurrency.
I think this idea of what it means to be a person of vision is interesting, we’re in this place now where we’re trying to make the future happen. Self driving cars, robots, AI, if you’d thought of all this stuff 100 years ago, you’d still be a visionary but you’d be writing science fiction, If you have these ideas 100 years from now, it would be too late.
AI is amazing, but I think it will continue to be a relatively specific product for the near future. So 10 years from now we’ll probably revolutionize the UX of a lot of products on the market right now, but will the world look fundamentally different than it does now? I’m not so sure.
Nothing fundamentally different from current state of the art, only gradual improvements.
So essentially we gonna have better and cheaper smart google alternative and advanced code (picture, audio, video, text) auto complete.
Unless another breakthrough happens.
Which can happen tomorrow, or in 50 years. You never know.
I think it won’t be a great thing anymore. More like commodity. Is anyone impressed by the difference between the lates iPhone and a model that is 5 years old? Not really, sure these phones get better and better but not to the point of surprise anymore.
I would look at:
1. Building blocks. Your core is GPT-4 level intelligence, similar to a transistor. Other things can be built around that. RAG. Functions. Agents. Building blocks lead to bigger ones. Other forms of blocks. Multimodal. Try out Suno and Midjourney if you haven't. Building blocks are composed of smaller blocks, and as engineers, this is where your salaries are made.
2. Amdahl's & the subsequent Gustafson's Law. More resources will have diminishing gains, until they hit a point where they unlock new uses. GPUs and cloud are a cool side effect from the computing revolution, and they solve all kinds of problems they were never invented for. You'll see LLMs slow down in problem solving then suddenly someone figures out how to build self-3D printing machines or something.
3. Old, uncool paradigms. Procedural generation is incredibly good at controlling quality. LLMs are like play doh, and proc gen gives it a skeleton. Evolutionary programming goes the other way. You can give an agent a problem to solve, and now it's able to figure out paths or branch agents on the way there.
4. Combos with existing blocks. LLM agents + crypto wallet leads to agent ecosystems. Multimodal vision/hearing can also be combined with all kinds of things. LLMs will be available to all apps, and it'll be cheaper than TTS was. RAG will make search cheaper, not enough to threaten Google, but you can quickly search interview questions or emergency medical treatments.
I'd say don't spend too much effort predicting. But play with new blocks as they form - several will be unimportant, some will matter. Two blocks together could be amazing. Brush up on the old paradigms that are no longer taught in average universities.
LLMs and AI will be as mainstream and in the hands of everyone as smartphones are today.
AI winter will come back. This time to stay.
In an attempt to make interesting/bold predictions:
- <10 yrs Google maps will use photogrammetry + Computer vision AI etc. to make 'street view' continuous, and you will be able to go anywhere and look with any perspective. New layers can start to be added, e.g. infrastructure layers that are captured from crowd sourcing from phones.
- Music will be continuously generated < 5yrs. Within 10, can adjust to auto and manual inputs that nudge it in the direction you want.
- <5 yrs - Business role-based email addresses automatically parsed and routed. E.g. most business bills captured and added to accounting platform.
- Proper HUD AR glasses that are on an open-enough OS that allows hacking + easy app dev, so not a DOA product. Auto labelling of objects, live translation etc. will actually be well implemented and used by many people.
- War is going to look even more terrifying, and the Terminator Salvation vision of hunter/patrol drones manning the skies is precisely what front lines will look like (except higher altitude).
- Defensive war: Multi-layer iron domes start to be erected on nation borders, and we start to hand over to AI such that humans are not immediately/directly in charge of decisions. I.e. defence ops handled by AI, but tactics and strategy still directly managed by humans.
- Offensive war: Opposing nations will start to talk re: agreements about a human needing to make final decisions - To what success I have no idea.
- Low altitude airspace mostly managed by AI, drone swarms + coordination start to be seen as commonplace. >10 yrs Propellor noise to somewhat improve after massive push from citizens, and we eat some efficiency in the interest of reducing noise pollution.
- 20 yrs Maybe... First AI enemy, i.e. a nation state 'battles' an AI that seems to be acting alone or cannot be pinned on a specific human group actor. Will be possible due to the 'scale' that AI affords, not the 'smarts'.
- A Japanese citizen will be more likely to receive 'care' from a robot than from a human... This ratio mostly due to heavy use in eldercare.
- We will hear of edge-cases (similar but smaller situation to tang ping + incel choices) where some people explicitly announce / decide never to interact with humans again, and opt to just interact with AIs only.
- EU legislation that demands open/reviewable algorithms before products be legally available to EU citizens
- 20 yrs: AI 'Oracles' that some people revere (just very insightful, accurate, consistent AIs that appear to exceed human wisdom)
- 10 yrs: MSPs/IT providers will handle ~5x as many endpoints per employee compared to 2024
- Some very very crude prototype of 'inter-animal language translation' that people will want to talk to their dogs with, but it won't really add value for a long time and will mostly just be used for zoology research.
Check out torroidal propellers: https://en.wikipedia.org/wiki/Toroidal_propeller