This is mostly true, but you’re glossing over the fact that AI will always be tied to concentrated capital due to the cost of training state of the art models. I understand why you argue that these flaws are not inherent to the technology, but in a practical sense, they might as well be considered inherent. It’s unlikely that the market will open up any time soon so that you can get your LLM from someone who isn’t a fascist billionaire tech bro.
This is mostly true, but you’re glossing over the fact that AI will always be tied to concentrated capital due to the cost of training state of the art models.
I think this is a false assumption.
We don’t know what the ultimate state of AI will be, we only know that we’re at the beginning of the journey.
Look at other technologies in similar positions. At one point computers were like this. Nobody but the military and the largest companies could afford them and now you can buy children’s toys with more storage and processing power than was available to the entire US military at the dawn of computers.
We had no idea what computers would become, even the experts making guesses that sounded wild at the time were off by several orders of magnitude. Bill Gates once said “640K [of RAM] ought to be enough for anybody”, modern PCs have around 100,000 times that.
The reason that AI seems like it’s only affordable to concentrated capital is because models using a large amount of parameter require a large amount of RAM. The 5TB of RAM estimated for some of the largest frontier models is a lot, about 100x a workstation PC. That’s a few orders of magnitude less than 100,000x. Given equal scaling speed, we’re closer to running frontier models on our desktop than we are to the dawn of computers.
In addition, the research is leaning towards there being a ceiling on how useful it is to add more parameters (use more RAM). Doubling parameters doesn’t result in doubling of capability so there is effectively a soft cap on the amount of parameters.
Hardware is growing near exponentially while the RAM demands of models have a ceiling.
Keep in mind that this only relates to the state of the largest LLMs. You can train generative models on consumer graphics cards. Object recognition, classification, etc are all tiny compared to LLMs. These are also AI and, I’d argue, that they actually produce more value than LLMs (which are failing to deliver on their promise of replacing white collar labor).
It’s unlikely that the market will open up any time soon so that you can get your LLM from someone who isn’t a fascist billionaire tech bro.
You can find open weight models (of all kinds, not just LLMs) freely available here: https://huggingface.co/
Many can be run on locally on gaming hardware. Some of the classification and object recognition models can be run on microcontrollers (a popular use it to use them to make aimbots in FPS games).
There are larger models that can be run on more expensive dedicated hardware (like NVIDIA DGX Spark, ~$5,000) if you want to keep your projects completely internal. Or you can run those open weight models on hardware hosted in datacenters ran by companies you research to be ethical.
You can use AI without using one of the AI companies.
This is already not the case for computers. And you’re trying to argue that we might one day be able to run the bigger models on consumer hardware, but I said it’s too expensive to train them. And you know that the cost isn’t just the RAM right?
Sure, my saying they will “always” be tied to concentrated capital is slightly hyperbolic. You never know. But it takes a whole lot of optimism to hope that LLMs will be decoupled from the billionaire class, hence why most analyses of the problems LLMs cause discuss them in this context.
This is already not the case for computers. And you’re trying to argue that we might one day be able to run the bigger models on consumer hardware, but I said it’s too expensive to train them. And you know that the cost isn’t just the RAM right?
If you replace everything I said about RAM with compute and replace the Bill Gates quote with Thomas Watson saying that the “world market [has room for] for maybe five computers” then it’s the same argument. They’re all ultimately built on technical advances in lithography and so growth in RAM and compute largely stem from similar sources.
Sure, my saying they will “always” be tied to concentrated capital is slightly hyperbolic. You never know. But it takes a whole lot of optimism to hope that LLMs will be decoupled from the billionaire class, hence why most analyses of the problems LLMs cause discuss them in this context.
I understand why it is done as shorthand.
I know this likely doesn’t apply to you, as you seem to be capable of understanding the nuance, it’s that a lot of people on social media (by people I’m including the bots) don’t make that distinction.
The same vitriol that is, rightfully, directed at OpenAI/Anthropic/Google/NVIDIA is also directed at things completely outside of the corporate LLM hellscape.
I’m mostly frustrated at seeing the sciences, researchers and open source AI projects catching strays and being painted with the same brush.
It’s useful shorthand among people who understand the nuance, but using that shorthand on social media invites misunderstanding and that is causing collateral damage.
Keep in mind that this only relates to the state of the largest LLMs. You can train generative models on consumer graphics cards. Object recognition, classification, etc are all tiny compared to LLMs. These are also AI […]
The linked Codeberg page refers to LLMs in their current, corporate-controlled shapIe. It is about using LLMs for coding.
This is mostly true, but you’re glossing over the fact that AI will always be tied to concentrated capital due to the cost of training state of the art models. I understand why you argue that these flaws are not inherent to the technology, but in a practical sense, they might as well be considered inherent. It’s unlikely that the market will open up any time soon so that you can get your LLM from someone who isn’t a fascist billionaire tech bro.
I think this is a false assumption.
We don’t know what the ultimate state of AI will be, we only know that we’re at the beginning of the journey.
Look at other technologies in similar positions. At one point computers were like this. Nobody but the military and the largest companies could afford them and now you can buy children’s toys with more storage and processing power than was available to the entire US military at the dawn of computers.
We had no idea what computers would become, even the experts making guesses that sounded wild at the time were off by several orders of magnitude. Bill Gates once said “640K [of RAM] ought to be enough for anybody”, modern PCs have around 100,000 times that.
The reason that AI seems like it’s only affordable to concentrated capital is because models using a large amount of parameter require a large amount of RAM. The 5TB of RAM estimated for some of the largest frontier models is a lot, about 100x a workstation PC. That’s a few orders of magnitude less than 100,000x. Given equal scaling speed, we’re closer to running frontier models on our desktop than we are to the dawn of computers.
In addition, the research is leaning towards there being a ceiling on how useful it is to add more parameters (use more RAM). Doubling parameters doesn’t result in doubling of capability so there is effectively a soft cap on the amount of parameters.
Hardware is growing near exponentially while the RAM demands of models have a ceiling.
Keep in mind that this only relates to the state of the largest LLMs. You can train generative models on consumer graphics cards. Object recognition, classification, etc are all tiny compared to LLMs. These are also AI and, I’d argue, that they actually produce more value than LLMs (which are failing to deliver on their promise of replacing white collar labor).
You can find open weight models (of all kinds, not just LLMs) freely available here: https://huggingface.co/
Many can be run on locally on gaming hardware. Some of the classification and object recognition models can be run on microcontrollers (a popular use it to use them to make aimbots in FPS games).
There are larger models that can be run on more expensive dedicated hardware (like NVIDIA DGX Spark, ~$5,000) if you want to keep your projects completely internal. Or you can run those open weight models on hardware hosted in datacenters ran by companies you research to be ethical.
You can use AI without using one of the AI companies.
This is already not the case for computers. And you’re trying to argue that we might one day be able to run the bigger models on consumer hardware, but I said it’s too expensive to train them. And you know that the cost isn’t just the RAM right?
Sure, my saying they will “always” be tied to concentrated capital is slightly hyperbolic. You never know. But it takes a whole lot of optimism to hope that LLMs will be decoupled from the billionaire class, hence why most analyses of the problems LLMs cause discuss them in this context.
If you replace everything I said about RAM with compute and replace the Bill Gates quote with Thomas Watson saying that the “world market [has room for] for maybe five computers” then it’s the same argument. They’re all ultimately built on technical advances in lithography and so growth in RAM and compute largely stem from similar sources.
I understand why it is done as shorthand.
I know this likely doesn’t apply to you, as you seem to be capable of understanding the nuance, it’s that a lot of people on social media (by people I’m including the bots) don’t make that distinction.
The same vitriol that is, rightfully, directed at OpenAI/Anthropic/Google/NVIDIA is also directed at things completely outside of the corporate LLM hellscape.
I’m mostly frustrated at seeing the sciences, researchers and open source AI projects catching strays and being painted with the same brush.
It’s useful shorthand among people who understand the nuance, but using that shorthand on social media invites misunderstanding and that is causing collateral damage.
The linked Codeberg page refers to LLMs in their current, corporate-controlled shapIe. It is about using LLMs for coding.
Your argument is rather leading away from that.