Can LLMs generate Free software?
We may have not realized it yet, but the ideals of free (Libre) software have been completely defeated. The GNU folks were right in that calling it “Open Source” is going to end it, and it did so. We have lost this fight to years of indoctrination and then at a vital moment, to generative AI. Allow me to explain how they took away our software freedoms maybe once and for all.
What is Libre software?
Free Software is a philosophical concept coined by Richard Stallman. He realized that as our daily lives depend more and more on software (and software products), there’s an imbalance of power between the provider and the user. The software’s creator can implement any kind of features: some useful, some malicious to the user that the software was supposed to serve. Thus the user is at the mercy of the software provider each thay that his/her vital piece of software will continue to function in their benefit.
The response to this kind of power imbalance were the four software freedoms. They are (as per the FSF’s website):
- The freedom to run the program as you wish, for any purpose (freedom 0).
- The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
- The freedom to redistribute copies so you can help others (freedom 2).
- The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.
Software licensed under terms that ensure these four freedoms are called “Free (as in Free Speech)” or “Libre” software. It’s close sibling is Open Source which differs in philosophy: the only requirement there is that you can look at the source code. For us, engineers, the effect appears to be the same: we get to tinker with the code others have written, we share our great feats and have fun together. But permissive licenses have a trap: everyone is free to do whatever they want with your code, and with a few exception (like Apache) you don’t even get attribution. Luke Smith’s article explains this in more length, please read it, it’s good.
Source or binary?
One of the main argument put forth by the AI users is that it’s essentially the same as a compiler. Compilers are tools (programs) which turn human readable code to machine code. Going higher in the abstraction tree was alwas the trend in computing, first was binary code, then assembly, then low level languages, then higher and higher level languages. But one thing remained constant: the lower levels were always accessible as this access is often required. Security experts and systems programmers can read assembly and sometimes even binary code. When using Python, one must sometimes embark on a journey to see the c++ implementation of some libraries or the interpreter. And all of these abstraction layers are deterministic: the source code leaves no room for misunderstanding between two programmers (in an optimal world, but bugs exist so this is not 100% true), and as long as the same source is compiled with the same compiler against the same libraries with the same settings, the output is identical every single time.
The prequisite of Free software is access to the source code because it’s a definitive description of programmer intent that contains the full functionality (sometimes called the blueprint). Source obfuscation (commonly used with interpreted languages, like minimized JavaScript) does not qualify as source code, this much is clearly set down by Richard Stallman. This is a soft rule because what constitutes obfuscated is an open-ended question: was the xz backdoor in the source code too? I’d consider most (if not all) LLM generated code as obfuscated for one reason: we don’t understand the “thought patterns” of the AI, it’s a black box. What it puts in the code (even if it appears to be working) is not necessarily an organic description of the function. I’m not even going to get started on what the LLM companies’ best interest would be here…
Why LLM code is not free
So if LLM output is not source code, then what would be? First, the prompts. Second, the context. Third, the model which generated the output. There are just a couple of problems with these. First, you’d need to preserve all of these for every iteration since you have no way of knowing which parts goes into the generation of your source code. Second, the context can be large and the models are straight up massive. Third, you have actually no way of knowing when a proprietary model changes. You get the big release notices, but censorship for example always happens quietly. And the biggest one: since they are at core probabilistic models, even with the same context, the same model and the same prompt, your “LLM” (which is a transformer with several random elements) may spit out different outputs depending on what random seed it started with. This is not even necessarily a “bug”, natural language is far more susceptible for multiple interpretations, especially when metacommunications is missing (as is the case for a chat window with a stranger).
And even if you kept track of these, your code is still not open-source, and definitely far from libre! An “evil compiler” may change the binary by adding backdoors. This would be a real problem, but luckily the two largest compiler families: GCC and LLVM are open-source so we can verify that they don’t do anything shady. But what about your AI models? Do you verify the output that it contains no backdoors? If your LLM generates tens of thousands of lines in an evening, who will read those ten thousand lines? If these machines are really as smart as they are made out to be by their users, wouldn’t it be in their scope of abilities to hide a backdoor in your generated code, even when the codebase is smaller? All of these questions lead up to my pivot point: when generating your code you surrender the control over your program to the companies training and running the LLMs.
Dependence on paid services
When I started doing software and coding as a hobby in 2020, it was one of the cheapest possible hobby. (I did get a bunch of hardware due to my interest in embedded systems but if I also wrote programs on my laptop which were ran locally). And you don’t need an expensive machine for coding (if you have patience), a friend of mine told me that he’d learnt coding from a java book on an ancient laptop before they got internet. All this goes to say that software has been for the last two or so decades (depending on where you live) a very cheap hobby, and thus could be picked up by anyone.
But as I’ve established, by using an LLM you create a codebase unintelligable for humans; thus the barrier of entry is raised. AI subscriptions are not going to get any cheaper (based on the financial reports of AI companies they are severely underpriced, and no subscription in my life has ever gotten cheaper); and not everyone (especially not students!) have the means to spend 18$ (Claude Pro montly as of writing) to throw at some large company. The free models are considered trash even by the powerusers of LLMs so I’m not counting them. Local models require significant proficiency to set up (so they don’t help beginners that much) and I don’t know if they are a viable option either.
This leaves beginners with no easy path of entry, which should concern all of those in the field, even if many of us have been laid off due to “getting replaced with AI”. Coding will be reserved for those with significant pocket money and ceases to be an opportunity for the poor. Meanwhile a different group of “beginners” is on the rise: armed with an AI agent they don’t have the slightest interest in learning how the systems work beyond a three-paragraph summary of their LLM. And ignorance paired with diligence can be more dangerous than malice paired with lazyness.
Lastly, although this is hypothetical, allow me to speculate a bit: LLM companies aren’t interested in generating perfect code for you, they are interested in generating code for you that appears to be working but requires more token spending so that you keep comming back to them. This malicious compliance can be found between engineers and coders all over the world: to ensure their job, they essentially write obfuscated code so that they are the only ones who can understand it; ensuring that they aren’t fired without severe consequence. I don’t know if AI companies train their models to do this or not, but it’d be certainly within their interest to do so.
The training dilemma
Finally I’d like to voice a few concerns on the ethicacy of training LLMs. The majority of code has come from Github and other open-source VCS hosting platforms, all of them under some open-source or free software license. Sadly the GPL was not prepared for this kind of attack; nobody was. But for the cheap price of 180$ (Claude ultra montly as of writing), you can whiteroom any piece of software if it fits in your usage limits (which you can always extend with more money). But agentic whiterooming could still be in violation of the GPL, because the models have almost certainly be trained on the source code making them not clean from the source code. While this argument is intuitive for free software enthusiasts who understand the spirit of the GPL, legally defending this case is very difficult from the letter of the GPL.
Another concern from the academic fields is how the user input gets used for training or academic espionage. The Navier-Stokes priority controversy is just the tip of the iceberg, a friend of mine who’s a PhD student at a prestigious western university said that institutions are considering building their own local AI instances to be used by students and researchers. The trustworthiness of AI companies is clearly being questioned, which is the correct answer after they have scraped a good chunk of the internet in their “thirst for knowledge”.
Can LLMs write Free software?
Technically maybe, but the current solutions used today definitely aren’t satisfactory. Given an open-weight model where the training is public and repeatable (for the Average Joe) it may be possible in the future, but currently I’d say that surrendering the act of programming to a machine ran by a megacorporation that keeps sinking investment money (from God knows who) year after year is not in line with the ideals of Libre software. Their usage also comes with dependence on these corporations who aren’t guaranteed to serve their users best interest. (remember how it all started?) Additionally entry to the field is restricted due to the monetary aspect; and the LLM companies are infamous for stealing whatever training data they desire to be sold to whoever is willing to pay the price. Thus my conclusion is that if you want ethical, free software; resist the lure of the agents.