A large language models (LLM) can be a very useful assistant in most fields, be it software development, engineering, research, law, medicine and medical science, etc. LLM themselves are not very knowledgeable though. They can be considered advanced version of the next word predictor that has been around for a while (remember Botnik’s Harry Potter that got much attention back in 2017 and 2018? They used a predictor but instead of generating one next word, it generated multiple ones and the user had to select what they feel appropriate). We also have weather models that also do something similar using different types of Markov process.
To make them useful, we need to use them in an agentic fashion and integrate different tools that they can use to understand the depth of the problem and figuring out a solution. The reason they can be so good is because they are very good at information retrieval. We, humans, have certain limitations regarding information retrieval: we only understand a certain form of information (plain language, database tables, etc.) and not others. That being said, while LLMs can be trained to understand most forms of information, they can still make bad decisions because of their underlying implementation: they are still “greedy” which means they tend to average out things and may ignore specialized instructions even when you explicitly ask it to do so, because it doesn’t understand the significance or usefulness of that special instructions unless explicitly trained it in that particular area. (Nowadays people use “Make no mistakes” as a joke in forums for the exact reason: you cannot ask LLM to not make any mistakes since it doesn’t understand what mistake is.) Moreover, there is also a major issue with LLM: context window limit. In an agentic setting, people can bypass this my compacting the context to some degree. But this process is lossy: LLM might still forget some important bits and pieces from the previous conversations. One way to prevent it is storing important information in persistent files (preferably stored in the project). But when you are reliant on LLMs to most of the work, it might actually be very difficult to go through the files to figure out whether these files are useful at all. Another issue is that: LLMs are very costly. Even when you self-host a good open-weight LLM (e.g., DeepSeek v3 or Qwen3 are pretty good models), you’d still need to pay for those tools such as a search API that the LLM can use to query the web as part of information retrieval. There are some free quotas available, but they are inadequate.
Corporates can still fix much of the issues by self-hosting a powerful LLM and paying for those APIs. But for most open source development, this is not the only issue: one major issue that even predates LLM is maintainability. Those who are really committed to developing and maintaining open source apps (not those who are doing it just to get better jobs), often face the serious hurdle of maintainability. Motivation is one thing, burnout is probably one of the frequent reasons for quitting open source development. Burnout can happen for many reasons: the project can be too complex to be maintained (considering most software are maintained by just one or two people), the users can overwhelm the maintainers with issues, major changes in the platform it targets, or other frustrations. Sometimes, even if you have enough motivation, you may simply not have enough time to contribute to the project. LLM adds to it: it can disconnect you from the project as you are relying on LLM to do things for you, and one day, you may find that you can no longer recognize the project at all due to all the changes made by LLM. Some argue that properly reviewing code written by LLM helps, but the problem is that reviewing is a tedious and skillful job. If you do not code for a while or have no idea about the library or framework, reviewing will not help at all. Rather, you’ll soon get tired of it and accept whatever changes made by the LLM. This is harmful for open source projects.
The issue doesn’t end here though. As a maintainer, you would also need to have a policy on whether to accept LLM contributions or not. If you are the sole maintainer, accepting LLM contributions could be shooting yourself in the foot. If your project is popular enough, you could be bombarded with LLM-generated features or fixes that can quickly go out of control, or even worse, you might end up questioning your life choices (and I am not exagerating a little bit).
Then, there is also the licensing issue. LLM might generate something that comes directly from its training dataset (or a derivative of it). Since it is usually fed world data without any licensing issues in mind, this can make a huge mess. Although some large projects are saying that any AI generated content is the author’s responsibility, whether it can hold in court can be argued. Because it is more or less clear that anything “substantial” generated by an LLM cannot be solely attributed to the author. Then, when you do co-author the work with LLM, you may not be able to apply any license to it. The complicated situation created by the legal issues (that different parties interpret differently) must not be ignored by maintainers who actually care about open source.
In this situation, SF Coservancy published an article that offers some recommendation. But some of them are very broad and may not applicable to projects led by just one or two individuals. So, here goes my opinion on how to utilize LLM in open source development without harming maintainability and licensing issues:
- Use a copyleft license. Many licensing issue can be vanished simply by using a copyleft license. But this might not be useful for libraries where the target audience may be those who also develop commercial closed source software.
- Testing and verifying a feature. Whether it is a new or an existing feature, if it becomes too complicated, you can ask an LLM to verify the feature and improve test coverage. (But be aware that you need to be very specific about what you want. If you ask it to generate exhaustive tests, the result will be similar to asking “Make no mistakes”.)
- Fixing bugs. Locating and fixing bugs can be very time consuming and one of the most unrewarding part (apart from generating unit tests) of any software development. This can be offloaded to AI to fix those bugs. Again, it still needs to be supervised as AI sometimes misunderstands the actual issue or simply applies a fix that is not so great as you expect.
- Implementing a feature that is simple in nature. If you know most of the requirements of a feature beforehand and ask LLM to translate that into code, the work can solely be attributed to you. You can leverage this to develop simple and repetitive features that you, otherwise, would be too lazy to implement yourself. It is also quite easy to review the implementation without becoming exhausted. For large features, while you can converse with LLM about it, but you should never let LLM dictate you in the design and implementation process. If you cannot decide something, the best way to come up with a good decision is by studying a little about it first instead of blindly trusting AI. It also prevents the licensing issue.
- If you are low on maintainers, accepting AI contributions is probably not a good choice. While you yourself may use as much LLM as you like, accepting contribution from other people can be troublesome for long term maintainability as you would need to spend a lot of time reviewing stuff which is not very rewarding.