81% of people more wary of dependencies due to AI

I recently polled three different communities (LinkedIn, Mastodon, Reddit) to see if technical folk felt more wary, less wary, or the same about software dependencies in the advent of developers using AI with varying degrees of oversight.

81.63% of people voted that they are more wary. This was much larger than I figured, I actually thought people might have said they were less wary.

How I asked

I’ve linked to the polls at the bottom of this article, but the premise and question were generally worded as;

The reality is most of us aren’t going through every line of code for every dependency (node module, ruby gem, etc.) we add to a project, however the assumption largely held was these are open tools written by folk who at least know enough to have made the tool in the first place.

AI tooling changes that assumption.

I’ve a question for folk working in product/web teams (boosts appreciated)

Does the fact that some folk are happy using AI output with varying degrees of oversight make you:

The three poll options were;

• More wary of adding dependencies
• Less wary of adding dependencies
• The same / Don’t care

The results

Across the three polls there were 49 votes. Not a huge amount, I speak to this later.

• (40) More wary of adding dependencies
• (0) Less wary of adding dependencies
• (9) The same / Don’t care

This genuinely surprised me. I thought that either most people would think it a moot point, or that people would be less wary.

I’m personally more wary, which I will touch on in a second, but lots of proponents of AI use in software development tout that you can give it tasks often ignored by people, I would have thought dependency validation and verification might have been one of them, I guess not.

Why I asked

I have some pretty strong biases against the use of LLMs in software development and it is my professional opinion that the software development landscape is a more dangerous place with LLMs/AI in it. That being said, other opinions are available and I wanted to see how the wider industry felt.

Should you be more wary of dependencies?

I think so. AI code helpers have been known to add dependencies they don’t actually need into your codebase, and since there aren’t tests in place for “do we actually need this” everything will pass.

Even if you are hand-selecting your dependencies, I have seen plenty of maintainers adopt an AI-first approach to development, and more and more people are shipping code generated by AI tooling.

Let’s assume the code is being read and understood by developers before a PR is generated, the sheer speed of which new changes can now come is going to lead to review fatigue and mistakes will happen.

Heck, even if they didn’t happen, the ease of which a new feature for a dependency can potentially be added means a tool you thought was really good at just doing X, now does X, Y, and Z, which probably isn’t why you initially reached for the dependency for in the first place.

What can be done about wariness to dependencies?

Honestly, as an industry, we probably always should have been more wary about our dependencies. There are some things I think teams should start adopting.

My advice can largely be distilled to; slow down.

Aim to reduce your overall dependency footprint. You can do this by regularly reviewing the dependencies you have and asking yourself. “Do we still need this?” and if it is, follow up with “Is this something we could manage internally?”. There are a lot of dependencies we add which amount to a handful of small methods we could probably manage ourselves.

Decide on some pre-approved dependencies. These are ones which on any project your team has the go-ahead to install without question. Anything not on the pre-approved list needs to go through some sort of approval process.

Understand the changes being made. When adding or updating a dependency, look through the changelog and understand if the changes being added are in keeping with how you want the dependency to be maintained. If extra features are being added, consider if there is a more appropriate dependency for your needs.

Improve your test suite to catch potential errors caused by dependencies. The scope of this will very much depend on your project, but treating all dependencies with a bit of wariness isn’t a bad practice.

Thoughts on the poll itself

Here are some thoughts I’ve had on the poll itself.

Small numbers

At 49 total votes, this was clearly a small poll, however the voting intent was strong enough that I did think it was worthy putting this article together.

In the future I would look to expand my network to get a better sampling of data.

Multiple sources

Because I started the poll in three different places, I have no perfect way of knowing if people voted more than once. There isn’t really any overlap between the networks, so I very much doubt it. If I had a big enough network (see above!) I would probably have one poll and point multiple places to it, alas!

More options needed

Someone on Reddit rightly pointed out that having “The Same” and “Don’t Care” as the same option isn’t right since they are two very different stances.

In future I would either split out options such as this, or remove “Don’t Care”. The reason I feel I could drop “Don’t Care” is because I think someone who bothers to respond to a poll such as this much care somewhat.

Poll data

Here are the links to the (now closed) polls, feel free to follow the accounts if you’d like your opinion registered in future polls!

• Reddit
• LinkedIn
• Mastodon