Ok in my post on the whole situation with routers in the US I mentioned something about supply chain attacks and as someone who works in the software world those things are absolutely terrifying and I think that it deserves a bit of a deeper dive.
So, for people that work in software, probably going to be a bit boring here, but for people that aren’t in software for a living you need to understand how software works when people are building products for you to use.
Software can be complex, so complex that in a lot of cases people will re-use code to prevent having to rewrite things from the start each time they go to do something. This can come in a bunch of different forms but it usually winds up as being a dependency that comes into play when you go to use a product.
So let’s say that I’m building software that tracks inventory. Details don’t matter much but at some point I’m going to have a requirement to store data for use of the product, and chances are that I’m going to be using some type of database engine to allow that to happen. So I can go and write my own database engine from the ground up and go through all the work of getting that developed, performance tuned, and ready to go.
Or I can just install PostgresSQL, a FOSS project that already does this. It has solid documentation, community support, and a history of being able to scale in a way that’s probably in excess of what I’m going to require.
The same thing applies to my user interface. I can build my own from the ground up, or I could use something like Gradio that provides me with a lot of building blocks to make building my UI quicker so that I’m not having to reinvent the wheel.
Now here’s where things start to get messy. While our application only has two dependencies each of those products have their own dependencies, and each of those dependencies have their own dependencies, and each of those dependencies have potentially more dependencies that come into play. How deep is that rabbit hole?
You can get to the bottom of that rabbit hole, and I’m sure that there are companies out there that will spend the time to do so. However I’ve worked around software developers for a long time and as sure as I am that there are companies that handle this properly I’m just as sure that there are many that do not.
So why is this important and how does it become an attack vector?
Let’s say that my inventory application becomes something crazy good, winds up getting used all over the place, even to the point where it’s now in government systems being used to track things that are considered very important. Let’s even say that whatever the application is tracking is important enough that there are people who are now very interested in breaking into those systems and getting access to the data that they contain.
So now my little application is a target. Potentially one that’s worth some money to somebody who is able to break into it.
So now that I’m a target people are poking at my software, looking anything that would allow them to get into my product and mess around. However, let’s say that for purposes of this post that I’m reasonably competent and that my app is solid enough that attackers can’t get in by going after my code. So now what do they do?
Well my product has some dependencies, and those products have dependencies, and lets say they walk down that chain and eventually come across something used by one of my dependencies – and now by extension used by my product. Now the objective is to get control over whatever that is and try to bend it to their will.
Perhaps they submit some helpful bugfixes that hide some malicious code. Perhaps they bribe or buy the control of that dependency from the person who is in control of it now. In either case they now have control over that dependency and by extension can now get into my chain of dependencies and start looking at ways to cause problems for my product. Perhaps they write code that steals passwords and other secrets. Perhaps they find a way to build code that lets them copy data out of my product. In either case the attacker has managed to breach my product.
Now let’s say that I’m really good that what I do and I maintain a full software build of materials and I know that this dependency exists and is something that I have to keep track of and watch out for problems with. In a lot of cases dependencies are just used by the people building the code without a huge amount of review of the actual code involved. That type of review is something that’s fairly labor intensive. So even if I am crazy good and willing to put the time in to review the code there’s still a chance that it might be outside of my area of expertise and I might not catch what was being done and flag it as being malicious.
Let’s say that I’m as good as possible, and I’m paranoid as hell and decide to fork whatever I’m using and freeze things so that I’m not pulling in the changes from the products once I have things working. At some point there’s still going to be some problem that I run into that will require me to update the dependencies that I’m working with either because of a vulnerability or functional bug that needs to be fixed. At that time there’s still a chance that things that I don’t want get into my product.
The software industry as a whole has processes in place to try to catch this stuff, but there’s still a lot that just seems to run on trust, and some programs pull in a lot of libraries for simple things. Hell, a few years ago a small utility (11 lines of code) in NPM was pulled from their repository over a dispute and managed to cause enough chaos in the course of a couple of hours that NPM wound up restoring the thing from backups to keep the world working properly. In this case the supply chain attack wasn’t being done to break into systems or steal data, it was a protest of sorts but the end result was very disruptive.
So, even doing everything right, and being very good about the security of my product there’s still a very real possibility that I’m getting breached, and if I’m not paying attention and happen to catch what’s going on this type of breach could run for a very long time before it’s caught, patched, and no longer a threat.
That’s why these supply chain attacks are terrifying. And if you have coworkers or other folks who don’t seem to see this as a issue that’s a pretty big red flag to watch for.