I currently have 17 GB free space on my root drive, out of 0.47 TB usable space, that’s 3% free space.
I also have an additional harddrive installed, 12 TB, it has 140 GB free, which may seem a lot but it’s actually only about 1%.
Your harddrive is probably pretty full too. Maybe not as crammed, but odds are it’s probably closer to full than empty. I ran a poll on mastodon and approximately half of the respondents, counting 81 people in total, had a harddrive that was more than 75% full.
Pretty much every time in my life I can think of, my storage has been pretty full, to the point where I’d have to regularly find and clean up files.
I’ve doubled the amount of storage space I have many times, starting at I think 80 MB in the ’90s. Now I have tens of terabytes. They are still mostly occupied.
Software exists for the task of identifying which files to delete when cleaning up your harddrive, and such software has existed for a long time. It may be an obvious statement, but this is a problem that people have and have had.
Harddrives tend to be pretty full, regardless of how big they are, getting more storage even by several orders of magnitude doesn’t seem to fix this.
Why is this?
You can construct an entropic argument that there are simply more ways for a harddrive to be full than ways in which it can be empty. Then if we assume we mostly make random changes of state without thinking about storage, we should tend toward a full harddrive.
I don’t think this is wrong, but at the same it’s an incomplete explanation.
The other part of the equation is that a full disk isn’t a problem until it’s so full you can’t put more stuff on it, and at the point it’s so irredeemably cluttered that when you do clean it up, you only have the patience to clean up enough to bide your time, judging the fate of every file on the harddrive is simply too much work.
This is something pretty universal, and at the root of many paradoxes of technology, society, and life.
In software:
- We don’t optimize software until it’s too slow, so it tends to always be kinda slow.
- We incur technical debt until working with the code is so painful we feel we must refactor it, or in less experienced developers, start over. Thus most codebases are pretty messy.
Outside of software:
- We don’t expand road networks until they’re unbearably congested
- We don’t mind our diets until we have to buy bigger pants
I’ve gone from having a full time job, to being self-employed, and despite having no obligations and nobody to tell me what to do, I still feel just as busy, if not more so. My schedule seems to be another harddrive that is full.
In most of these cases, waiting until we’ve reached the pain point where we’re forced to confront the issue makes solving the problem a much bigger ordeal than if we’d addressed them along the way.
At the same time, premature optimization is seen as a bad thing.
How can we reconcile these two facts?
The solution is perhaps to weaponize Jevons paradox and impose real constraints to optimize against, constraints that are smaller than the sum of all available resources.
This is very obvious when we’re taking about personal finances, where we call it a “budget”, but repeatedly surprising elsewhere.
Let me tell you, if you deploy and dogfood your software on a raspberry pi, then it will be fast on a Threadripper too. If you can navigate a codebase in vim in an 80x25 terminal, then you can navigate it in a powerful modern IDE as well.
Resources and capabilities are tricky. Intuition often sells the idea that they allow us to do more, but often they allow the same, but at greater cost.