File under: Development, Database, Fragmentation

blog/ Fragmentation Bites

An old problem, that I thought had gone forever

Defragmenting your drives used to be a big thing. There were lots of utilities you could buy that promised to make your drive go faster, work more efficiently and store more.

Long, excruciatingly boring articles in print magazines warned naive users that we should defragment our drives weekly, or daily, or whatever. Users would advise each other constantly. "You should defragment your drives daily" "Oh yes, that's very important".

It wasn't important, but people did it anyway. And then Operating Systems got better and nobody ever had to do it again. Until today.

My faithful SQLite database, which I had been using for several years, stopped working with a "disk I/O error". When I enabled more logging with

.log stderr

I saw

(1034) os_win.c:32118: (665) winSync(H:\mydb\json.sqlite) - The requested operation could not be completed due to a file system limitation

Panicking just a little bit, because it held 125 Gib of data that wasn't backed up, I checked on line to discover that "A heavily fragmented file in an NTFS volume may not grow beyond a certain size ".

So for the first time in 20 years, I am actually running "defrag". But now I'm doing it on a 2 Tb USB drive, so it is actually taking longer than my original 2Mib hard drive.

Progress!

PostScript: It took more than a week!