Disable 'Indexing' in Windows 7 to Speed Up Deletion of Many Files

I’ve been on Windows 7 since the RC was released — that coupled with the fact that I’m a blogger makes me an expert, you should trust me and take my word on everything unconditionally — and while I’ve been overall pretty happy (20 tips and tricks, 8GB performance) there have been a few niggles here and there (Windows rot).

One of the more prominent ones I’ve noticed is the poor performance when dealing with large amounts of small files. Before Windows 7 was officially released, generally slower file operations (Copy, Move, etc.) were reported and to some extent fixed in the Beta 2 and RC releases, and a bit more with a follow-up hotfix to the public release. I don’t know if SP1 had additional fixes for file ops or not.

At the time I had an inkling that the performance issues weren’t so much that Microsoft’s device drivers had gotten worse, but that too much work was being done to copy, move or delete files. The most likely culprit was updating the system-wide search index with the second culprit being Shadow-Copied files needing to have their revision records updated.

Given that Shadow Copy is optional and must be turned on, I looked at indexing first.

ASIDE: As an Eclipse Java PM in my past life, I had workspaces that contained 100s of thousands of .java source files, so trying to erase or move these took hours — I decided it was long enough to investigate the issue and try and trim down the time.

I first ran into this problem when I tried to delete a set of workspaces I had on my hard drive. Executing the Delete operation hung at the “Calculating…” discovery step of the delete for more than 10 mins before I tried to figure out what was going on:

No files were getting deleted, the damn thing was just counting the files it would delete in a day or two once it was done counting.

I popped open the properties on the parent directory to see what the hold-up was and found out it was pretty huge:

So I did some digging online and found folks suggesting that you disable ‘Indexing’ on directories with tons of files that you want to perform a mass-operation on. You can do that from the Advanced… menu:

This operation itself took about an hour to apply, but given that I was looking at a 6hr delete for the previous operation, it seemed to be worth it:

My guess is that Windows 7 went through and updated the NTFS metadata on every single file contained in that parent directory to flip the switch on indexing. Not sure what Mac or Linux/EXT4 or Sun/ZFS would do in this case or if properties can be hierarchical, but that seemed like a very 1990s way of adjusting file properties.

Once the operation had finished I re-executed the Delete operation on the parent folder and saw the “Calculating…” finish much faster this time, maybe about 3 or 4 mins. The delete operation itself was scheduled to finish in under 30 mins, quite a bit different from the previous estimate of “LOL, you will die waiting” that Windows 7 had given me before:

As far as I know this tip effects directories with tons of files, not necessarily directories with very large files.

If you guys have any of your own favorite Windows 7 tips, let us know!

, , , , , , ,

7 Responses to Disable 'Indexing' in Windows 7 to Speed Up Deletion of Many Files

  1. James March 18, 2010 at 2:29 pm #

    I am now almost done deleting about 300,000 files (although it has said 35 seconds remaining for the last 5 minutes, so who knows…) Anyway, your tip should be handy next time!

    ooh, 30 seconds remaining!

    • Riyad Kalla March 18, 2010 at 3:05 pm #

      Oh man that has to be painful… even if you do the index stuff that’ll take forever to run on all those files. Hopefully you won’t need to do that again for a few years 😉

  2. Will Smith January 27, 2011 at 3:55 am #

    My 75Gb SSD (windows C: drive) has just filled up. After much hunting and uninstalling minor things, I found out the culprit – Windows 7 Indexer. Because it was also indexing a large backup hard disk, it was using a ridiculous 32Gb of data!

    So I’ve turned off indexing, deleted all the indexes, and suddenly I have 38Gb free of 74Gb. So HALF my C: drive was just indexer’s data. Crazy…

    • Riyad Kalla January 28, 2011 at 8:53 am #

      Will, that is nuts that Windows’s index would grow *that big* and not provide some sort of indicator to you.

      It seems to me that it should give you a warning when the index itself is consuming 1/2 of your disk space… damn.

      Good to know.

  3. Mma75Online October 6, 2011 at 7:36 pm #

    For me i use task manager to delete files quickly
    ctrl+alt+del — start task manager — File ( menu ) —- new task ( run ) —- Browse
    then i select the folder and use shift + delete & its done

  4. Christian February 11, 2012 at 4:07 pm #

    You could do it from the command prompt

    Removing a directory.

    RMDIR [/S] [/Q] [drive:]path
    RD [/S] [/Q] [drive:]path

    /S Removes all directories and files in the specified directory
    in addition to the directory itself. Used to remove a directory

    /Q Quiet mode, do not ask if ok to remove a directory tree with /S

  5. DR July 20, 2012 at 3:58 pm #

    Thank you! Who would have thought that trying to delete a whole bunch of files would be such a pita! Instead of waiting for a few hours while my system was locked up and completely useless, your tip allowed me to delete the files in a matter of seconds.

Leave a Reply