The answer courtesy of fmb501 at blenderartists (thread here) :
This appears to be a bug in the Blender installer, but the fix is
fairly simple.
Open the Windows icon in the bottom left corner of your computer
display and search for "command prompt"
Right click on the result and select "run as administrator."
There will be a small window with a black background and a blinking
cursor.
Type "chdir c:\program files\blender foundation\blender" and press
Enter (chdir is the command for changing directories with the command
prompt)
You should now be at that directory.
Type "blender -R" and press enter. You should receive a message
indicating that the registration has been successful.
Close any Windows Explorer windows and then open Windows Explorer
again.
Navigate to a directory that has your blend files and you should now
see thumbnails for all of your blend files. If there some blend files
that don't have thumbnails, it's probably an old file before the
thumbnails were added. Just open and save it again and you should see
the thumbnail.
I'm not sure why Blender Foundation hasn't fixed this instalation
problem by now. I'm wondering if you have to install blender as an
administrator to get it to regiser the thumbnail dll.
Anyway, until it gets fixed, you will likely have to register as
described above with every new installation.