Blender needs to be registered in your system to generate thumbnails. This is done automatically when installing with the executable installer file. But any other kind of installation (portable, zips, Steam, ...) need to be registered manually.
Manual registration is done using a command line.
Open the command prompt where your Blender.exe is, type blender -R
and press enter. It should register that Blender application in your system and trigger thumbnailing of .blend files in the file explorer.
See Command Line - Blender 4.1 Manual for more about Blender's CLI.
If you use Blender Launcher, just RMB the build you want to register, and hit Register Extension
.
You might also need to enable thumbnails in Windows if it doesn't display thumbnails for anything.
Search Adjust the appearance and performance of Windows
in the start menu, and from there make sure Show thumbnails instead of icons
is enabled.