Sound real-time spectral analysis frequency peaks

Question

Is there a way to analyze a sound file in blender (preferably in real-time) to identify the peak frequencies so that I can map certain frequencies to certain object attributes (e.g. material color)?

I know that the animation nodes addon provide nodes that analyzes sound and can bake frequencies to frequency ranges (EQ) which can be used to create "spectrum analyzers", and even blender itself provides a way to bake sounds to f-curves, but I am rather interested in being able to do some sort of Fourier transform analysis of a sound and extract the first N higher amplitude frequencies.

For my project I have a sound file that has a certain, discreet set of frequencies (f1, f2.. fn), and some of them sweep from one frequency f1 to frequency f1' over time. I want to be able to identify the EXACT frequencies f1, f2,...fn in the sound file at any given time.

These frequencies are to be mapped to specific colors in my scene based on a pre-selected frequency-to-color map.

Note: I know the Animation Nodes audio analyzer node allows to bake frequencies into EQ bins of arbitrary sizes, though in order to identify precise frequencies in the sound at any given frame these bins would have to be made extremely small in order to detect variations in the frequencies as they sweep through the spectrum. However, the smaller the bins, the larger the baking time. Because of this I am not sure if the NA are the proper tool for the task in question.

Answer 1

If you are familiar with python, i suggest you have a look to the scipy.fft module.

To use the scipy module inside blender you need to install python3.x on your computer and then pip3 install scipy. Then in a terminal, lunch python and type import scipy; scipy. If everything went well, you have the path to the scipy module.

Then in blender, type:

import bpy
import sys
sys.path.append(PATH_TOSCIPY')
import scipy
from scipy.fftpack

Then you'll need to compute the fft and do you stuff (see more details for this here https://stackoverflow.com/questions/23377665/python-scipy-fft-wav-files)

Cheers