I don't know what I changed but I've been trying to change my settings to get less noise and a faster render, but now it wont do the amount of samples I want it to. 1 sample takes as long as 1000 samples and has 0 difference in noise.
Adaptive Sampling min samples is not the actual amount of samples that cycles is using to render the scene.
From the manual:
The minimum number of samples a pixel receives before adaptive sampling is applied. When set to 0 (default), it is automatically set to the square root of the total (max) sample count.
So, what is happening in your case, is that your scene is set to render at 128 samples (the default until changed), so changing the adaptive samples to a higher setting doesn't change anything. If you raise the sampling to something more like 500, you will see a difference.