I'd like to use Squeak Smalltalk and take advantage of the dynamic IDE (which could be modified to handle Blender explicitly).
Other languages support (than Python) is considered as antifeature for now:
Supporting multiple languages is not something we're currently interested in.
Embedding a language runtime has a significant maintenance overhead, so there would need to be very compelling reasons (besides developer preference) to add support for other languages or to move away from Python.
Blender has Python API. Read about it in the documentation. You can interact with Blender through it. It's not uncommon for Blender add-ons to use some other software that Blender interacts with through the API. In some cases, like for example V-ray or Octane render engines need to change Blender itself to work so they modify Blender's source code, that you can also do since Blender is open source and you get custom Blender versions with these render engines. You can find out more about developing Blender here
There is not such an API. The API that allows Blender to include a Python interpreter relies on Python being implemented so that it is easy to include but also requires an extensive Python specific set of APIs provided by the C++ code.
To use Squeak, which was not implemented to support being an imbedded interpreter, you would have to modify the Blender C++ code to provide a set of APIs allowing Squeak to interact with Blender's internals and you would have to modify Squeak, as you suggest
It is doable, but it would require extensive work.