I have Aphantasia for all senses, aka I can't recreate any sense in my head, but from what I've heard online is that some people have some but but not all like they can recreate sight but not sound or other combinations so it's probably more like a spectrum then just one but yes I think Aphantasia is more then just visual.
I think it normally has to do with the ganzfeld effect using what little info the eye sees to create something, It could be interesting if Aphantasia removes/reduces the ganzfeld effect or similar effects of sensory deprivation. I also have Aphantasia and I don't think I see anything if I close my eyes but I have no idea what the experience is for someone without Aphantasia.