I hoped someone else would answer this. Maybe an experienced hanger can add comments.
The diagram you refer to when making this statement is simplified.
It may be worth looking at posts covering hanging angle (BTC,OTS etc) for instance HairyGuy’s recent BTC thread. You will never completely isolate particular ligaments, even ots hanging will cause stress to the suspensory ligaments. The tunica is probably the greatest resistance to growth after initial gains (if you’ve seen initial gains).
Is it because you are unhappy with the gains you are currently getting that you ask this question? If so what is your current program?