I had a similar problem before with erections (not PE related though) when often the shaft would fill up, but the glans would stay pretty much as is. What sorted it was cutting down on the cigarettes.
You see the ‘prune-knob’ look quite often in porn too, guess that’s how you spot the smokers.