A blue badge supposedly allows parking without having to pay.
Possibly, but this is dependant on the local authority in question. It's also not relevant to the alleged contravention here.
The current GSV available doesn't seem to even show the sign that was referenced, though it does show the Loading Only paint on the ground, whether that is there now, would be up to the driver to find out.
But if the driver saw the Pay & Display sign, as you say, it's unknown to me how they would have missed the loading restriction directly above it. So, IMO the fact it was angled away from where you'd normally expect to see it, doesn't seem to be convincing at all.
There are other avenues available, and I easily could have missed a million things here, so I'll sign out and let the others continue.
Good luck