I was in the usual 7am-7pm loading exception for nearly everywhere else,
It's very easy to get caught out in
London, especially if you don't read the sign.
I suggest you submit a non-confrontational representation based on your narrative here; you might get a result, but TfL, like the London councils are very unforgiving as refusing reps means most people then cough-up. From looking at the location on GSV, and reading your narrative, it's difficult to see any winning possibilities based on the statutory grounds for appeals, but submit your reps anyway.
Others may suggest a different approach.