..and
The contravention alleged is: 'Parked in a restricted street during prescribed hours.'
IMO, their reply has added another dimension to the above as I'm assuming that you (OP) provided a copy of the BB with your representations from which they deduced that the badge did not belong to you.
IMO, your first task, should you decide to appeal, would be to convince the adjudicator that the badge was being used properly. Nothing in this thread touches on this but I think you need to address the implication.*
*- which is that if you were not using the BB validly which would mean that, clock or not, the contravention would be established.