I wouuld have thought that I would get a warning as a first offence rather thant the £120 fine
Whilst government ministers encourage warnings to be issued when a scheme like this is first implemented, the Birmingham CAZ has now been in operation since 2021, three years ago so any period of issuing warnings has long ended. Would you expect only a warning if you didn't pay to park in a payment car park ?
Here is their website, and you can see the signs you would have passed on your way in. Frankly, I can't see any credible appeal argument, other than being a stranger to Birmingham. Trouble is, if you submit reps, Birmingham usually don't re-offer the discount, so you'd be risking another £60.