Notice to Keeper (NtK) is not PoFA compliant [9(2)(a) and 9(2)(e)(i)] so they cannot recover anything from the Keeper, even though they say they are relying on PoFA 9(2)(f) because they have to comply with ALL the requirements of the Act and they clearly haven't.
Additionally, you have the permit that was paid for.
Others will give you the advice on how to appeal and what the likely outcome will be. In short, this will likely go all the way to a court claim where you you will easily win or, more likely, they will eventually discontinue once they realise they are not dealing with low0hanging fruit on the gullible tree.