There are many things that could potentially constitute careless driving. Currently they constitute idle speculation.
What is perhaps notable is that the s. 172 notice was apparently not incorporated into a NIP. My understanding is that this is far more prevalent when a NIP is seemingly required due to an RTC, but beyond that, and taking the OP's claim that it was not a NIP at face value, I would only suggest that it was unlikely that an earlier NIP was sent anywhere. I do not recall ever hearing of a NIP-less s. 172 for an Operation Snap or similar on the basis that the issuer felt that no NIP was required due to the reasonable practicability exception, or even for mobile phone offences.
As any fule kno, the RTC exception only applies if the driver was aware of the RTC.
In general, if a defence relies on the accused's credibility it is helpful to set out his stall early. As the OP suggests, he should include a covering letter with his s. 172 response, asking for clarification of what exactly he is being accused of.