If you are convicted, the court are required by law to give you 3-6 points, unless they decide to ban you outright, or unless they find that there are Special Reasons Not To Endorse.
An outright ban to avoid totting up would subvert Parliament's intention in the totting up legislation.
SRNTE is mitigation concerning the commission of the offence (why you did it) that is so compelling that the court ought not to endorse your licence - where you are technically guilty, but are pretty much blameless. For example if you bought insurance in good faith and the broker turned out to be a con-man, you would still technically be guilty of driving without insurance, but it would be manifestly unjust to endorse your licence if you had no realistic way of knowing. If your argument is that you didn't notice the terminal signs due to inattention (as opposed to having any evidence that they were in any way defective), and that the repeater signs should have been 150 yards closer, IMHO the only way that would fly would be if 2 of the magistrates recognised you from the lodge, or if the prosecutor had spent the entire day so far going out of his way to annoy them.
If there is evidence of cones/roadworks/whatever that would give rise to the expectation of a reduced speed limit in the mind of a careful and competent driver, then even less so (if that is possible)
Any mitigation that falls short of SRNTE won't stop you totting up. Particularly if you try to argue that the TSM is somehow "guidance to the law" (whatever that means).