There are essentially 3 elements to the offence of using a hand-held device while driving - using, holding and driving.
If you were holding it at the material time, that element appears to have been satisfied.
To the extent that we can ascribe a coherent and logical gestalt mind to Parliament, I personally would find it hard to accept that they intended that somebody manipulating a phone whilst entirely stationary in a queue of traffic should receive a mandatory 6 point endorsement. If you had been controlling the speed and direction of a moving motor vehicle at the time, then it seems certain that that would have been Parliament's intention.
As far as we are aware, there is no case law specifically on the issue of what constitutes "driving" for the purposes of the mobile phone legislation. For drink driving, being stopped at traffic lights still constitutes driving, which is entirely consistent with the purpose of the legislation - apart from the potential for very poor judgment, a drunk driver would not be able to stop being drunk instantly when the lights changed.
Whilst the meaning of "using" has widened substantially since the High Court decided that the original iteration was very narrow, ([Without looking at the revised wording]I would also be minded to argue that connecting the phone to the car's bluetooth is not in and of itself actually "using" the phone - it is merely preparatory, rather than a being useful in and of itself. For the purpose of TV licensing, "using" a TV does not include merely having one available for use - it means actually using, whereas for insurance, using a motor vehicle on a road or other public place includes having it parked up (on the somewhat circular basis that if not, people would get away with using a vehicle without insurance).
In the alternative, if manipulating the phone to connect to the car's bluetooth is a "use" in and of itself - ironically a "use" that would solely negate the risk of using the phone for interactive communication whilst subsequently controlling the speed and direction of the vehicle after the traffic had started moving, then doing so is potentially protected by the qualified right to give and receive ideas and information under Art. 10 ECHR - which can only be negated by law that is necessary to protect the rights and safety of others. Whilst connecting to bluetooth in case you need to call your wife to tell her that you're going to be late for dinner is not the most compelling of "freedom of speech" cases, any infringement of that right by public authority would need to be justified as necessary and proportionate. Basically, "the law is the law" and other such trite tautologies won't cut it.
Your options are to accept the fixed penalty, or reject/ignore the fixed penalty and wait for them to take you to court.
If you accept the fixed penalty, or lose in court, you get the same 6 points, but a much heavier fine + costs + surcharge if you lose in court.
If you manage to win in court (either at first instance, or on appeal if you have the appetite for it), then you get no points and no fine.