Just submit some reps stating that you saw the double-yellow lines and assumed they only applied up to the low wall alongside the carriageway.
As you will no doubt have already gathered, opinion on here is divided or scattered. I have to say I don't think this is a case that is winnable at adjudication. However, don't cough-up without first submitting some reps. One never knows, they may shoot themselves in the foot in their reply !