The sign does not have text saying Goods Vehicle Loading Only, although it shows a lorry symbol. The only way to decide one way or the other is to look at the TRO that imposes the restriction.
TSRGD 2016 (the latest I could find)
https://www.legislation.gov.uk/uksi/2016/362/schedule/8/made
suggests this sign is meant to be to Diagram 618.3B
However, if the City want to limit loading to Goods Vehicles, they must include the word by before the pictogram
(https://www.legislation.gov.uk/uksi/2002/3113/images/uksi_20023113_en_036)
and 2002
(https://www.legislation.gov.uk/uksi/2016/362/images/uksi_20160362_en_202)
Otherwise, it could be interpreted to read
Except for
loading
Goods Vehicles
between
5 pm - 11.00am
as if therefore there is a general exception for loading, but Goods Vehicles can only avail themselves of the exception bewteen 5pm and 11am