Not worth the hassle of owning physical with all the security risk.
I've been buying and selling gold for about 15 years have units in GBS (ETF) which you can hold within an ISA, but now predominantly use Bullionvault. I've also used Goldmoney (similar to Bullionvault)
You can purchase the physical, store it in London, Geneva, New York - wherever and buy/sell it with ease
https://www.bullionvault.com
Yes, there are charges and you may be able to pick up the physical cheaper in the form of bars, ingots or coins, but do you really want the extra hassle and risk when you eventually want to move it on?
Another option is a Gold Mining ETF - GDX or GDXJ (Junior Miners)
If your timing is right, that's really the way to play a rising gold price, but can be much more volatile and risky.