Its likely that the watch is stopping and starting, but I can`t explain why it runs OK when not worn.
Edit: possibly the battery isn’t mounted tightly and could be moving slightly during wear, thus causing intermittent stopping. That’s the best straw I can grasp. If that isn’t the answer it looks like it needs a trip to Rolex.