I think that happened really after Dot was revealed as the Caddy, that was actually the Fawlty Towers time to end the series on a real high, but as each series goes on they become more and more unrealistic and twisty and then the ending/payoff gets built up and up only to then naturally fail to deliver as it was not in the writer's mind when they conceived the series, but a later add on that needed to be thought up due to the show's continuing success.
That is why film sequels are nearly always far worse than originals, and for Game of Thrones the showrunners only planned to make up to the end of series 3 and the Red Wedding, and each season after that naturally deteriorated, esp the last 2 when they ran out of Martin's source material.