A Daily Mirror report mentioned crowd trouble at a Forest-Burnley game on 18 April 1964 (which Burnley won 3-1) culminating in some fans trying to attack referee Horner and his linesmen at the end. I'm not sure if that were his final match (the season ended the following weekend) but it wouldn't be surprising if he decided to call it a day after such an experience.
Thanks LF, I'm sure that's the incident I remember. We took the Daily Sketch not the Mirror in our house, so that would have been where I read it and I'm pretty sure the report said it was HH's last game-not that this is a guarantee of truth of course but it's quite likely.