Tottenham fans turned out in their thousands to celebrate the club's 125th anniversary and saw former greats such as Dave Mackay and Glenn Hoddle paraded before the game.
But, despite the presence of a number of Spurs legends, boss Martin Jol must have been fearing for his position at the club after Paul Robinson produced a couple of howlers, which led to Villa storming into a 4-1 lead.
However, the Tottenham players showed they are fully behind their manager and were determined to make the party one to remember and strikes from Pascal Chimbonda, Robbie Keane and Kaboul in the second half salvaged a crucial point for the home side.
Tottenham opened the scoring on 20 minutes. Chimbonda saw his close-range header well saved by Villa keeper Scott Carson. From the resulting corner, Dimitar Berbatov found himself unmarked in the box and made no mistake directing his header into the corner of the net.
Tottenham's lead was short-lived and Robinson produced his first mistake on 23 minutes which led to Villa's equaliser.
The England keeper somehow managed to drop a routine corner from Gareth Barry and Martin Laursen had the easy task of knocking the ball into an empty net.
Things went from bad to worse for Robinson when Laursen added his second of the game on 33 minutes.
Barry's cross picked out Zat Knight, whose header back across goal took a wicked deflection off the Danish defender and went through the legs of the Tottenham keeper.
Jol must have been fearing the inevitable after Gabriel Agbonlahor added a third for Villa on 40 minutes. The England Under-21s forward turned Michael Dawson on the edge of the box before firing a low shot into the corner of the net.
Villa had strong claims for a penalty turned down when Robinson appeared to bring down Agbonlahor on 47 minutes.
And just when the Villa fans thought it could not get any better for their side, Craig Gardner made it 4-1 on 59 minutes, when his free-kick from the edge of box flew past Robinson into the bottom corner of the net.
Chimbonda pulled a goal back for the home side on 69 minutes after Jermain Defoe's shot came back off the post and the defender slotted the rebound home from close-range.
Tottenham made it the tightest of finishes when Keane fired home from the penalty spot on 81 minutes after Darren Bent was brought down inside the box by Marlon Harewood.
And the unthinkable happened in the closing seconds when Kaboul fired home an unstoppable shot into the top left-hand corner of the net.