Bale, 24, has been heavily linked with a move to the Spanish capital for much of the summer, although it materialised on Monday that Spurs were also considering bids from at least one other club.
The Wales international still seems likely to leave White Hart Lane before the close on the transfer window in a week's time, but a move to the Bernabeu would not suit the player, according to Hoddle, who also accepts Spurs would miss him despite investing heavily over the summer.
"I have already expressed my doubts as to whether Gareth, undoubtedly a fantastic young player, is ready for such a move as a person," the ex-England boss told zapsportz.com.
"But are Spurs, as a club, ready to let him go? I don't think they are - I liked the look of the new boys in a team that, generally, were okay but not too much more than that. They were too good for Swansea who, frankly, disappointed me a bit."
Bale missed the 1-0 win over Swansea on Sunday as he had been given time off, with suggestions a trip to Spain over the weekend would have also seen him finalise details over a deal that was likely to break the world record transfer fee.
But Hoddle reckons Spurs chairman Daniel Levy may dig his heels in and keep the player, especially if the only other options than to sell to Madrid were to let Bale move to one of the club's Barclays Premier League rivals.
"Spurs were just missing that wow factor. Or Bale factor, if you like," Hoddle added.
"Ideally, Daniel Levy would inject the missing magic by keeping Bale. I know he that ideally he would love to keep him and really see how high Spurs could fly.
"He is loath to sell him to another English club so if there is a second bid, as was touted (on Monday morning), then I know Daniel would not welcome losing his top player to a direct rival. In fact, he doesn't really want to sell him to Real Madrid.
Source: PA
Source: PA