Emmanuel Adebayor headed Spurs in front in the 35th minute and then tapped in their third after a Chico Flores own goal had doubled the visitors' lead.
Wilfried Bony hit the bar and had a good penalty appeal for a push by Michael Dawson turned down in the first half, before getting Swansea's consolation late on.
The win moves Tottenham up to fifth, level on points with fourth-placed Liverpool as Sherwood, who opted against sticking with a 4-4-2 formation here, remained unbeaten in the Premier League since taking over from Andre Villas-Boas.
Sherwood, who admitted Swansea should have had a penalty for Dawson's push on Bony when the game was goalless, has been delighted by his start as Spurs boss since signing an 18-month deal.
But he is in no doubt as to what is expected of him when it comes to the London club's finishing position.
"It's been good, we know there are going to be tougher tests ahead, but it's been a good start for me," he said.
"If the season were to end tomorrow I don't think the chairman (Daniel Levy) would be too happy because we are fifth.
"The final league position has to meet the expectation of the club otherwise it's 'goodbye Charlie'.
"The club needs to finish in fourth place, we want to finish in fourth place. Anything other than that will be a disappointment but realistically we should be in amongst it."
Adebayor has now scored six goals in seven appearances under his new manager, but Sherwood insisted the striker's revival is not down to him.
"I haven't said anything to him, I have just given him the stage to play on, he hadn't had that stage for a while," said Sherwood.
"I haven't made Emmanuel Adebayor a good player, I think we already know he has been a good player at every club he has been at.
"We just needed to get some consistency out of him and I am sure if he continues to enjoy playing he will keep performing well.
"He is enjoying it, if you enjoy yourself playing football you will get the best performances out of yourself.
"He is no different to anyone else. He has a smile on his face, is scoring goals and is enjoying it.
"We are trying to encourage the boys to roll the ball into him because he makes things happen. He dictates the pace of the game for us and he likes to get it wide as he knows there is a chance he can score from the cross.
"After his first goal he grew even more and started dominating the game for us. But there's been no magic wand from me. He has always been a good player."
Swansea, meanwhile, remain just three points above the relegation zone after another defeat and another injury, as Jonjo Shelvey's hamstring problem saw him join Michu, Michel Vorm, Pablo Hernandez, Jonathan de Guzman, Nathan Dyer, Jose Canas and Garry Monk on the sidelines.
Their injury woes and the alarming manner in which they faded following Adebayor's opener will be very troubling for manager Michael Laudrup, who admitted his side are in a difficult spot having failed to win in eight league games.
He said: "Everything is black and negative right now. Let's start again tomorrow or the day after and see if we can get some players back to fitness and back in the team.
"Then it is a matter of getting a win, like with a lot of teams in our situation.
"Even if it seems a dark, difficult moment for us, we can say the other teams are still there around us.
"We just need a little more luck in certain situations."
Source : PA
Source: PA