Sherwood suffered his first loss since being appointed as permanent manager thanks to goals from Santi Cazorla and Tomas Rosicky at the Emirates Stadium.
Tottenham rarely looked like making it into the hat for the fourth round of the FA Cup.
Arsenal dominated large parts of the first half and the Spurs defence were always vulnerable to the pace of Theo Walcott and the impressive Serge Gnabry.
Sherwood deployed a 4-4-2 system while Arsenal lined up 4-2-3-1, allowing Arsene Wenger's side to pack out the midfield and retain possession.
Sherwood is a firm believer in the 4-4-2 formation, though and he says he only changed it in the second half because midfielder Nacer Chadli had to come on for injured striker Roberto Soldado.
When asked whether he thought he should have played more midfielders to match Wenger's line-up, the Tottenham boss said: "No.
"They outnumbered us in the middle of the park but we outnumbered them out wide. They can't have it all ways. I think we were fine.
"We didn't lose the game because we were outnumbered in the middle of the pitch.
"A lot is made of systems - 4-4-2, 4-3-3 or whatever you want to call it... It's about passing the ball to your own team and keeping hold of it because when you lose the ball you are always going to be out of shape - otherwise you are going to be a rigid, boring team.
"It's about funnelling back in, shuffling across. I don't think we were overrun in the middle of the park. I think they did all right."
Sherwood's main goal may be to get Tottenham into the Champions League, but the Spurs boss knows this FA Cup defeat will be a blow to morale, particularly after the high that came with the New Year's Day victory at Manchester United.
Sherwood, who lost three of his five north London derbies as a player, was not too downhearted about the third round defeat, however.
The 44-year-old says his team were always going to be up against it because of a crippling injury list that forced him to hand 19-year-old Nabil Bentaleb his full debut.
"I am disappointed with the result, but we didn't disgrace ourselves," the former Blackburn midfielder said.
"We showed we belong on the same field and in the same company as Arsenal.
"It has been six games in 17 days for us and we probably have the smallest squad in the league with the amount of injuries we have had.
"We were fatigued. A little bit of sharpness was missing but it could have gone either way.
"They showed great character and I am proud of them."
Cazorla put Arsenal ahead in the first half after the hosts took advantage of some slack marking by Vlad Chiriches and Kyle Walker.
The second goal came as a result of a huge error by Danny Rose.
The left-back, who scored the winner against Arsenal on his debut four years ago, dallied on the ball when he was the last man stationed on the halfway line.
Rosicky won possession and ran 40 yards before lifting the ball over the advancing Hugo Lloris.
Sherwood backed Rose to recover from the mistake as he said: "Danny is a good lad. He doesn't shirk any responsibility.
"I will never berate anyone for making a mistake. But he has got the character, he has got big enough shoulders to move on and I am sure he will be ready to play on Saturday."
Source : PA
Source: PA