The east London club has vowed to keep the athletics track while Spurs plan to rebuild it as a football-only stadium.
Brady told BBC Sport: "It's a corporate crime to spend ?500m on a stadium and just four weeks after the Games have finished, bring the bulldozers in."
A decision on the future of the Olympic Park site is expected this week.
An announcement was originally due on 28 January
but was postponed because the Olympic Park Legacy Company (OPLC) wanted more time to study the bids.
And the OPLC also has a third option of converting the stadium into a 25,000-seat mixed-use venue for sporting, cultural and community events.
DAVID BOND'S BLOG
Why did organisers decide back in 2006 and early 2007 to rule out a design which could have accommodated Premier League football?
An athletics legacy was one of the key promises made to the International Olympic Committee by the London bid team in 2005 when it won the right to stage the Games.
"The Olympic Stadium was built on a promise, made in the Queen's name, to have a legacy for athletics," added Brady.
"Through [West Ham's] design there will be over ?90m spent on reconfiguring the stadium so it sits perfectly for athletics and football and cricket and major events.
"Once the conversion is done, with the roof and some of the seats, there wont be a single seat within that stadium that has a worse view of the pitch than Wembley."
The Hammers, in a joint bid with Newham Council, want to create a 60,000-capacity arena that will also be used for concerts and community events.
London 2012 chairman Lord Coe
and former
Olympics minister Tessa Jowell
have both backed West Ham's bid to to take over the stadium.
And Lamine Diack, president of world athletics governing body the IAAF, and an International Olympic Committee member,
told BBC Sport last month that London 2012 organisers will have "lied" and Britain's reputation will be "dead"
if an athletics legacy is not left at the Olympic Stadium.
Tottenham, who are joint bidders with with sports and entertainment company AEG, say they would create an athletics legacy by contributing to the refurbishment of the National Sports Centre at Crystal Palace.
Spurs boss Harry Redknapp is firmly
against keeping the running track between the stands and the pitch at the Olympic Stadium.
Any OPLC recommendation will have to be ratified by the Department for Culture, Media and Sport, and the Department of Communities and Local Government, and the London Mayor's office.
The ultimate deadline for the OLPC is the end of the financial year, although currently there is no suggestion it will take that long to make its decision.
Source: BBC Sport
Source: BBC Sport