The Ministry of Defence has already contacted Richmond Council’s planning department about building 192 homes on the Kneller Hall site.

Defence minister Mark Lancaster also confirmed it would cost at least £30m to upgrade the Royal Military School of Music’s “ancient and inefficient” home.

He was quizzed at a debate called by Twickenham MP Tania Mathias during a debate heard at Parliament on Wednesday morning.

Dr Mathias said the 150-year-old mansion is the “heart and soul of Whitton” and pleaded with Mr Lancaster not to replace it with housing.

She said: “This isn’t about a building, this is about a community and it is no ordinary site because Kneller Hall effectively is Whitton and Whitton is Kneller Hall.

“It is tragic and extraordinary that in peace time the MoD has managed to create such hostility from such a peace-loving community.

“This is the community of Whitton and the MoD may have estate agents but what they need are historians and psychologists.”

She added that more than £1m has been spent on Kneller Hall in recent years and reiterated her opposition to housing on the site.

She added: “This is not what I believe, as a Government, we want when we say that we need more housing.

“Whitton is not a suburb, it is not a dormitory town, it is a unique community and that comes from Kneller Hall.”

The defence minister claimed there could be as many as 192 homes built on the site as part of the MoD’s “prosperity agenda”.

Kneller Hall is one of 12 sites up for sale which the MoD hopes will generate £500m in land receipts, providing 15,400 homes across the UK.

He said: “The facilities which are currently housed there are ancient, they are inefficient and they are not fit for purpose.

“How can it be that an organisation that contributes so much at home and overseas is expected to operate out of an old and failing site?

“It is really in best interests of defence and military capability and best use of taxpayers’ money – disposal would offer better value for money and crucially, better military capability.”

He confirmed an MoD representative would be willing to attend a public meeting about the sale.

He also said that the Metropolitan Open Land surrounding Kneller Hall would remain “untouched” and the protected trees would be preserved.

Whitton Councillor Grant Healey said: “I thought it [the debate] was excellent on Tania Mathias’s part and she made some good points about why Kneller Hall should be kept.

“As councillors, we are pursuing our campaign within the council.”

Mr Lancaster did say that the military would continue to perform in the park each summer, even if Kneller Hall was to close in 2018.

Councillor Gareth Elliott has given notice of a motion at Tuesday’s full council meeting calling on the council to place on record its opposition to the sale of Kneller Hall.