Boris Johnson refused to bow to pressure from the Prime Minister and the Conservative Party leadership to apologise for his "offensive" comments about burkas.
In his Daily Telegraph article Mr Johnson said that he felt "fully entitled" to expect women who wear face coverings to take them off when talking to him at his MP surgery.
Middle East Minister Alistair Burt criticised Mr Johnson for comments he said, "many people would find offensive".
Conservative Party chairman Brandon Lewis said he had asked Johnson to apologize, though didn't say if there would be any sanctions if the request was ignored.
She said: "I've written to @PennyMordaunt and @BrandonLewis to ask what action will be taken against Boris Johnson for his ugly and naked Islamophobia".
The Muslim Council of Britain said Mr Johnson's "intentional usage of the words" contained in his Daily Telegraph column appeared to be an attempt to "pander to the far right".
Describing the Government's response, he added: "It's not acceptable, the fact of the matter is the relationship with Muslim communities is extremely poor with this Government and at a time when many members of the Muslim community are feeling targeted and are being targeted, the fact is the community wants reassurance centrally from 10 Downing Street".
The former foreign secretary used his column to comment on Denmark's introduction of a burka ban.More news: Ontario brewers that offer buck-a-beer may get LCBO incentives
More news: Mendocino Wildfire Becomes California's Largest, and It's Still Growing
More news: Del Potro, Fognini advances to Mifel Open final
Mr Johnson has long campaigned for equal treatment for women in developing countries, especially the provision of education for girls.
Lord Sheikh, the founder of the Conservative Muslim Forum, set up to encourage British Muslims to get involved in political life, said Johnson's comments were "totally out of order".
Conservative peer Sayeeda Warsi, a former party chairwoman, accused Johnson of adopting the "dog-whistle" tactics of right-wing firebrand Steve Bannon, US President Donald Trump's former top aide.
"It is Islamophobia and I think it's pretty outrageous".
A CST spokesperson added that "it is very obvious why these comments would cause deep offence".
"When Muslim women are being attacked in the street, his comments are unsafe and can not be laughed off", she said.
Johnson received support from some quarters amid the furor over his comments, with Tory MP Nadine Dorries saying he "did not go far enough".
Reacting to news the first person has been fined for wearing a full-face veil in public in Denmark, Mr Johnson warned against a similar "heavy-handed" movement in the UK.