Brunson, who used to run a Protestant church in the city of Izmir in western Turkey, was recently moved to house arrest, but Washington said that was not enough.
The standoff appears to be one of the most serious crises between Turkey and the United States in modern history, along with the rows over the 1974 Turkish invasion of Cyprus and the 2003 US-led invasion of Iraq.
Pompeo discussed the detention of American pastor Andrew Brunson, who has been in custody in Turkey since 2016 on terror charges.
"Brunson needs to come home, as do all the Americans being held by the Turkish government", Pompeo told the press prior to his side meeting with the foreign minister at a Southeast Asian security conference.
Turkey's foreign ministry previously said it would retaliate against Washington's decision, calling it a "hostile stance".More news: China keen on military drills with Asean, insists on keeping US out
More news: Amid bodyguard crisis, Macron loses ground in new poll
More news: Retailers Cash in on Sales Tax Holiday Weekend
On Wednesday, the White House announced that it was imposing sanctions on Turkish Justice Minister Abdulhamit Gul and Interior Minister Suleyman Soylu.
The Erdogan government has vowed to retaliate against the USA sanctions, announced Wednesday, but has not revealed what they will be.
On the sanctions, he said: "I hope they'll see this for what it is, a demonstration that we're very serious".
Speaking to Turkish journalists after his meeting with Pompeo, Turkish Foreign Minister Mevlut Cavusoglu described their discussion as "extremely constructive" and said the two would continue to work toward resolving disputes between the two countries.
On Wednesday, Washington moved to block the assets of Turkish Justice Minister Abdulhamit Gul and Interior Minister Suleyman Soylu in the USA and prohibit American citizens from "engaging in transactions with them". The Turkish government accuses Mr Brunson of links to the coup attempt of that year, and he faces up to 35 years in prison if found guilty.
Erdogan has denied speculation that there had been an agreement to swap Brunson for Turkish citizens being held overseas, particularly 27-year-old Ebru Ozkan.