Conservatives who belong to the House Freedom Caucus are trying to corner any Republicans thinking twice about impeaching the head of the Internal Revenue Service. They’ve framed the debate as a strict binary, telling fellow members of the GOP that they either can be with conservatives or with the IRS.

Members of the Freedom Caucus took their case directly to the House floor, setting the stage for a Thursday impeachment vote. Lawmakers ultimately brokered a deal late Wednesday with House Judiciary Chairman Bob Goodlatte that postponed Thursday’s vote but guaranteed impeachment proceedings would take place Sept. 21.

Reps. John Fleming, R-La., and Tim Huelskamp, R-Kan., introduced their “privileged resolution” Tuesday to impeach IRS Commissioner John Koskinen.

That move would’ve led to a roll call vote Thursday, the closest conservatives have come in their effort to remove the top taxman. They accuse Koskinen of obstructing a congressional investigation into IRS targeting of conservative groups.

But late Wednesday, on the eve of the vote, the Freedom Caucus announced its deal with the Judiciary Committee. As part of the agreement, Goodlatte, R-Va., will convene impeachment proceedings for Koskinen next week.

“This hearing will give every American the opportunity to hear John Koskinen answer under oath why he misled Congress, allowed evidence pertinent to an investigation to be destroyed, and defied congressional subpoenas and preservation orders,” the Freedom Caucus said in a statement. “It will also remove any lingering excuses for those who have been hesitant to proceed with this course of action.”

Prior to Wednesday’s announcement, House Democrats indicated that they would vote in unison in support of Koskinen and against impeachment. Minority Whip Steny Hoyer, D-Md., told reporters Tuesday that “every Democrat is going to vote against this.”

And if liberals poached enough conservatives, they could have spoiled the effort. Lawmakers wouldn’t need to vote against impeachment: They either could vote to table the resolution or refer it to committee.

Centrist Republicans—among them Tuesday Group Chairman Charlie Dent, R-Pa.—had pushed for an alternative to a floor vote.

“There has got to be some level of due process afforded here,” Dent told The Daily Signal on Tuesday. “If there’s going to be an impeachment vote, it should go through a regular order process and you shouldn’t try to sneak something this important through.”

Conservatives balk at that characterization. They argue that ongoing reluctance to impeach from House Speaker Paul Ryan, R-Wis., and Judiciary Chairman Goodlatte soured the process.

Rep. Mick Mulvaney, R-S.C., described the Freedom Caucus strategy as an emergency valve.

“This privileged resolution is regular order,” Mulvaney, a founder of the Freedom Caucus, told The Daily Signal. “It’s regular order when the other parts of the process break down.”

The conservative push to impeach is nothing new. They’ve been calling for the IRS commissioner’s retirement since October, arguing that he obstructed the congressional investigation into the agency’s treatment of conservative groups seeking tax-exempt status.

The White House has decried that targeting but remained unwavering in its defense of Koskinen, who was brought in to reform the agency.

Koskinen, who has hired a personal defense lawyer, has described allegations of wrongdoing as “unwarranted” and the articles of impeachment lodged against him as “without merit.”

To the chagrin of conservatives, the tax chief met with lawmakers on Capitol Hill last Wednesday to make his case for why he should keep his job.

Republicans will huddle in a closed-door conference meeting Thursday morning to discuss their party’s official position.

Though there hasn’t been an official GOP vote count, the Freedom Caucus backs the privileged resolution to impeach, as do Republican Study Committee Chairman Bill Flores, R-Texas, and Oversight Chairman Jason Chaffetz, R-Utah.

The Freedom Caucus received a boost from conservative columnist George Will over the weekend. Will lent their cause intellectual firepower, writing that “Congress should fulfill its constitutional duty to police executive branch lawlessness.”

“What we have in the houses of Congress are agents of our own obsolescence,” Rep. Raul Labrador, R-Idaho, said Tuesday, referring to Will’s article. “And it’s because of the leadership of the Republican Party.”

This story was updated to reflect late-breaking developments Wednesday night.