A coalition of 23 leaders in the conservative movement today urged Speaker John Boehner not to punish Republicans who voted against him for the House’s top job.

Boehner, who removed two of those dissenters from the powerful Rules Committee after the speaker vote, is now rethinking the decision, according to Politico. A total of 25 Republicans did not vote for Boehner.

>>>> The 25 Republicans Who Did Not Vote for John Boehner

“It is unacceptable and disappointing to see that some conservatives are already being punished by you and your leadership team for disagreements over policy and the direction of the conference. This must end immediately,” the 23 conservatives wrote in a letter from the Conservative Action Project.

Reps. Daniel Webster and Richard Nugent, two Florida Republicans who opposed Boehner, lost their seats on the Rules Committee. Rep. Randy Weber, R-Texas, said he was being punished for voting for another candidate.

“I’ve already lost the authorship of one bill,” Weber said. “Look it shouldn’t be that way.”

Rep. Louie Gohmert, R-Texas, who received three votes for speaker, called Boehner a “sore winner” for seeking retribution.

Some of Boehner’s allies want the speaker to take a tough stand after surviving the most divisive speaker election since 1860. That could spell trouble for Reps. Scott Garrett, R-N.J., and Mark Meadows, R-N.C., two subcommittee chairmen.

“We need to get to the bottom of the guys who voted against [procedural motions], and we need to understand why they voted against that, and then we need to know why people voted against the speaker yesterday,” Rep. Devin Nunes, R-Calif., told Politico.

Nunes is reportedly drafting a resolution prohibiting members who vote against the speaker from chairing a subcommittee.

Boehner, however, signaled the punishment might only be temporary.

“We’re going to have a family conversation,” Boehner said at a Wednesday press conference. “Which we had this morning about bringing our team together. And I expect that those conversations over the next couple of days will continue.”

Last week, Boehner’s spokesman told The Daily Caller, “Boehner has said publicly that there will be no retribution for ‘no’ votes.”

Conservatives want Boehner to keep that promise.

“It is incumbent upon you, as the leader of Republicans in the House, to provide unifying and inclusive leadership,” they wrote.

The letter was signed by the following 23 leaders:

  1. Edwin Meese III, former attorney general for President Ronald Reagan
  2. David McIntosh, former member of Congress from Indiana
  3. Becky Norton Dunlop, former White House adviser to President Ronald Reagan
  4. L. Brent Bozell III, chairman of ForAmerica
  5. Tony Perkins, president of the Family Research Council
  6. Ken Cuccinelli, president of the Senate Conservatives Fund
  7. Marjorie Dannenfelser, president of Susan B. Anthony List
  8. Colin A. Hanna, president of Let Freedom Ring
  9. Matt Kibbe, president of FreedomWorks
  10. David N. Bossie, president of Citizens United
  11. Jenny Beth Martin, co-founder of Tea Party Patriots
  12. Tim Wildmon, president of American Family Association
  13. William L. Walton, vice president, Council for National Policy
  14. William G. “Jerry” Boykin, retired U.S. Army general and executive VP, Family Research Council
  15. J. Kenneth Blackwell, chairman, Constitutional Congress
  16. T. Kenneth Cribb Jr., former domestic adviser, President Ronald Reagan
  17. Alfred S. Regnery, chairman, Law Enforcement Legal Defense Fund
  18. Richard A. Viguerie, chairman, ConservativeHQ.com
  19. Andrew C. McCarthy, former federal prosecutor
  20. Frank J. Gaffney Jr., president and CEO, Center for Security Policy
  21. Allen Roth, president, Secure America Now
  22. Paul Caprio, director, Family PAC Federal
  23. Richard Manning, Americans for Limited Government

Two of the signers, Meese and Dunlop, are affiliated with The Heritage Foundation.