On multiple fronts, conservatives are challenging Virginia Gov. Terry McAuliffe’s executive order to restore voting rights to ex-felons who have completed their prison sentences.

This week, Judicial Watch, the conservative advocacy group, filed a lawsuit in the Circuit Court of Bedford County, seeking to prevent ex-felons who gain from McAuliffe’s April 22 order from voting in the November presidential election.

The lawsuit was filed on behalf of several Virginia voters, who claim their “lawful votes” will be “diminished” by the votes of the ex-felons.

Republican leaders of Virginia’s House and Senate filed a similar lawsuit in May, this one before the Supreme Court of Virginia. Charles J. Cooper, who ran the Office of Legal Counsel under President Ronald Reagan, filed the suit on behalf of Virginia House Speaker William J. Howell, Senate Majority Leader Thomas K. Norment Jr., and four other voters in the state.

In both cases, the plaintiffs argue that McAuliffe, a Democrat, violated the state constitution and exceeded his authority by restoring voting rights to 206,000 ex-felons as a group—at one time—rather than individually.

“Our complaint is that this was done contrary to the law,” said Jim Peterson, a senior attorney with Judicial Watch, in an interview with the Daily Signal. “The Virginia Constitution says the only way convicted felons can get their full civil rights—including voting rights—restored is by making an individual application to the state, which then processes and considers whether that individual is entitled to getting his rights back. What they have done here is do away with that whole process and just say everybody can get their rights back.”

McAuliffe, and proponents of extending voting rights to those with felony records, argue that it helps former convicts adjust to society after prison and probation or parole.

Supporters say the governor’s decision particularly impacts black residents of Virginia. According to The Sentencing Project, a Washington, D.C., nonprofit that has advocated for restoring rights to ex-felons, 1 in 5 African-Americans in Virginia cannot vote because of felonies.

“What the governor has done is not terribly radical,” said Marc Mauer, executive director of The Sentencing Project, in an interview with The Daily Signal.

“This has just placed the state more in the mainstream of what other states are already doing,” Mauer said. “There is a growing consensus that we need to do more on re-entry services for people coming back into the community from prison. Clearly, these people need a job, a place to live, and a good peer network because we want them to be connected to positive institutions in the community. If people come out of prison and we say to them, ‘You have to abide by the rules and regulations of society, but, by the way, you are still a second class citizen when comes to voting rights,’ I think that sends a harmful message.”

McAuliffe’s order applies to all ex-felons who have completed their prison sentence and have been released from probation or parole, including those convicted of violent offenses like murder and rape.

It also permits ex-felons to serve on juries, run for public office, and apply to have their right to own a gun restored.

The Virginia governor says that nearly 80 percent of those affected by his order were convicted of nonviolent offenses.

Mauer, of The Sentencing Project, says Virginia is one of four states—the others are Kentucky, Florida, and Iowa—that bars anyone with a felony record from voting for life.

Virginia’s constitution gives the governor the power to grant voting rights to convicted felons if they petition to have them back.

But McAuliffe, and Republicans who oppose his new ruling, disagree on whether the text of the constitution gives the governor the authority to restore voting rights to an entire class of people at once.

Nationally, as part of the broader criminal justice reform movement, there has been a push toward restoring voting rights to felons.

The Brennan Center for Justice at New York University reports that about 20 states have eased their voting restrictions over the last two decades. Mauer says in all but 12 states, felons regain the right to vote after completing their prison sentence.

Kentucky’s Republican governor, Matt Bevin, recently overturned an executive order issued by his Democratic predecessor that was similar to McAuliffe’s effort. Bevin later signed into law a less liberal policy that allows felons to petition a judge to have their convictions removed, providing them the right to vote.

Critics of McAuliffe’s action note that Virginia’s previous Democratic governor, Tim Kaine, attempted to issue an executive order automatically restoring voting rights to ex-felons who completed their sentence, but his staff counsel advised him that he did not have that authority.

Some 5,000 ex-felons have already registered to vote under McAuliffe’s order, Mauer said.

Republicans have also criticized the implementation of the governor’s order.

According to The Washington Post, McAuliffe mistakenly restored voting rights to several violent felons who are currently in prison or on probation.