As bad as the political persecution of conservatives by the IRS is—and it is really bad—if the IRS were to replace half of its workforce with tea party members, problems would remain. Let me explain how I know this.

President Obama’s announcement that Acting Commissioner Steven T. Miller would take the fall for the IRS campaign against conservatives brought to mind decade-old memories of my experience with Miller.

In 2002 and 2003, I was a member of the 18-person Advisory Committee on Tax Exempt and Government Entities (They call it the ACT. I guess ACTEGE was too unwieldy or perhaps French-sounding.) We worked with the Exempt Organization Division of the IRS, of which Miller was then head. My first memory was hearing Miller and his colleagues regularly call taxpayers “customers.” I asked how I could shop at a different IRS if I didn’t like the customer service there.

Over the course of each year, we endured long briefings by various IRS officials. Most were mind-numbing, but two stood out. The head of technology for the entire IRS told me in a hallway conversation that he had read about the Internet but had never been online. This was in 2002 when, according to, there were 631 million Internet users. But not the top tech guy at the IRS.

Most relevant to the recent scandal was a presentation on applications for tax-exempt status. We learned all sorts of things about processes, procedures, the growing number of applications, and “limited resources.”

Not a word about politics, but it was a very revealing disclosure about how the IRS actually works. A presenter told us that one reason for the delays in approvals for tax-exempt applications was that the National Treasury Employees Union had negotiated a contract that allowed for both flex time—that is, workers could work nontraditional hours—and flex place, meaning they could work from home. But employees could not remove confidential application forms from their offices when working at home.

The next morning, Steven Miller corralled me over coffee before the session began and earnestly told me that the presenter had been wrong, and that the flex place guidelines had no impact on the timing of approvals. I said that I was sorry I couldn’t follow up on this, because it would make a fine op-ed piece. He did not appear amused.

Even after the current scandal is cleaned up, the IRS’s built-in bureaucratic problems will remain. The tax code is too long and too complicated. The IRS has too much power and will be granted more when Obamacare kicks in. And the union will continue to bargain on behalf of the employees, no matter how their service affects the “customers.”

John Von Kannon is vice president and senior counselor at The Heritage Foundation.