Rick Smith’s 14-year-old, family-run gourmet kitchen utensil business has a lot to lose if the Internet sales tax becomes law. And he’s not alone.

While a proposal to tax Internet sales is seemingly stalled in Congress at the moment, a number of online retailers remain fearful of the ill-conceived Marketplace Fairness Act.

Small businesses across the country would be negatively affected by the Internet sales tax, which would require them to collect sales tax from out-of-state online customers. Complying with close to 10,000 tax codes—and being at risk for audits from 46 states, the District of Columbia, and other U.S. territories— would put a heavy burden on these entrepreneurs.

“Even under the best of circumstances, audits are stressful, invasive, time consuming and costly,” Smith wrote in Wednesday’s Wall Street Journal. “[T]he hard expense of extra accounting assistance and, more important, the demands on my time are a real challenge and take away from my ability to grow my business.”

A few advocates of the bill argue the Internet sales tax would encourage economic growth, but this analysis is based merely on wishful thinking rather than facts. Smith worries that the big-name business supporters would overshadow the large number of businesses that would be severely hurt, even with a small-business exemption.

“The small-business exemption excluding companies with remote sales of less than $1 million sounds generous,” Smith said, but it leaves many small enterprises subject to burdens. “The definition of ‘small’ also will be a disincentive against growth.”

Smith writes that a business’s physical location should continue to be the basis of a state’s authority to impose taxes. He supports the current rule that no state should be able to force retailers to collect sales taxes from buyers unless the retailer has a physical presence in the taxing state.

“Any attempt by other states to pass their tax-collection burden on to me is a grave threat to my business,” Smith said. “The MFA is flawed legislation that doesn’t bring true simplification to the process.”