Residents of an Indiana town have been threatened with legal action by an atheist group if they don’t remove their nativity scene.

Brookville, Ind. has displayed the nativity outside the courthouse every holiday season for the last 50 years.

But according to WLWT, residents of the Indiana town say that for the last three years, the Wisconsin-based Freedom from Religion Foundation has demanded they remove the scene, threatening them with a lawsuit if they fail to comply.

According to the group, the nativity is displayed next to the Franklin County Courthouse and thus violates the separation of church and state.

“If people don’t like the look of it I think they can look the other way, or don’t look at all,” Brookville resident Wayne Monroe told WLWT. “It’s been a tradition here for many, many years and I hope it’s for many more years. I think we deserve the right to put up what the community wants and I don’t think anybody else should tell us what to do.”

“I think it’s going to take a lot more that some outfit from Wisconsin to have it removed,” Monroe added.

“I don’t understand why people want to take away our love of Christ from us,” Ron Anderson, another Brookville resident told WLWT. “It’s His birthday, we need to celebrate that and remember that. When you start taking [rights] away we lose track of who we are as American citizens.”

According to WLWT, the commissioners said that they “have been ignoring the letters.”

Franklin County commissioner Scott McDonough, who lives in Brookville, was recently interviewed on Fox and Friends and said that the atheist group was “misreading” the law.

McDonough urged other towns to “stop letting them bully you:”

“They go around threatening to sue municipalities, other government agencies if they permit any type of religious display They use the misconceptions about the separation of church and state to make everyone afraid of getting into the middle of a lawsuit, when the Constitution does protect you.”