Rio de Janeiro, Brazil(CNN) When the wrecking crews rolled into Vila Autodromo this week, Luiz Claudio Silva knew it was time to leave home.
The western Rio neighborhood where he had lived for the past 25 years was being demolished to create an access path to the city's main Olympic venues.
Twenty-five years of memories were gone in a matter of minutes as the two-story house he shared with his family was razed to the ground.
"I built that house brick-by-brick and now it's gone," Silva told CNN. "They came without any warning and just destroyed it all."
For the past three years, Silva and his wife Maria da Penha have led a resistance movement fighting to preserve the quiet favela located by the terrain that was designated by the city for the 2016 Games.
Most of the 800 families who once lived there were offered a buyout or alternative housing in new condominiums by the mayor's office, but many such as Silva and Penha did not want to leave their community.
Amateur video of a confrontation between police and residents surfaced last June, when police kicked one woman and left an elderly man with a bloody head injury.
According to Rio's mayor Eduardo Paes, the original project required less than a third of the families to be evacuated. But, during a press conference this week, he said the majority of Vila Autodromo's residents wanted to go.
"Most of the people there were living in precarious conditions," Paes said. "What we were offering were better and more attractive apartments than what they had."
Many of the families relocated to federal and municipal housing projects equipped with modern apartments, swimming pool and other amenities, though according to local media reports some residents have complained of shoddy workmanship in the housing.
This week, Paes showed journalists slides of the projected urbanization plan for the area, where 30 houses and two schools would be built before the Olympics in August.
While residents said they had never seen the plan, Paes stated the remaining families would be offered housing in the new neighborhood.
Silva and Penha remain skeptical and believe their house was destroyed to make room for upscale condominiums and the real estate speculation surrounding August's Rio Games.
For now, they will remain in Vila Autodromo's church -- one of the neighborhood's last standing buildings -- where friends and neighbors have helped moved their belongings.
"We lost the battle but we didn't lose the war," Silva said, as he wheeled his dusty green couch to the church entrance. "We know what our rights are. We have the law on our side and we will stay here up until the end."
Are you attending the 2016 Olympics? Tell us on CNN Sport's Facebook page