Around 11 p.m. on Wednesday night, armed police stormed a flat above a shop on Birmingham's Hagley Road, arresting three men. It was one of six raids across London and Birmingham in the aftermath of the shocking attack in Westminster earlier in the day.

Birmingham, around 125 miles (200 kilometers) from London, is often described as England's second city. It has often hit the headlines for the wrong reasons; in September 2015, a commentator on America's Fox News caused widespread outrage when he declared that Birmingham was a "no-go zone" for non-Muslims. That statement is patently ridiculous, in a city of 1.1 million people and a wider metropolitan area of over 3 million.

Yet in recent years, the city has often been in the position of having to defend itself against outside criticism.

"Every time there's any terrorist incident, after the initial shock my first thought is 'please don't be a Muslim, please don't be a Pakistani, please don't be from Birmingham,'" Farhan, a resident of the Small Heath neighborhood, told DW. "We're all worried about the backlash."

Last night's raids have echoes of 2005, when two weeks after the 7/7 bombings, one of the men who had attempted to bomb the London underground was found hiding in a house in Small Heath. In 2014, a national scandal ensued after an allegation that groups of Muslim governors were conspiring to "take over" state schools and run them along religious lines.

Read: UK schools caught up in extremism debate

Deep divisions

Birmingham is a diverse city, with a wide range of ethnic groups and religions, but it is also one deeply divided and segregated along these lines. Traveling the short distance from the city center, recently redeveloped with modern civic buildings and elaborate shopping centers, down the Alum Rock road lays these divisions bare. High street chains give way to Arabic bookshops, Pakistani greengrocers, and halal butchers. These innercity residential areas are vibrant and full of activity, but also clearly rundown, bearing all the signs of economic deprivation.

In the 2011 census, 21.8 percent of the city's population defined themselves as Muslim, a proportion that has risen with every census and is considerably highter than the national average of 4.8 percent. Around 13.5 percent of Birmingham's population is Pakistani, many hailing from the same regions in Azad Kashmir and northern Punjab. While other ethnic minority groups have moved to different areas, this community has remained centered in deprived districts around the inner city, such as Sparkbrook and Small Heath, where it has been joined by people from Somalia, Kosovo, and the Middle East.

The first wave of Pakistani immigrants settled in Birmingham because it was once a manufacturing center. Like many British cities, Birmingham was blighted by industrial decline. In recent years, it has undergone an economic recovery, with an expanding service and commercial sector. But the Muslim community has not had an equal share in this: One in five of Birmingham's residents may be Muslim, but they are three times more likely to be unemployed.

Fertile ground

The combination of economic deprivation and religious conservatism are seen as fertile ground for extremism to grow, and various extremist groups - including the now jailed Anjem Choudary's Al-Muhajiroun - try to recruit here. Parents in Birmingham's Muslim areas speak of their anxiety about keeping their children safe from the dual threats of gang culture and extremism. Yet given the fact that most of Birmingham's Muslims are peaceful and law-abiding, there is also understandable frustration at being painted as a suspect community.

"There is conservatism in religion, and there is extremism, and the two are very different," says Shabana Mahmood, MP for Birmingham Ladywood. "Legislators are looking for an easy formula - if you look a certain way and you do these things, then you are at risk of being an extremist, and we've got to do something about you. It's frightening. You are commenting on a person's individual relationship with God. That's totally unconnected to a political and supremacist ideology."

In 2010, there was uproar after police used 3 million pounds (3.48 million euros) of government counterterrorism money to install 200 spy cameras in Sparkbrook and Washwood Heath, in an initiative known as "Project Champion." Critics argued that this tactic smeared the entire community as potential extremists. The cameras were dismantled in 2011 after sustained protests.

"Nationally, Muslims are characterized as a suspect community," said Dr. Imran Awan, a criminologist at Birmingham City University. "So anything to do with counterterrorism and security makes people in these areas very nervous."

The facts of what happened on Wednesday are still emerging; we know that the attacker was British-born and known to MI5 and that the car used in the attack was rented in the Birmingham area.

Along with the rest of the UK, the city waits to hear more.