Hart was at fault for the third and final goal, which was scored easily by Chris Wood after he parried Jóhann Berg Guðmundsson's strike from the edge of the box.
'It's such a big stadium, so much area to cover, it's hard for the security to be able to stop everyone but as I said, we went 1-0 down and had fans run onto the pitch.
"That atmosphere can never come back in the last five games otherwise the club is in serious trouble".
"The club are obviously going to have to look at things with the local police and make sure that trained stewards and police are in place - you're nearly looking at it and saying, "this is out of control", Brooking said. It does make the atmosphere really tough to play in'.
"The actual level of aggression was something I couldn't believe West Ham fans would get involved with", he added.More news: Duke overwhelms Notre Dame to advance to ACC semifinals
More news: Long Island nursing student killed
More news: Bihar Bypolls: 57% voter turnout recorded in Araria Lok Sabha constituency
As West Ham scraps to stay in the Premier League, sitting three points above the relegation zone, resentment has grown over the decision to uproot from the Upton Park home of 112 years to Olympic Stadium in 2016.
It read: "West Ham United have immediately launched a full and thorough investigation into the incidents which marred the second half of today's match and are committed to taking decisive and appropriate action".
And Brooking said Sunday that Sullivan had been struck.
"A lot of the stewards who are on duty on matchdays are not West Ham people - they're actually stadium people and sometimes until they get a job there haven't had any experience dealing with football crowds". Obviously, the police were not inside the stadium at this point.
West Ham, who are tenants at the stadium, are incensed at the poor stewarding that potentially put one of their star players in peril and the Daily Express understands they will demand the site's managers E20 bring on board one of the country's most senior policing advisors so all parties can work together to prevent a repeat.
"I think with that situation the mum was ushering them away so we got them over the barrier and I think they had the time of their lives sat on the bench with us". We shouldn't run away with the idea that 50,000 people were tearing towards the directors' box.