Joel and Ellie don't just have to survivor infected, but those living outside the quarantine zone will do anything to survive, even kill other humans.

After the fall of the government and military, many survivors took the law into their own hands and took over whole cities, some forming gangs that scavange together for greater protection, others going it alone. Most survivors in the game are hostile towards others.
