“Terrorists are criminals who should be held accountable. We should not make any religion or nation responsible for these criminals, but fight with the perpetrators," Stoltenberg said in an interview with German daily Bild published Friday.
The NATO chief said European countries faced terrorist threats by various different radical groups and individuals.
“I am coming from Norway where a white Christian Anders Breivik perpetrated an unthinkable massacre in 2011,” he said.
Stoltenberg underlined that it was still early to comment on the identities and goals of terrorists that targeted French magazine Charlie Hebdo.
“It is important now to first arrest these perpetrators and then give them the punishments they deserve,” he said.
Asked about the threat posed by ISIL, the NATO chief underlined that Christians and Muslims should join forces to counter the terrorist threat.
“My expectations is that all of us, Christians, Muslims or whoever we are, should stand up for universal values. We can have different opinions, but we have to defend our values and also help overcome prejudices,” Stoltenberg said.
“Muslims are the biggest victims of Islamist terror. Most of the Muslims condemn the terror and we are condemning the terror. We must fight together against terrorism," he said.
The terrorist attack on Charlie Hebdo left 12 people dead, including journalists and cartoonists regarded as legendary across France. Editor Stephanie "Charb" Charbonnier, Bernard "Tignou" Verlhac, Jean Cabu, George Wolinski and renowned economist Bernard Maris were among those killed.
French police identified the two gunmen as Franco-Algerians Said Kouachi, 34, and Cherif Kouachi, 32.
On Friday, police had surrounded a small printing business in the village of Dammartin-en-Goele, 25 miles (42 kilometers) northeast of Paris where suspects were holed up.
European countries have witnessed an increase in suspicion and negative feelings towards Muslims in recent months as far-right and right-wing populist parties cash on a growing fear of Islam.
Many have been influenced by reports of atrocities committed by ISIL in the Middle East.
Muslim organizations underline that the source of the radicalization of some young immigrants is not Islam itself, but the sociological problems they face such as discrimination, unemployment, or lack of future perspectives.