Protesters – who raised the black flags of ISIL – marched across the streets of Gaza City before they stopped outside the French Cultural Center, amid chants in support of ISIL leader Abu Bakr al-Baghdadi.
Last week, French satirical magazine Charlie Hebdo published cartoons mocking prophet Muhammad as part of a new edition that was printed following an attack on its offices in French capital Paris, which left 12 people dead.
Demonstrators also burned down the French flag and raised photos of the alleged magazine attackers during the rally.
Gaza's police, which is run by Palestinian faction Hamas, tried to prevent the demonstrators from reaching the French Cultural Center, but to no avail, according to an Anadolu Agency correspondent.
"We are out to express our anger at the cartoons mocking the prophet," Abu Zain al-Abedine, one of the demonstrators, told AA.
"We are out to let everybody know that prophet Muhammad has his own army in Palestine," he added.
The French cartoons have caused massive uproar across the Arab and Islamic world as protests broke out in several countries against the lampooning drawings.