Security forces exchanged fire with the militants for 20 minutes early on Sunday in Amboni caves in Tanga, before the latter ran away.
A policeman was killed and five others wounded in the exchange.
"We reached a decision to call backup from Tanzania Peoples' Defense Forces (army) and other security forces after realizing that we were dealing with people who are more than bandits,” Paul Chagonja, the police commissioner in charge of operations, told The Anadolu Agency.
"So far, we cannot confirm or deny that the group is related to Al-Shabaab, because we have not arrested any of them yet," he added on the phone.
He revealed that security forces that arrived at one of the caves – where the militants were hiding – found explosives, a motorbike, three bicycles and a rope.
He said the rope was most likely used by the militants in climbing to another cave and escaping the area.
According to Chagonja, the militants escaped through an opening in the roof of the cave.
He appealed to local residents to report any injured or dead victims in their neighborhoods.
The group, Chagonja said, is made up of ten people.
Earlier in the day, an internet video claimed that Sunday's attack was carried out by Al-Shabaab.
The video, which is in both Arabic and Swahili, called on Tanzanian youths to join hands with the militants in their war against those "who suppressed Islamic believers."
Al-Shabaab is most active in Somalia, where it has been fighting government troops for several years now.
In recent months, the group has started staging attacks against neighboring countries, including Kenya.