Pakistan's army said a rocket fired during the battle hit a house on its side of the border, killing nine civilians. It denied any insurgents had crossed the frontier.
Extra troops have been deployed on both sides of the mountainous frontier in an attempt to prevent militants who find sanctuary in Pakistan's wild tribal regions from mounting crossborder raids and sustaining the five-year-old war.
NATO said militants attacked Afghan and alliance troops late Friday in the Bermel district of Paktika province. NATO and U.S.-led forces returned fire, killing about 60 fighters, an alliance statement said.
"These individuals clearly had weapons and used them against our aircraft as well as shooting rockets against our positions ... This required their removal from the battle-space," Col. Martin P. Schweitzer, a U.S. commander, said in a statement.
Schweitzer said some munitions fired in the clash might have landed over the border in Pakistan, but insisted his forces only targeted "bad guys."
Pakistan Army spokesman Gen. Waheed Arshad said one of several rockets which landed in Pakistan hit a house in Mangrotai area, killing nine civilians — five men, three women and one child.
Arshad said there also were reports of civilians killed on the Afghan side. Several civilians, including women and children, had straggled into Pakistan to seek medical attention, he said.
"They are not militants. They are not armed, they are civilians who were wounded," Arshad said.
An intensification of the fighting has translated into mounting civilian casualties that threaten to undermine the Western-backed effort to stabilize Afghanistan and prevent a Taliban comeback.
Police said Friday that a NATO air strike in the southern province of Helmand had killed 25 civilians as well as 20 militants who were firing on NATO and Afghan troops from a walled compound. NATO said the insurgents caused the innocent victims by hiding among them and defended the rights of its troops to defend themselves.
On Saturday, Afghan President Hamid Karzai accused NATO and U.S.-led forces of mounting "careless operations" that killed more than 90 civilians in the last 10 days.
Such operations "will have no benefit for Afghanistan," Karzai said, and urged foreign troops to improve coordination with Afghan forces in order to prevent civilian deaths.
In its Saturday statement, NATO said the insurgents that fought in Bermel were the largest formation observed in the area since January, when U.S. forces said they had killed around 130 of 180 insurgents crossing from Pakistan.
Schweitzer said that the number of insurgents reported moving over the Afghan-Pakistan border had also increased in recent months. However, he cautioned that the rise may be down to more troops observing activity and better cooperation with Pakistan.
Across eastern Afghanistan, where the U.S. leads NATO operations, militant attacks rose 250 percent in May compared with the same month a year previous, according to U.S. military information.
In an interview with The Associated Press, Schweitzer, who commands 6,000 troops in eastern Afghanistan, said tribal customs and open border access contribute to the freedom of movement for insurgents.
"The ability to move back and forth through that tribal network is pretty significant," he said. "Folks who've got a nefarious end state can manipulate that."
He said he was "absolutely convinced" that insurgents are trying to create friction between U.S., Afghanistan and Pakistan along the border, though he didn't elaborate on how they were trying to do that.
"The enemy would love .... to get us fighting each other, but I am telling you we are becoming more and more one muscle everyday," he said.
Karzai has in the past accused Pakistani intelligence of aiding the Taliban in order to destabilize his government, which some in Pakistan view as too pro-India.
However, Pakistani President Gen. Pervez Musharraf insists his country is a victim of instability spreading from Afghanistan and that the deployment of 90,000 Pakistani troops on his side of the border has not been matched by Kabul.
U.S. officials have tried to calm tempers among its two key allies, and commanders from all three sides hold regular meetings to exchange intelligence.
The eastern region has also seen a surge in suicide bombings, raising concern that the resistance is beginning to resemble that in Iraq.
Suicide attacks in the region increased 230 percent in the first half of 2007 compared with the same period in 2006, U.S. spokesman Maj. Chris Belcher said.
However, Schweitzer said the increase shows that the Taliban are less able to mount large-scale attacks.
"That to me is not the barometer that it's getting bad," he said. "What would be the barometer to me that it's getting bad is if they do large scale attacks everywhere and they're being effective."
In other fighting reported Saturday by the U.S.-led coalition:
_Afghan and coalition soldiers killed nearly 20 militants during a seven-hour gunbattle in Shah Wali Kot district of Kandahar province.
_Coalition troops detained about 20 suspected al-Qaida fighters after a clash at a compound in Ghazni province.
_Afghan and coalition forces killed several militants who fired rockets at their base in Shaheed Hasas district of Uruzgan province