Speaking after talks with Israeli Prime Minister Ehud Olmert, Mr Annan also appealed for the unconditional release of Israeli soldiers held in Lebanon.
The UN chief met Mr Olmert in Jerusalem after visiting Lebanon as part of a regional tour to bolster the ceasefire between Israel and Hezbollah.
Talks have also focused on the deployment of UN troops in Lebanon.
Mr Annan said he hoped Israel will have withdrawn from southern Lebanon once 5,000 UN peacekeepers are on the ground.
The UN resolution which led to the ceasefire calls for a 15,000-strong force - significantly larger than its current size of about 2,500.
Pull-out hope
At a joint news conference following their meeting, Mr Annan said the blockade should be lifted to help Lebanon recover from the conflict.
Mr Olmert did not respond directly, saying only that the ceasefire must be implemented in full.
"The international community must not divert its attention," until that happens, he said.
Israel has previously said it will not lift the blockade until concrete steps are taken to prevent weapons for Hezbollah being smuggled into Lebanon.
Israel has also insisted it will not pull out of Lebanon until the expanded UN peacekeeping force is deployed alongside the Lebanese army in the south.
Mr Annan said he hoped this could be achieved soon.
"We have about 2,500 troops on the ground and we are going to try to double that number and get to 5,000 in the coming days and weeks," he said.
"I hope that as we do that, the Israeli withdrawal will continue and by the time we are at that level, Israel will have fully withdrawn and we will have an effective, credible force that we can continue to build on."
The international community has so far pledged about half the target number of 15,000 troops.
Captives call
The secretary general also appealed for the "unconditional release of the abducted [Israeli] soldiers", a key Israeli demand.
On Tuesday, Mr Annan met the soldiers' parents and promised to do "everything in his power" to bring about their release, said Benny Regev, brother of Eldad Regev, one of the missing men.
Mr Regev also said the UN chief had promised to push for "signs of life" of the captives.
After visiting Israel, Mr Annan will travel on to Iran and Syria, countries with close links to Hezbollah.
Source:BBC
Güncelleme Tarihi: 20 Eylül 2018, 18:16