In a news conference, he said the Syrian people "desperately need action" but criticised the UN Security Council for "finger pointing and name calling".
Mr Annan authored a six-point peace plan for Syria which was intended to bring an end to the fighting.
But the plan was never fully adhered to by either side and the violence has continued.
UN Secretary General Ban Ki-moon said Mr Annan would not renew his mandate when it expires at the end of August.
He said he had accepted Mr Annan's resignation with "deep regret".
Speaking to reporters in Geneva, Mr Annan said: "When the Syrian people desperately need action, there continues to be finger pointing and name calling in the Security Council."
"It is impossible for me or anyone to compel the Syrian government, and also the opposition, to take the steps to bring about the political process.
"As an envoy, I can't want peace more than the protagonists, more than Security Council or the international community, for that matter," he said.
The veto-wielding members of the UN Security Council have failed to agree on what action to take on Syria.
