The man initially demanded money but when he was refused, a shot was fired and the hostages taken.
Claiming to be linked to al-Qaeda, he demanded to speak to the elite Raid police unit that killed Islamist gunman Mohamed Merah nearby in March.
He freed two hostages before police stormed the bank and detained him.
The gunman was said to have been wounded in the thigh, but not seriously. His two remaining hostages are not thought to have been harmed.
'Religious reasons'
The man went into the branch of the CIC bank at 10:10 local time (08:10 GMT), taking the manager and three other people hostage.
The area around the bank was cordoned off and two nearby schools were closed.
The area around the bank was cordoned off and two nearby schools were closed.
Continue reading the main story
Mohamed Merah
French citizen of Algerian extraction, aged 23
Described himself as an al-Qaeda member who had spent time in Afghanistan and Pakistan
Behind murders of seven people
11 March, shot dead Sgt Imad Ibn-Ziaten in Toulouse
15 March, shot dead Corporal Abel Chennouf, 24, and Private Mohamed Legouad at a cash machine in Montauban
19 March.
attacked Ozar Hatorah Jewish school in Toulouse, killing teacher Jonathan Sandler, 30, sons Gabriel, 4, and Arieh, 5, and head teacher's daughter Myriam Monsonego, 7
22 March, Merah shot dead after siege at his flat
Merah's path to murder
Merah 'wanted to kill more'
Special police units from the GIPN (Groupe d'intervention de la police nationale) arrived from Marseille and Bordeaux.
No officers from Raid - which has its headquarters north of Paris, about 600km (400 miles) from Toulouse - were deployed to the scene.
A police union source told the regional newspaper Ouest-France it was not clear whether the man's claim about al-Qaeda was "serious or a fantasy".
More than four hours into the siege, police said a woman hostage had been freed in exchange for food and water. A second woman was released some time afterwards.
Public prosecutor Michel Valet told reporters that the man "wishes to let it be known that he is not acting for money, but for religious reasons".
The hostage-taker was believed to be around 30 years old and known to the authorities, Toulouse newspaper La Depeche reported. He was from Castres, to the east of Toulouse, and one source told the paper he was a schizophrenic who had broken off his treatment.
His sister, who was brought to the scene during the afternoon to help in negotiations, revealed that her brother was not religious and had been in and out of social care since childhood.
After almost seven hours, the siege came to an end. The BBC's Hugh Schofield in Paris said that three explosions had been heard, possibly stun grenades, shortly before news emerged of the gunman's arrest.
The gunman had come out of the building with one of the two hostages, reportedly threatening him with a gun, prompting the special police unit to launch its assault.
He was shot in the thigh, La Depeche reported, before being arrested. He was then taken to hospital.
Merah siege
The bank in which the drama unfolded is a few hundred metres from Mohamed Merah's flat, in an area adjacent to the barracks where Raid police were based during the March siege that ended in his death.
