Some 500 people, mostly men and children, assembled to join President Susilo Bambang Yudhoyono for a mass prayer at 7:40pm (1240 GMT).
Three of the mosque's ancient Moorish-style domes have been repaired and freshly painted ahead of the anniversary, but its minaret was on Monday still surrounded by bamboo scaffolding.
Hundreds of bodies were piled at the Grand Baiturrahman mosque in the days after the tsunami smashed into Indonesia's Aceh province, killing some 168,000 people.
The clerics and others at the mosque, which suffered damage mainly to its minaret and library, played a major role in helping thousands of trauma-stricken people in the weeks afterwards.
Muhammad Husin, who has been working at the room where worshippers' shoes are kept since 1977, worked for days to help clean an estimated 700 corpses.
The 70-year-old, who escaped from the huge waves by hiding on the mosque's second floor, said while manning his post Monday night: "I am thankful to be safe and for the opportunity to serve this mosque a little longer."
Another employee, janitor Azhar Husin, said the memory of residents fleeing the waves and corpses strewn over the mosque grounds "sometimes makes me traumatized."
"But since I am a man of faith and work here every day, I have found peace," Husin said.
Sardikin, a 35-year-old fish seller, dropped by the main mosque a few hours before the prayers to remember his parents "and pray that their good deeds in their lifetime are accepted by God and that there will be no repeat of the tsunami."