The following is from מו׳ר הגאון הגדול הרב Yosef Dov Halevi Soloveitchik ז׳ל.
R’ Yosef Karo in the standard Shulchan Aruch (תרפא:ב) concludes that we light the Chanuka lights before the Havdolo candles. However, when one comes home after Shule, many follow the Taz, that first one performs Havdala and then lights Chanuka licht.
הגאון הגדול הרב Rav Moshe Soloveitchik ז׳ל (the Rav’s father, and eldest son of R’ Chaim Brisker) explains that there is a difference between lighting in Shule and lighting at home.
The prime purpose of lighting in Shule is a requirement on the congregation to publicise the miracle. As the Vilna Gaon is quoted by his students (note the Biur Hagro was not actually written by the Gaon, but by his students) (תרעא:ז) that every place where the Rabbis required the concept of publicising a miracle, they also required that this miracle is also publicised by the congregation. They bring a proof from Hallel on Pesach Night at Maariv (which by the way, the Rav used to say in davening even though he davened Nusach Ashkenaz; the Rav was not afraid to “correct Nusach” e.g. He also said the Avoda of Yom Kippur according to Nusach Sefard because he felt it was a Halachically more accurate description of the Avoda of the Cohen Gadol). On Pesach night, Chassidim, and those who daven Nusach Sefard, don’t follow the Ramoh and, per R’ Yosef Karo, the Mechaber
בשו”ע או”ח סי’ תפ”ז ס”ד “בליל ראשון של פסח גומרין ההלל בצבור בנעימה בברכה תחלה וסוף, וכן בליל שני של שני ימים טובים של גליות
[As an aside, I remember the Rabbi of an important Shule in Melbourne, who used to daven at Chabad in the evening on Pesach Night because it was near his home and his Shule probably didn’t have a Minyan or it was too far for him to walk to as he got older, and when Chabad/Nusach Sefard started Hallel, he would leave Shule. He sat behind me. I was young, but I thought and continue to think that this was not the correct behaviour, but I will leave that issue as he is in another world.]
Back to Pesach. Even though we are required to say Hallel over a cup of wine (at the Seder) that is our personal requirement. However, the congregation, has a separate requirement to say Hallel as a congregation ציבור. When does a congregation get the “halachic designation” of a congregation? If they davened Mincha together, they are a Tzibbur/Congregation that group “is existentially formed” and now must perform the congregational פרסומי ניסא. For this reason, we light in Shule between Mincha and Maariv, even though many have the custom to light after Maariv at home. The reason being that the congregation assumes it’s requirement to light, as soon as they are designated as a congregation, and that occurs immediately after Mincha, because they have an on following requirement to continue with Ma’ariv.
Therefore, in respect of Shule, after Ma’ariv, where they no longer have any congregational duties, there is no more “congregation” and no special requirement to have פרסומי ניסא. Most people might still be there, however, they aren’t halachically a congregation requiring the lighting once they have completed their davening.
So let’s turn to Motzai Shabbos where we can only light the candles once Shabbos goes out. It would seem that since they have already davened, they no longer are designated as a congregation and no longer a requirement to light as a congregation. In order to avoid this conundrum, the Minhag has become to light the Havdala, as a congregation, after Chanuka lights, because at that time, they are still a congregation requiring Havdala, and therefore the פרסומי ניסא of a congregation has not dissipated. Note that the definition of a congregation is not whether most are there or not. Rather, it is about whether those who are there, are still considered a congregation because they haven’t completed their full davening.
Rav Soloveitchik wondered about gatherings where there was no congregational activity, such as Ma’ariv, e.g. at a fundraiser or the like where most would have already davened Ma’ariv in their own congregations. As such, Rav Soloveitchik questioned whether in such circumstances there was a congregational פרסומי ניסא that was incumbent halachically.
One could turn attention to the “Chanuka in the Park” type celebrations. From my observation, it is sometimes dubious that there is a congregational requirement for publicising the miracle. However, if one assumes, quite reasonably, that many of the people will consider this their private and only lighting of Chanuka candles on that evening, it perhaps would be that an entrance fee be charged, nominally, so that they can become part of the pseudo-mega-household that is lighting the Menora (as opposed to a congregational Chiyuv) to do so. Yes, I see many Chabadniks put Tefillin on people, but the minute they have finished with their Tefillin, those people have done a personal Mitvah, but not a congregational activity that is still incomplete.
It is somewhat ironic, but exact, that congregation isn’t defined by numbers, but by responsibility. Even a group of 10 is a congregation, and as long as they still have a congregational task, they must light Chanuka lights in the Shule. Yet, one could have 5000 people with no Halachically congregational requirement to light Chanuka lights because they aren’t a צבור halachically, because they are not involved in a Tzibbur mandated affair.
Don’t get me wrong. I’m not arguing against lighting Chanuka candles in public places. I’m simply repeating the precise halachic categorisation of these acts, as per the words of the Rav and his father.
There are other explanations, including the need to have Chanuka at home with food, and at Shule it’s only for those who don’t know how to perform at home. I’m not going there.