This quote has become the battle cry of the Charedi world, that you have to unquestionably follow the decisions of the Gedolei Hador. Many also quote Rashi (Parshas Shoftim) where he quotes the Sifri that אפילו על שמאל שהוא ימין, even if they are making a mistake you still need to listen.
I would like to discuss whether the modern day application is necessarily correct and some other interpretations of this principle.
There are 2 premises here:
1. This din applies to the Gedolim today
2. You need to listen even if you know that they made a mistake.
Premise 1 would seem to be a שיטת יחיד.
The Chinuch says this in Mitzva 495,496 however, as the Minchas Chinuch points out there he doesn;t know what the source for this is. In fact, the Rambam seems to disagree as the Rambam puts all of these dinim in the Perek where he discusses the ב"ד הגדול. It would seem that according to the Rambam this din would only apply to the ב"ד הגדול.
The Gemara in Horiyos (2b) explicitly contradicts premise 2.
A little background. The beginning of Horiyos discusses the halachos of a פר העלם דבר של ציבור. If the ב"ד הגדול paskens that something where the punishment is כרת is permitted and the majority of the people follow the psak and do the aveira, the individuals do not need to bring a korban chatas, rather the ב"ד brings a פר העלם .דבר של ציבור
The first Mishna in Horiyos discusses the case where a member of the ב"ד disagrees or even if a talmid who is ראוי להוראה disagrees with the psak of the ב"ד. The Mishna says that if they rely on the psak of the ב"ד they will not be covered by the פר העלם דבר של ציבור, but rather will need to bring their own private חטאת. The Gemara asks why are they considered a שוגג if they knew the pask was wrong. The Gemara answers that they made a mistake in the מצוה לשמוע לדברי חכמים. They thought that they needed to listen even when the ב"ד made a mistake.
You see explicitly from the Gemara that you are not supposed to listen to the ב"ד הגדול if you know that they made a mistake.
This seems to contradict the Sifri that Rashi quoted. In addition, the Yerushalmi also seems to contradict the Sifri. The Yerushalmi writes that I might think that I should listen if the chachamim tell me על שמאל שהוא ימין, the Yerushalmi says קמ"ל that you should only listen when they say על ימין שהוא ימין.
The Rishonim and Acharonim offer a number of resolutions to these contradictions.
A number of mefarshim of Rashi claim that the Sifri is only talking about where you think that the ב"ד is wrong but you are not sure. In that situation you just listen to the ב"ד. However, if you are sure, then you should not listen and this is the Gemara in Horiyos and the Yerushalmi.
The Ramban in his השגות on the Sefer Hamitzvos (שורש ב) offers the following resolution. He says that if you disagree with the ב"ד הגדול you should follow your opinion until you go to them and present your arguments and they reject them. He says that this is pshat in the Gemara Horiyos. However, once they reject your arguments and maintain their psak the din of the Sifri applies that you must listen.
The Ramban is difficult because the case of the Mishna in Horiyos includes where a member of the ב"ד that ruled on the case disagrees. It is hard to imagine that he did not present his arguments when the case was heard and yet, the Gemara still says that he should not follow the psak of the ב"ד.
R' Elchanan in קונטרוס דברי סופרים offers the following resolution. Part of the איסור of בל תוסיף is that the חכמים are not allowed to label a דין דרבנן as a דין דאורייתא. The חכמים are allowed to create דינים דרבנן but they must be clearly labeled as such. Based on this he explains the Yerushalmi as follows. The Yerushalmi is talking about a case where the חכמים are labeling a דין דרבנן as a דין דאורייתא, that is what ימין שהוא שמאל means and therefore you are not allowed to listen. The Sifri on the other hand is talking about a completely different case. The Sifri is talking about where the חכמים tell you not to perform a mitzva, for example not to blow shofar on shabbos. Even though they are telling you not to do a mitzva (ימין שהוא שמאל) that is within their authority and therefore you have to listen.
According to R' Elchanan, it comes out that the Sifri has nothing to do with listening to the חכמים in general, it is talking about a very specific case.
The bottom line is that the application of this principle to the statements of the "Gedolim" regarding elections etc. and saying that everyone has a חיוב to listen would seem to be on very shaky ground.