This question has always bothered me and it came up again as I started learning daf yomi and the first mishna talks about צאת הכוכבים being the זמן for ק"ש.
Everyone should know the famous שיטת ר"ת (which is accepted by the major rishonim) that צאת הכוכבים is 4 מיל (at least 72 minutes) after sunset. Until 3.25 מיל after sunset (at least 58 minutes) it is definately day. The Gra has his famous question that this contradicts reality, 50+ minutes after sunset it is completely dark, how can that be considered day. Yet, this opinion is the opinion of many of the rishonim and is the opinion quoted in the ש"וע and the רמ"א. I have not yet seen a good explanation of how ר"ת fits in with reality and it bothers me a lot that this is the majority opinion of the rishonim when it so clearly is contradicted by reality.