QUESTION: To determine long‐term vegetation changes in revisitation studies, it is crucial to know how much of measured species turnover over time can be attributed to pseudo‐turnover (i.e. turnover caused by imperfect data acquisition), and which factors contribute to observation bias and pseudo‐turnover. Independent simultaneous surveys provide a powerful tool to quantify pseudo‐turnover and to indentify factors causing it, which may vary strongly between lowland and mountain areas. LOCATION: Alpine mountain summits (2616 m to 3418 m a.s.l.) in the southeastern Swiss Alps. METHODS: Plant inventories of 48 summits were collected by two independent observers simultaneously. Pseudo‐turnover between observers was compared to species turnover over one century based on historical species lists of the same summits. Variables linked to observer characteristics and external (observer‐independent) factors were tested for their influence on pseudo‐turnover and number of species missed by one of the observers, and plant characteristics were tested for their effect on species detection probability. RESULTS: Mean pseudo‐turnover between observers (13.6%) was almost three times smaller than species turnover over one century (41.4%). Pseudo‐turnover and the number of species missed increased with difference in botanizing time between observers and with a longer ascent to the summit, especially in combination with a high species richness on the summit. Species had a higher probability to be missed if occurring on many summits but with a low abundance, if small in stature and if belonging to certain taxonomic plant groups (e.g. Asteraceae). CONCLUSIONS: Our critical evaluation of turnover over time vs pseudo‐turnover confirms that floristic changes on alpine summits over time represent an ecological pattern. In mountainous terrain, factors related to observer characteristics play a major role, as we found the best correspondence between simultaneous records when the difference in botanizing time was small and the ascent was short. Our results help to improve data quality in mountainous terrain by pointing out possible causes for observation bias. Long‐term vegetation studies in alpine ecosystems should make a strong effort to identify and minimize such causes in advance, for instance by reducing between‐observer differences in botanical skills, fitness and time management through appropriate training.