In grapheme-color synesthesia, the letter "c" printed in black may be experienced as red, but typically the color red does not trigger the experience of the letter "c". Therefore, at the level of subjective experience, cross-activation is usually unidirectional. However, recent evidence from digit-color synesthesia suggests that at an implicit level bidirectional cross-activation can occur. Here we demonstrate that this finding is not restricted to this specific type of synesthesia. We introduce a new method that enables the investigation of bidirectionality in other types of synesthesia. We found that a group of grapheme-color synesthetes, but not a control group, showed a startle in response to a color-inducing grapheme after a startle response was conditioned to the specific corresponding color. These results implicate that when the startle response was associated with the real color an association between shock and the grapheme was also established. By this mechanism (i.e. implicit cross-activation) the conditioned response to the real color generalized to the synesthetic color. We suggest that parietal brain areas are responsible for this neural backfiring.