Our indigenous plants aren't toxic!

Many popular fruits and plants have the same "toxic" compounds that are found in our indigenous plants.  Examples include apricots, cherries, plums, and apples.  They all include cyanogenic glycosides, so why do our indigenous plants get a bad rap?