This isn't true. Islam as a whole becomes a threat to the rest of the world when moderates and fundamentalists support and strive to work towards getting a government set up that supports Sharia Law.
Sharia Law is dangerous to modern societies.
My answer to the question would be, that you start worrying about Islam the moment they try mixing government and religion and promoting Sharia Law to oppress non believers.
I am shocked that I am the only person in this thread who has mentioned Sharia Law.