lots of dutch people speak english to each other.
That is just so not true. Really, where have you heard this?
I don't know about "lots", but I frequently talk English to a large number of my dutch friends. (What happens even more is that we switch between the languages several times in the same conversation (sometimes even in the same sentence), based purely on which language is best suited to express a concept.)