The hardest thing for me is avoidance of idioms. I tend to speak more slowly to those whose English I know is a second language, and I certainly think about my words much more methodically than I normally do. (Which may be a good habit for me anyway, since I’ve been known to put my foot in my mouth. See! Idiom!) On the other hand, I’m incredibly happy to learn the local idioms, since I find them fascinating. And, like any idiom, they often don’t make much sense at first blush. (Another one!)