You know whats funny. i was also working on this back home base of mine on a pvp server... when i was crafting a bed and a clock and some item frames, i hear a enderman teleport right behind me o_o in my room... i didn't bother until i turned around and he stole one of my lapis lazuli floors... Dx so you know what. i hit him. and he hit me back, but the prob is he didnt hit me again... o_o... so i went like yeah, "please place that back before i kill you >_>" i watched him, caaarefully... and he teleported elsewhere.
Once, I was derping around on Creative and trying to make an epic house, when an Enderman popped inside. We chilled in the living room for a bit, then he gave me a housewarming gift- a Grass block that was in the most perfect spot to be a coffee table.
Then, on a multiplayer server, we were working on building a city, and there was this Enderman wandering about with his mouth open. No idea why, no one attacked him, he wasn't making his weird hiss noise... His mouth was just open. I figure he was going "ZOMAHGOODNESS! "
I also had an Enderman hide inside my cave house on Hardcore. I was practically crying because I was defenseless and he wasn't leaving. Eventually, I made him go out into this mineshaft that I found next to my house. Once he was out, he stood at my door and stared awkwardly at me for about 10 minutes :I
I've pissed off a enderman once, started attacking it, (somehow the attacks didn't affect it) but then it calmed down and was teleporting about my little island base that I was working on like "Oh cool place dude, like what you've done, and its tall enough for me to walk about, coool!" and for some reason now I will find a enderman or two hiding in there when it rains. (it has a roof)