he's finally moving to the other branch!

I was told by my boss that he's moving to the other branch. I won't see him anymore. I'm relieved but also so sad that we won't see each other anymore. I went in to give him some stuff of his and to say goodbye and he wouldn't even look at me. I said I just wanted to say goodbye since I won't ever see you again. (Well, when the baby's born and I take him to court I will) and he wouldn't say a word. I said so that's it? You can't even look at me? He said no actually I can't. So I said I just wanted to say goodbye so have a nice life then. And walked out crying. It amazes me he treats me like the villain. What exactly did I do that was worth his hatred? I didn't give into his bullying and manipulation? So now I'm the bad guy. I'm the one who deserves more abuse. Ok.
Then my boss is telling me how if she hears I told anyone its his I'll get fired for slander! Um... No. Slander is false accusations to try and ruin someones reputation. Never did that! And its the truth! All the other people who are going around telling everyone? That's slander because I never said it to anyone! The ones who are talking? Did they ask me? NO! Therefor they are going on rumors and hearsay. I'm going to a lawyer about this bullshit. I'm tired of it.