Should i cancel the date?

well it so happened that this colleague of mine dropped me off to my home after a drunken night incident.although i was drunk i remember almost all the happened along.he was insistent that he would drop me home and even offer to carry me wen i was unable to walk out of the cab.anyways he dropped me off to my room and went back home.i really like this guy and thought i could take a step forward by taking him out for dinner so i asked him out bt the way replied back gave the impression that he is not much interested in going out.fine with me bt he reply didnt go well with me and i was a bit rude in my reply back.now he has finalised a date out for dinner bt now i sincerely dont feel like going out with him.this has happened everytime.one moment he would be sweet and charming as hell and the moment i try to take a step forward he lashes out like anything. i am bit confused here..sud i still go for dinner with him..honest advice please.