Microsoft just demoed real-time translation in a Skype call. They wisely chose the relatively-compatible English and German languages for the demo, but there are so many ways it can go wrong even between such close cousins.
And when you think about expanding beyond the simplest cases, well, consider the following tray of meatballs: