A few things:
1. I don't have any specific knowledge of German law, but I'm fairly confident that some of the things Rumsfeld is reputed to be responsible for are crimes pretty much anywhere.
2. Belgium passed a
similar law in 1993. A number of prosecutions were brought under it, including some of US officials and the first president Bush. The government intervened and terminated the prosecutions under fairly heavy diplomatic pressure.
3. A similar case, in which Belgium made out an arrest warrant for the foreign minister of the Congo, went to the ICJ. The court found that the minister was immune from prosecution, but there were a lot of differences in the facts.
link
EDIT: 4. Even if Rumsfeld never went to prison, he'd still be unable to travel to Germany, or maybe to anywhere in the EU.