The ruling by the Berlin court of appeals stunned relatives of Hans Sachs, and came despite Germany's signature of the Washington Principles in 1998 agreeing to the restitution of art plundered by the Nazis.
Mr Sachs took 40 years to build one of the world's largest collections of rare posters only to see it looted at gunpoint by the Gestapo on the orders of Joseph Goebbels in 1938, the Times reports.
The court ruled that the museum could keep the stolen art works even though Peter Sachs was named the legal owner of his father's collection which included works by Henri de Toulouse-Lautrec, Ludwig Hohlwein, Lucian Bernhard and Jules Cheret. It has been valued at at least €4.5 million (£3.9 million).
"I cannot imagine that the Government will endorse the position that a Jew in Germany is once again deprived of what is legally his," said Suzanne Glass, Mr Sachs' great-granddaughter, who attended the hearing.
"The judge ruled that Peter Sachs is the legal owner but he has also ruled that he has no legal remedy to repossess the collection. It puts the German government in an untenable position."
Peter Sachs, 71, lives in Florida and his lawyer, Matthias Druba, said that an appeal to the German Supreme Court was likely.