On 8 September 1942 Ullmann was deported to the Theresienstadt concentration camp. The particular nature of the camp at Theresienstadt enabled Ullmann to remain active musically: he was a piano accompanist, organized concerts ("Collegium musicum", "Studio for New Music"), wrote critiques of musical events, and composed. He wrote: "By no means did we sit weeping on the banks of the waters of Babylon. Our endeavor with respect to arts was commensurate with our will to live."
On 16 October 1944 he was deported to the camp at Auschwitz-Birkenau, where on 18 October 1944 he was killed in the gas chambers.
One writer has described Ullmann's milieu in these terms: "Like such other assimilated German-speaking Czech Jews as Kafka and Mahler, Ullmann lived a life of multiple estrangements, cut off from Czech nationalism, German anti-Semitism and Jewish orthodoxy”.