Thursday, March 17, 2016

Overview of reports on compromise of Allied communications in WWII

When researching WWII signals intelligence and codebreaking operations it is important to track down all the available sources of information, from POW interrogations and captured documents to detailed reports summarizing Allied cipher policy, decisions and compromises.

It is reasonable to assume that in the postwar era all the Allied countries tried to assess the extent of the compromise of their communications during WWII. Thus I’ve tried to track down these reports. In the following space i’m going to list the reports I know of and give a short summary of each one. I’ve used most of them in my essays but there are some that I haven’t been able to locate (yet).

USA

Military cryptosystems

1). The ‘European Axis Signal Intelligence in World War II’ volumes are available from the NSA’s website. They are dated May 1946 and have a lot of information on German cipher machines, the German codebreaking agencies and their major achievements. Unfortunately I don’t think that the EASI volumes are entirely accurate.

2). The report Special conference on M-209 security is dated 1950 and looks into the German exploitation of the US M-209 cipher machine. It is difficult to explain why the people involved looked only at the available TICOM reports and not the actual files of the German Army’s codebreaking agency Inspectorate 7/VI. These had been retrieved in 1947 from a camp in Austria.

Overall it seems to me that US reports are not as thorough as they should be.

State Department

I am not aware of any State Department communications security report written after the end of WWII. The ones available were produced during the war and submitted to the US Joint Chiefs of Staff.

1). The reports of June 1941 and November/December 1943 are available online via the NSA’s Friedman collection.

2). The report of June 1944 can be found in the US National Archives and Records Administration - collection RG 457- Entry 9032- box 1384 - 'JCS Ad hoc committee report on cryptographic security of government communications'.

The problem is that these reports only have theoretical evaluations of the State Department cryptosystems, not information on their compromise by the Axis powers.

OSS

The Office of Strategic Services was the predecessor of the CIA and during WWII its overseas stations gathered military and political intelligence on the Axis powers. For example in Bern, Switzerland the local OSS station (headed by Allen Dulles) kept in contact with members of the German Resistance.

According to an OSS veteran a report on WWII crypto security was prepared but I don’t have any details.

Soviet Union

Both in the Soviet era and today in modern Russia the files and archives of the signal intelligence organizations remain closed to researchers. This makes it very difficult to find information on Soviet cryptosystems and on whether the Soviet authorities evaluated their communications security during WWII.

According to Anatoly Klepov's “The historical truth” of Beria and Suvorov about cryptography and radio intelligence, there were evaluations of Soviet cryptosecurity in 1941, in 1943 and at the end of WWII. These investigations came to the conclusion that Soviet ciphers had not been compromised by the Germans but that they were theoretically solvable. This led to the introduction of new, more secure systems.

UK

Army

There are files dealing with British Army communications at the National Archives, collections WO 219 and WO 193. However these deal with cipher policy, not cipher compromises.

Navy

The report ADM 1/27186 ‘Review of security of naval codes and cyphers 1939-1945’ is available from my Google docs and Scribd folders. It is dated November 1945 and it was written after examining the files of the German Navy’s codebreaking agency B-Dienst. Thus it has an excellent analysis both of Royal Navy cipher policy and the compromise of RN codes and ciphers.

RAF

The report AIR 20/1531 ’R.A.F. signal communications: security’ can be found in the National Archives but the information on the compromise of RAF systems is only based on decrypted German messages, not other sources.

SOE

The Special Operations Executive was an intelligence organization created during WWII. I don’t know if a report on the compromise of SOE codes was ever prepared. The British authorities certainly had the available information, since they captured the War Diary of Inspectorate 7/VI in 1947 and this included the summary reports of Referat 12 which dealt with Allied agents’ codes.

The report HS 7/41 ‘Section II: field cyphers; appendices L, M, N and O’ has a file titled S.O.E. FIELD CIPHERS which is available from the Arcre website. This file points out theoretical and practical problems with SOE cipher security but does not have information on the compromise of said systems by enemy codebreakers. 

France

French military high level codes were solved by the Germans in the 1930’s and up to the Battle of France. Unfortunately I am not aware of any French report summarizing their cryptosystems and their compromise during the war.

Czech Republic

During WWII the communications of the Czech resistance movement were solved by the Germans. It seems that at the end of the war the Czechoslovak government found out about this case and personnel of the Defense Ministry prepared the report ‘Dopady lúštenia šifrovacieho systému čs. londýnskeho MNO z rokov 1940-1945 na domáci odboj’.

I requested this report from the Czech Defense Ministry’s history department but they were not able to locate it. Let’s hope that some Czech researcher will be able to find it.

Poland

Polish military, diplomatic, resistance movement and intelligence service codes were read by the Germans during WWII. I don’t know if there are detailed reports in the Polish government archives but the following sources have valuable information.

1). The Cryptologia article ‘From the Archives: Polish Interwar MFA's Cipher Compromised?’ by Jan Bury presents two reports written in 1940 and 1947 by the intelligence officer Major Tadeusz Szumowski, concerning the security of the Polish diplomatic Code 45.  According to Szumowski this codebook had been compromised and the Germans could read the Polish diplomatic traffic since the leadership of the Polish Foreign Ministry chose to keep using it despite his warnings. Szumowski was correct about this case.

2). Recently the book KODY WOJNY. Niemiecki wywiad elektroniczny w latach 1907–1945 a losy polskich, sowieckich, alianckich kodów i szyfrów has been published by Marian Zacharski. I don’t know if the author has information on the Polish ciphers from Polish sources but he has uncovered material from the recently released TICOM files on the extensive compromise of Polish cryptosystems during the war.

Conclusion

Compromises of communications security are usually difficult to acknowledge by the countries that suffer them. For example since the 1970’s countless books have been written about the successes of Bletchley Park, yet detailed information on the Axis solution of Allied codes only started to become available in the 2000’s when TICOM reports and other relevant documents were released to the public archives by the US and UK authorities.

I’ve listed some reports that deal with the compromise of Allied codes but I hope that researchers will be able to locate more. Specifically:

1). State Department postwar reports on WWII compromises (especially the M-138-A strip cipher case)

2). The OSS report on WWII cryptosecurity.

3). The Soviet reports of 1941, 1943 and postwar period.

4). Reports similar to ADM 1/27186 for the British Army, RAF and SOE.

5). French reports on their WWII cryptosystems.

6). The Czech report ‘Dopady lúštenia šifrovacieho systému čs. londýnskeho MNO z rokov 1940-1945 na domáci odboj’.

7). Polish government-in-exile reports on their codes and ciphers. 

Friday, March 4, 2016

Did the German codebreakers solve the Japanese Purple cipher machine? – Information from mr Otto Leiberich

In the late 1930’s the Japanese Foreign Ministry distributed the Purple cipher machine to its most important embassies and it was used to encipher high level messages to and from Tokyo. Unfortunately for the Japanese the introduction of this new cipher machine wasn’t able to secure their diplomatic communications.

The codebreakers of the US Signal Intelligence Service were able to solve this device in 1940 and according to Russian historians the codebreakers of the Soviet Union, led by Sergei Tolstoy, also solved it.

The British codebreakers were not able to solve this system on their own but they received information and a copy of the device from their American allies in 1941.

Speculation on the German effort versus the Purple cipher machine

US reports based on the interrogation of German cryptanalysts claim that the Germans made an effort to solve the Purple cipher machine but were not succesful. However the German historian Jürgen Rohwer mentioned in his book ‘Stalin's ocean-going fleet’ that in 1996 he received a letter from a mr Cort Rave, who claimed that the Germans were in fact able to solve the Purple machine and decode the Japanese messages.

According to US reports Kurt Rave was an OKW/Chi employee detached to the Foreign Office Cryptanalytic Section (Pers Z S), so his testimony has to be taken seriously.



I have looked into this case and presented all the available information in German success with Purple?.

Unfortunately I haven’t been able to get a copy of the letter written by Rave. When I emailed mr Rohwer in 2011 he said he would look into it but he never got back to me.

Another person who made similar statements regarding the German solution of the Purple machine was Otto Leiberich, chief cryptologist of the German cipher department in the period 1972-1990.


Zwei Erfolge verdienen eine besondere Würdigung: die Entzifferung des Purple-Verfahrens der Japaner und die Entzifferung der amerikanischen Chiffriermaschine M 209.

Während des Krieges hatten die Japaner eine Chiffriermaschine entwickelt und zum Einsatz gebracht, die der amerikanischen Aufklärung größte Probleme bereitete. Da gelang es einer amerikanischen Gruppe um den Kryptologen William Friedman, diese Maschine, die als purple machine bezeichnet wurde, zu rekonstruieren und zu entziffern.

Dies gilt seither in Amerika als der größte Erfolg in der Kryptologie-Geschichte. Angeregt durch eine kürzlich ausgestrahlte Fernsehsendung fragte ich bei einem ehemaligen Kollegen nach, der während des Krieges auf diesem Gebiet tätig gewesen war, und erhielt bestätigt, woran ich bis dahin nur eine ungefähre Erinnerung hatte: Auch die Deutschen hatten die Sendungen der verbündeten Japaner bearbeitet, insbesondere die Meldungen, die der japanische Botschafter Oshima aus Berlin nach Tokio sandte. Einer Gruppe von Kryptologen und Technikern der Chiffrierabteilung des Oberkommandos der Wehrmacht (OKW) unter der Leitung des Mathematikers Erich Hüttenhain war die Entzifferung ebenso gelungen wie den Amerikanern. Hin und wieder war ein Bericht schon entziffert und weitergeleitet, wenn Tokio wegen Übermittlungsfehlern um nochmalige Übersendung bitten mußte. Wenn also die Entzifferung der purple machine der größte Entzifferungserfolg während des Zweiten Weltkrieges gewesen wäre (er war es nicht!), so hätten ihn Hüttenhain und sein Team ebenfalls errungen. Leider existieren in Deutschland hierzu keine Unterlagen mehr
.’

Since I was interested in this case I managed to contact mr Leiberich in 2013 and we had a short telephone conversation. Since his article was written a long time ago he didn’t remember all the details but he was able to give me the following information:

1). He did not have any documentary evidence and his statements were based on discussions with his colleagues.

2). Rave said it was a cipher machine and he remembered specific messages.

3). Regarding the years of solution he said probably 1943-44.

4). He had asked Rave why he hadn’t mentioned this success to the American interrogators and he responded that he feared it would be dangerous for him to say too much.

The last statement can be confirmed, in part, by the ‘Final report of TICOM team 3’, p12 which shows that some of the captured Pers Z higher-ups wanted to give away as little as possible and tried to influence their coworkers to do the same.