Sunday, February 2, 2014

State Department’s strip cipher – reuse of alphabet strips and key lists

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1,B1,C1,D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until late 1944) and by reading these messages the Axis powers gained insights into global US policy.
The strip cipher was not a weak system cryptologically, even though it could not offer the security of cipher machines. The success of German and Finnish codebreakers was facilitated in many cases by the poor way that the system was used by the State Department.

Use of the M-138 strip cipher by the State department
Each embassy had 50 ‘circular’ alphabet strips and 50 ‘specials’. The ‘circulars’ were used for communications between embassies and for messages from Washington to all embassies. The ‘specials’ were used for direct communications between Washington and a specific embassy.

The way the system worked was that each day 30 alphabet strips were chosen out of the available 50 (both for the ‘circulars’ and the ‘specials’). The strips used and the order that they were inserted in the metal frame was the ‘daily key’. 


The strip system did not have a separate ‘key’ for each day. Instead there were only 40 different rearrangements for the entire year.
The daily key table indicated which of the 40 keys was valid for the specific day. For example in the following table assuming that the date is April 10 then the numerical key to be used is 30.





Using the 30th numerical key we would insert in the panel first the alphabet strip with the number 19, then in the second position no43, in the third position no24……… and finally in the 30th position no21.

Obviously having only 40 keys was not the best choice. Enemy codebreakers would only need to solve the traffic of 2-3 months and then they would have recovered almost all the numerical keys. Although I’ve accused the State Department for misusing the M-138-A in this case they were simply following the same instructions as the US Armed Forces.

According to Special Research History SRH-366 ‘History of Army Strip Cipher devices’ the US Army also used a daily key table (but with 50 keys) till mid 1942. From then on they introduced a separate key for each day and the list was changed each month.


This change however was not implemented by the State Department. Was this the only mistake of the State Department’s cipher unit? Apparently not.
Reuse of alphabet strips and key lists
According to the information presented so far it is clear that the strip system was based on the daily key table, the numerical key and the alphabet strips. Enemy codebreakers would need to recover all three parts in order to solve large numbers of messages. The only other way was to attack messages individually and that was too costly in terms of time and manpower.

The work of German and Finnish codebreakers was made much easier due to the fact that all three parts of the strip system were reused. Let’s have a look at the available information from postwar reports.
From TICOM I-201 ‘Interrogation of Franz Weisser , Dr Phil Studienassessor of Anglo-American section of OKW/Chi’, p3

The small Finnish staff under Colonel HALLAMAA also collaborated quite effectively giving hints. Thus it found out that the systems used in different countries after some time emerged again in other countries.
From TICOM I-145 ‘Report on the US strip system by Reg Rat Dr Huettenhain’  

Only a little of the material received could be read at once. Generally it was back traffic that was read. As, however, the different sets of strips were used at different times by other stationsit was possible, in isolated cases, to read one or the other of the special traffics currently.
The use of the same alphabet strips by different embassies is confirmed by a Japanese message listing the different strips and the posts that used them. This was information they received from the Finnish codebreakers.



According to a US report from November 1943 the same strips were used by 2-4 stations but an effort was underway to introduce new ones, used by only 2 holders at the same time:

The old systems did not use channel elimination and the same set of strips was sometimes held by 2 to 4 stations, however new systems using channel elimination and limited to 2 holders are being distributed as fast as possible



Thanks to the continued use of alphabet strips the Axis codebreakers only needed to recover the daily key tables and numerical keys.
Or did they?

According to David Kahn in 'Finland's Codebreaking in World War II':                                                        
Each post had its own set of strips; the key changed daily but was the same for all posts. This cryptographic weakness was probably permitted for logistical reasons.’

If the daily key for the ‘special’ strips was the same for all posts that would mean they had the same daily key table and numerical key.

Originally I thought that this could not have been allowed by the State Department and instead it was a reference to a limited number of daily key tables and numerical keys for use with the ‘special’ strips but according to a series of security studies submitted to the US Joint Chiefs of Staff in 1944 it is exactly what was happening during the war.


In the US National Archives and Records Administration - collection RG 457- Entry 9032- box 1384, the file 'JCS Ad hoc committee report on cryptographic security of government communications' shows the results of studies of the State Department’s communications security for the period 1941-44.
Investigations were carried out in June 1941, November and December 1943 and June 1944.

 

 



These investigations identified numerous problems with State Department security such as poor procedures for securing classified material, unclear classification procedures, use of compromised codebooks, operator mistakes, inadequate training program for cipher personnel etc etc.

In the field of cryptology the main problems were the limited number of cipher machines in use, the continued use of codebooks that were considered to be compromised and in the case of the M-138-A strip cipher the use of the same keylist by several holders of ‘special’ strips. This problem was identified not only in the 1944 survey but also in the 1943 one.
 
 
 


Other problems associated with the use of the M-138-A were operator mistakes when choosing the cipher columns and the fact that the same keylists were used with consecutive sets of alphabet strips.
Another file in collection RG 457 - Entry 9032 - Box 214 - ‘M-138-A numerical keys/daily key table/alphabet strips’ has several sets of daily key tables, numerical keys and alphabet strips (both ‘circular’ and ‘special’). It is possible that since these alphabet strips and keylists are complete they were sent from the State Department to the Army’s SIS- Signal Intelligence Service so that they could be compared with the material intercepted on Japanese links. In late 1944 intercepted Japanese messages on the link Budapest-Tokyo, contained US diplomatic alphabet strips and keylists. At that time the Germans and the Finns had given the Japanese officials a lot of material on the strip cipher for transmission to Tokyo.
 


According to documents found in this file a limited number of ‘special’ strips and key lists were used and these were coupled for use by specific embassies. Also the ‘circular’ strips valid in the period 1943-44 used the same daily key table and numerical keys. For example the daily key table and numerical key No19 was used with strips 25-4 and 113-1 by the Calcutta and Vladivostok consulates.  No 3 daily key table and numerical key was used with strips 33-1 and 34-1 etc etc


The following key list and numerical keys were used with the ‘circular’ strips 0-3 (introduced in March 1943) to 0-8 (introduced in ?).






   
Conclusion

From the JCS security surveys it is obvious that the State Department’s communications security had many weak spots both in handling classified material as well as transmitting them securely. The main problem was the limited number of secure cipher machines and the continued use of systems that were either considered to be compromised or were vulnerable to operator errors.

The M-138-A strip cipher was cumbersome to use but offered adequate security provided it was used properly. Unfortunately the State Department’s cipher unit has to take the blame for using this system in an awkward manner and thus making the work of the Axis codebreakers much easier than it should have been.
The major problems were the use of only 40 ‘keys’ for the period of validity, the use of the same 'special' strips by several embassies and the coupling of the ‘special’ strips with only one keylist. Also the fact that the ‘circular’ strips valid in 1943-44 used the same keylist.

The State department implemented security changes in 1944 and these limited the Axis success but did not defeat it. It seems that from January ’44 the keylists remained the same but the alphabet strips were changed each month.
A more secure procedure was introduced in May ’44, when the new M-138-A manual specified that all messages had to use channel elimination. According to German and Finnish accounts their solution of the strip cipher ended in September 1944 so it would seem that after a few months this measure finally secured the US diplomatic traffic. 

No comments:

Post a Comment