NSA (National Security Agency) "Eavesdropping On Hell", by Robert J. Hanyok

Historical Guide to Western Communications Intelligence and the Holocaust, 1939-1945

Chapter 2: Overview of the Western Communications Intelligence System during World War II

Step 3: Processing the Intercept

Once the enemy’s messages were intercepted, and if they were not decrypted at the site, then they had to be forwarded to theater or national centers for processing. Generally, tactical military communications, that is, those between Axis units of division-size and below, were decrypted and translated at a large intercept site or at intermediate processing centers located in such places as Hawaii, Egypt, Ceylon, and Australia. Allied tactical field intercept units also could monitor, decrypt and translate low-level encrypted or plaintext enemy messages and pass them along to their immediate commands. So-called higher level cryptographic systems - and this meant all traffic encrypted by Enigma, Purple, and other machines as well as Axis and neutral diplomatic messages - were processed at the major analytic centers of Bletchley Park outside of London, Berkeley Street in London, England, OP-20-G Headquarters on Nebraska Avenue, N.W., Washington, D.C., and at Arlington Hall Station in northern Virginia

Early in the war, intercept sites within the U.S. or U.K. sent the intercepted messages to the national centers over cable by encrypted teletype- writer. Overseas facilities sent their material by air transport. This meant that intercept could take weeks to arrive at a processing center. Later in the war, when enciphering communications equipment had been installed at major overseas intercept sites, the intercepted cipher text could be transmitted by enciphered radio or underwater cable to the national centers for decryption and translation. To securely transmit this intercept back to the processing centers, the Western Allies used their own high-level cipher machines, notably the British TypeX or the American SIGABA, both of which were considered unbreakable by Axis cryptologists. [52]

By war’s end, this enciphered communications system linked most of the Allied intercept and processing centers together in a worldwide network that allowed intercept from a remotely situated monitoring site to reach the appropriate analytic center. In the United States, so-called traffic or signal centers were set up in Washington, San Francisco, California, and Seattle, Washington. These centers relayed the intercepted traffic from overseas stations via cable to the Signal Security Agency at Arlington Hall, Virginia. British or Commonwealth intercept destined for the SSA came from London through the British Security Coordination located in New York City.

Above: Allied Communications Network (Source: RG 457, HCC, Box 843, “Lines of Communication - Traffic Exchange”)

The BSC also relayed Canadian intercept from Ottawa, Canada. U.S. field stations as far away as Kunming, China, and Asmara, Ethiopia, could get their intercept transmitted to SSA Headquarters within a day via this series of radio and cable relays. The chart on page 37 from June 1945 illustrates the network.

Once the intercepted messages arrived at the centers, they were sorted by target country and service element. The copy was then distributed to the appropriate target country office for analysis. So-called traffic analysts performed the initial evaluation of message “externals.” (The British referred to traffic analysts as “discriminators.”) This traffic analytic process can be likened to the study of the outside of a mail envelope. Even without reading the enclosed correspondence of a piece of mail, the recipient’s and sender’s identity and addresses, the postmark, weight and size of the envelope, the postage class, special handling instructions, and the history of prior correspondence between the two can reveal much about the content of individual letters.

At Arlington Hall Station, analysts from the B-I section performed the initial review of the intercept. The externals they studied included parts of the intercept such as station callsigns, message precedence (or its urgency), message serial number, the number of recipients and relays (if used), and tips to the cryptographic system. These items were studied for clues to the content of the message and perhaps the identity of the ultimate recipient. For example, many diplomatic messages were sent to radio terminals with only the notation of the ministry for which it was intended, such as “Ministry of Foreign Affairs.” If possible, the analyst would try to determine to which department the message was addressed. The analyst would review the files on previous intercepts to see if there were any clues that indicated the message’s recipient.

Many countries employed numerous cryptographic systems. For example, France (including the Vichy, Gaullist, and Giraudist factions) used over 100 systems for its armed forces, diplomatic, and colonial communications. Switzerland had available over ten manual systems and a simplified version of Enigma just for its diplomatic messages. [55] Not all countries used all of their systems all of the time. Some systems were reserved for messages of certain levels of importance. Some military or naval encryption systems were designed for use by a single command or region. For some countries, most of their communications were encrypted using relatively few systems. For example, although the Vatican was known to have nine diplomatic systems available for its apostolic delegates, it used only three systems, and one of those was so seldom used by the Vatican that American and British codebreakers could make no progress against it. [56]

It has been popularly portrayed that Allied cryptologists often could read enemy messages before the intended recipient. In a few instances this was true, such as occurred with some Purple decrypts prior to Pearl Harbor. Generally, though, this capability was rare. The ability to exploit encrypted traffic in a reasonable time after intercept was possible only after the cumulative application of physical resources and months of human analytic effort devoted to a single system such as Purple or the constellation of Enigma variants. [57] Allied cryptologists did not reach that level of expert or timely exploitation for all Axis cryptographic systems.

Another aspect to the cryptanalytic process was the constant risk that Axis cryptographers would change or replace current codes or ciphers that the Allies were exploiting. This danger existed for all major Axis cryptographic systems. There was a notable example of this in January 1942 when the German Navy replaced the three- wheel Enigma used by the U-boats with a newer four-wheel version. As a result of this move, the Allies were unable to exploit U-boat messages encrypted in the new Enigma until November 1942. In another example, in September 1941 the German Police began to change over its current manual encryption system, a double transposition cipher to another, a double playfair. [58] This change was completed by early November. The playfair was quickly broken; in fact it was a simpler system. This switch illustrates the inherent risk that faced Allied codebreakers every day during the war. [59] (See pages 49-50 for more on the background to this change.)

It is also true that not all foreign cryptographic systems warranted the same level of cryptanalytic effort that had been made against the Purple machine or the various versions of the Enigma. There were a number of reasons for this. As pointed out above, there were quite a number of systems to attack, far more than the number of available cryptanalysts. For many systems, including manual codes and ciphers and some machine systems, not enough intercept was acquired to allow for their successful or timely cryptanalysis. Depending on the systems’ complexity and potential for intelligence, some codes or ciphers would be exploited relatively quickly; others, because of their difficulty, took a great deal of time and resources to break. For example, the high-level German diplomatic systems, known as Floradora and GEE, took, respectively, four and five years for Allied codebreakers to solve. [60] A few machine cipher systems, such as the German Gestapo Enigma, known as TGD, defied Allied cryptanalysis completely during the war. [61]

The British had a large section in Hut 6 at Bletchley Park devoted to the decryption of the police ciphers. There were good technical, cryptologic reasons for this extensive effort. The texts of decrypted police messages sometimes provided cribs - plain text assumed to be present in enciphered messages - to those messages encrypted with the Enigma by other services such as the Wehrmacht. Also, many Wehrmacht andLuftwaffe operational units retained reserve manual ciphers similar to the police system in case of Enigma failure, or used them for messages of a less importance. [62] As for intelligence value, the police messages were important because they provided information on much more than the massacres of Jews and other groups in Russia. They contained information about economic, political, and social conditions in Germany and occupied Europe, the suppression of resistance groups in the conquered countries, the political situation in Italy after the fall of Mussolini, news of Allied POW escapes, the effects of Allied bombing on the morale of German civilians, and the SS plunder of art and other cultural items from other countries. [63]

For systems that had been broken, there was virtually no retroactive effort by the Allied cryptologists to solve so-called “back traffic.” For example, there was no plan to go back and read the high-level German diplomatic messages that had accumulated prior to the cryptanalytic breakthroughs in 1944 and 1945. In another case, by early 1944, the War Department’s Special Branch conceded that there was no way that some 200,000 intercepted Japanese military messages, encrypted and encoded in systems that were now exploitable, which had been stockpiled, would ever be read by the Army. Arlington Hall conceded that there was too much current intercept arriving from the field and just keeping up with that flow negated any chance for work on the old messages. Also, the Army’s analysts had to stay on top of the current systems in case the Japanese changed them. The author of a memorandum reviewing this situation at Arlington Hall could only muse about how much “priceless intelligence” was lost. [64]

How much of the total intercept ultimately was exploited cryptanalytically is not known. There is no quantitative data about how many possible messages may have been missed (or copied too poorly for use by the code-breaking sections). There is not much data about intercepted messages encoded or encrypted in cryptographic systems that were unbreakable. Likewise, there is little statistical information available to determine an overall success rate for Allied codebreakers. The few records with statistics concern how many messages were intercepted and how many were broken. If the incident about the Japanese Army messages related in the preceding paragraph can be used an indicator, then it is possible that a substantial number of messages were intercepted but not exploited. In reality, the general rate of successful decryption or decoding of intercepted messages may not have been particularly high, and this estimate includes messages that were partially exploited. For example, in fiscal year 1944, the Allies intercepted about 576,000 Axis and neutral diplomatic messages. The SSA cryptanalytic branch at Arlington Hall, B-III, managed to solve about 89,000, or a rate of slightly less than 15.5 percent.

This low rate, though, does not mean that the Allied codebreakers failed to produce intelligence. For one thing, the priorities established by the Allied staffs ensured that the communications most important to the war’s prosecution received the most coverage and analytic attention. Axis military systems, such as U-boat Enigma traffic, were exploited in much larger numbers and a greater rate relative to intercept than targets of lower inter- est such as neutral diplomatic networks. [65] As for the diplomatic problem, the major targets of interest, Japanese embassies in Axis and neutral capitals were the highest priority and accounted for approximately 54 percent of all diplomatic translations. [66] Success for the Allied cryptologic effort must be measured in the intelligence gathered that affected the course and outcome of the war. Still, the general result of the codebreaking was that a relatively small percentage of the intercepted messages were available for the next set of analysts- the translators.

Translations of the decrypted high-level Axis and neutral messages were done at Huts 3 (military and air) or 4 (naval) at Bletchley Park, at Berkeley Street in London (diplomatic and commercial), and at the B-III section of Arlington Hall Station. A translation took a few steps to complete. Initially, a linguist would complete a worksheet. Usually this took the form of the decrypted text with a word-to- word translation. Next, the linguist would compose a draft English text based on the worksheet. A final version of the text would then be written. Usually this form carried any comments considered necessary for the reader of the translation. For example, if a particular passage proved to be difficult and open to other interpretations, that bit of text in the original language would be included in a footnote.

The translation problem facing the cryptologic agencies was enormous. The nature, size, and logistical needs of the language problem proved to be as daunting as that of decryption. The target countries, Axis, minor allied, and neutral, used three- dozen languages. The mixture was an almost devilish variety that ranged from relatively familiar European languages, such as French, German, and Spanish, which were taught at many American and British schools, to the unfamiliar and rarely taught Amharic, Arabic, and Thai. [67]

At the beginning of the war, one of the major issues confronting the Allied cryptologic organizations was that they had to produce a large enough cadre of linguists to handle the range of languages found in the decrypted messages. This early lack of translators created a bottleneck in the processing of intercept. For example, during the month of March 1943, SIS had received over 114,000 intercepts, mostly Japanese Army, but also weather and diplomatic messages. Yet, Arlington Hall could produce only 4,500 translations. The problem was a lack of translators to handle the load. [68]

Once training programs were running and producing linguists, other problems intruded. Again, wartime priorities often defined the direction of the linguistic training effort. For the Americans, Japanese was the language emphasized in its training programs. This emphasis was driven partly by the division of effort with the British, though the latter had their own Japanese training program. Other languages of interest were not ignored, but American linguistic training was dominated by the needs of the war against Japan. Besides the dominance of Japanese language needs, another problem was other organizations that competed with the cryptologic agencies for these rare linguists. The most important was the Allied Translator and Interpreter Section (ATIS), which performed critical roles such as POW interrogation and translation of captured enemy documents in all combat theaters.

Once the linguists were assigned to analytic centers, their work was supposed to be verified at various steps for correctness and readability. Nonetheless, the quality of translations the cryptologic agencies produced during the war varied. In the archival record collections, researchers easily can find numerous excellent translations, and many with serious shortcomings in grammar, syntax, and vocabulary. Many of the problems probably arose from technical gaps in linguistic resources. Linguists not only had to become proficient in a language, as they went along in their work they had to learn or develop specialized glossaries for military, nautical, and aerial jargon, equipment references, and so on. [69] The text of diplomatic messages often contained exactingly precise language that was difficult to render into equivalent English. Many diplomatic messages reported conversations and interviews with individuals that had been translated from the language of the host country. Also, messages were composed with a background context that assumed a prior understanding between original sender and intended recipient. If the linguist had not seen any previous messages, allusions to other individuals, issues, and the significance of topics in the text could be opaque to an Allied translator. For the recipients of the translations, though, these shortcomings may have not always posed a problem: the main ends to translation were availability and utility, not elegance and erudition.

The rapidity at which a translation could be completed varied, as well. The above difficulties affected the turnover from decrypt to translation. Further, if an intercept was considered important, and the decryption could be done quickly, it would be translated relatively quickly. If not, it could be some time before a translation was completed and disseminated, perhaps weeks or even months. An example of this difference can be illustrated by two diplomatic intercepts of late June 1944 from Budapest. Both messages had been received at Arlington Hall one day after being intercepted by the U.S. Army’s monitoring station in Asmara, Ethiopia (MS-4). The first intercept was a 26 June 1944 Hungarian diplomatic message to Ankara, Turkey, that reported world reaction to the initial roundup of Hungarian Jews. It took Arlington Hall from 27 June to 16 December 1944 to complete its decryption and a formal translation to be issued - almost six months. The second intercept was of a Vichy diplomatic message from Budapest to Ankara, Turkey, that reported the situation of a group of Lebanese Jews trapped in Budapest. It took Arlington Hall from 13 June to 24 June to decrypt and translate this message, or only eleven days. [70]

The specific reason (or reasons) for the long delay in issuing the translation of the Hungarian intercept is not known. The quick processing of the Vichy intercept probably can be explained by the long familiarity of Arlington Hall analysts with Vichy diplomatic cryptographic systems that had been solved about two years earlier, as well as the comparatively easier French plain text. The delay in processing the Hungarian intercept suggests more difficulty with Hungarian cryptography, as well as problems with the Magyar language. The long processing time for the Hungarian intercept does not mean that information of intelligence value it may have contained would have been held back until a full translation was done. In a heat of operational necessity, intelligence could be passed informally.

At the beginning of the war, the United States cryptologic agencies were producing only a few separate series of translations, notably the diplomatic series by the SIS and Japanese Navy translations by OP-20-G. These early series of translations suffered from a number of limitations. There were too few being written; they were not timely; and the translations suffered from missing portions of text. As the war progressed, both army and navy code-breakers got better at breaking Axis and neutral cryptographic systems, and began to exploit more systems. As the number and variety of readable intercepts increased, the number of resulting translations also grew.

The initial translation series proved to be inadequate to handle the growing flood of intelligence. The analytic centers created additional specialized translation series that were categorized by topic, source, or administrative needs. By the end of the war, the U.S. cryptologic elements were producing over sixteen separate series of translations. Some were for a general intelligence audience, such as the Japanese military translations. Others were for the cryptologic elements themselves. These series usually carried the notation “CI” for “Code Instruction.” Most of the translation series had no or very little information about the Holocaust beyond a passing reference. The two that contained the most information were the “H” (Multinational Diplomatic) and the “T” (Reserved, or Restricted Diplomatic) series. [71] These two series are discussed in some detail in Chapter 3, pages 61 to 64. The British created fewer translation series compared to those of the SIS.

Although a precise accounting of the final tally of finished translations produced by the Western cryptologic agencies is beyond this study, it should be pointed out that the intercepts that were translated represented only a part of what previously had been decrypted, which, in turn, was a portion of what originally had been intercepted. Even by the end of the war, the overall rate on some targets was still relatively low. In July 1945, an Inspector General survey of OP-20-G activities indicated that only 10 percent of all intercepted Japanese naval messages were being fully processed and disseminated to the various military commands. [72] Recall that the number of decrypts produced by B-III in 1944 was 89,000 (out of 576,000 intercepts, or 15 percent); the estimated number of translations for the same period is about 50,000. This represents about 56 percent of the decrypts, but only 8.6 percent of all diplomatic intercepts. [73]

The 1944 American total of diplomatic translations issued is surprisingly close to that of GC&CS at the start of the conflict. In 1940 the Diplomatic and Commercial Section of GC&CS reported that it had received (intercepted by radio or cable, or acquired through espionage) 100,000 telegrams, read 70,000 of them, but circulated only 8,000. The British numbers also represent another aspect of this problem mentioned earlier: the relative dearth of intelligence of importance that was contained in the intercepted messages. The low figure of translations disseminated through the British government, some 8 percent of all intercepts despite a 70 percent success rate of decryption, suggests that many messages that London intercepted contained little intelligence of importance to the prosecution of the war. The intercepts were largely of minor neutrals and allied governments in exile. [74]

Once the translations were completed, other intelligence analysts reviewed them for usable information. At Hut 3, this work was done in the indexing sections. These individuals, many of them women, extracted those elements of intelligence that contributed to the larger information matrixes that were important to the war’s prosecution - order of battle, equipment listings, names of individual officers, etc. At Hut 3, a senior indexer would flag the information that needed to be entered into large index card catalogues for further use. The information was entered onto index cards, crossreferenced, and stored in cabinets within Hut 3 for future reference. [75]

No information was overlooked. Even that data that appeared innocuous and subsequently not important by the Axis, could reap benefits from good indexing. For example, prior to the war, the British intercepted messages from the Italian Air Force in North Africa. The messages contained seemingly mundane information: specifics about repairs of the engines to Italian aircraft. Many of the messages listed engines by serial number. Alert British analysts compiled these serial numbers. The payoff was that, by knowing the status of almost every aircraft engine, the British could develop a complete picture of the size, composition, and availability of aircraft to the Italian Air Force in North Africa when hostilities began in September 1940. [76]

Notes

[52] Stephen Budiansky, Battle of Wits: The Complete Story of Codebreaking in World War II (New York: The Free Press, 2000), 286.

[53] “Cryptanalytic Short Titles,” NARA, RG 457, HCC, Box 941, Folder 2740; “Tentative Trigraphic List of Cryptanalytic Short Titles,” Box 1370, Folder 4289; Frank B. Rowlett, The Story of Magic: Memoirs of an American Cryptologic Pioneer (Laguna Hills, CA: Aegean Park Press, 1998), 253-5.

[54] Hinsley, Vol. 2, Appendix 4, 656-668.

[55] Rowlett, Ibid.; “Cryptanalytic Short Titles.” “Cryptanalytic Branch Annual Report for Fiscal Year 1944,” NARA, RG 457, HCC, Box 1116, Folder 3567.

[56] “History of the Solution of Vatican Systems in SSA and GCCS, 1943-44.” (Washington: September 1944) RG 457, HCC, Box 1284.
[57] For example, it took Frank Rowlett’s team in SIS from April 1939 to September 1940 to break Purple. GC&CS codebreakers took from September 1939 to July 1940 to master the Luftwaffe’s Enigma, and until June 1941 to exploit the Kriegsmarine version of Engima.

[58] For an explanation of the double playfair cipher, see “Study of German Police Traffic,” NARA, RG 457, HCC, Box 202.

[59] For a discussion of the relative difficulty of both systems, see William F. Friedman and Lambros Callimahos, Military Cryptanalytics Part I (Washington, DC: National Security Agency, 1956), 181- 188, about Playfair systems. See Lambros Callimahos and William F. Friedman, Military Cryptanalytics, II (Washington, DC: National Security Agency, 1959), 431- 433, for transposition systems.

[60] For the work against the German systems, see Budiansky, 218-220 and 310-11; Alvarez, 234. Also see NARA RG 457, HCC, Box 942, Folder 2746, 11 May 1945, “GEE Problem”; Box 1317, Folder 3945, 1944, “History of Special German Diplomatic Net, Summer 1942 to Winter 1944.”

[61] Nigel West. The SIGINT Secrets: The Signals Intelligence War, 1900 to Today (New York: William Morrow, 1988), 307. See Alvarez, 176, for the Allied failure against Swedish systems. Hinsley, Vol. 2, Appendix 4, 656-668, for a summary of success against all German systems.

[62] Hinsley, Vol. II, Appendix 5, 670-2.

[63] Phillips, 7-11.

[ 64] “Memorandum for Colonel Clarke,” 3 January 1944. James L. Gilbert and John P. Finnegan, U.S. Army Signals Intelligence in World War II. A Documentary History (Washington, DC: Center of Military History, 1993), 63-4.

[65] According to one historical review conducted by the U.S. Navy, some 300,000 German Navy messages were “read,” meaning they were decrypted by the Americans or Commonwealth codebreakers. Of these, about 69,000 were German U-boat messages. About 49,600 U-boat translations were published between early 1941 to June 1945 for a rate of around 72 percent. These translations are notated “SRGN” and are available at NARA, RG 457, Entry 9019 and at the library of the National Cryptologic Museum. “Historical Review of OP-20-G, RS #77967, 8 October 1945, untitled back- ground notes, Author’s private collection.

[66] This is a rough calculation determined by comparing the number of translations of the Japanese diplomatic collection at NARA, Entry 9011 (SRDJ), with that of the Multinational Diplomatic collection, Entry HCC, Boxes 286-516.

[67] Rowlett, 253-5; Contrary to popular opinion, the Vatican messages were in Italian, not Latin. Switzerland had diplomatic codes for German and French texts to account for that nation’s major language divisions.

[68] Memorandum for Colonel Carter W. Clarke, Subject: “Origins, Functions, and Problems of the Special Branch, MIS.” 15 April 1943, Gilbert and Finnegan, 55.

[69] For example, see NARA, RG 457, Box 590, for “Glossary of Japanese terms,” and Box 619 for “Glossary of German Army Equipments.” For German Police messages, Boxes 202 and 1386 contain glossaries of abbreviations and shorthand terms used by the Police in their messages.

[70] MND Translations, Budapest to Ankara, 26 June 1944, H-157087, RG 457, HCC, Box 455; and Budapest (Charmasse) to Ankara (AmbaFrance), 12 June 1944, SIS #127775. RG 457, HCC, Box 424

[71] Briefly, these series are: A – “Sensitive Diplomatic Traffic,” Baker Cables - Translations exchanged with Britain, CG - Coast Guard translations of Axis illicit messages, CI/AI/GI - “Code Instructions,” D - Japanese Military Attaché translations, F - Japanese Air Force, FT - “File Texts”, J - Japanese military, JR - Japanese Water Transport, H- Multinational Diplomatic, L - Military Attaché other than Japan, M - German traffic, not further identified, MC - Multinational (or Miscellaneous) Code Instructions, and T - “Restricted Diplomatic.” Interestingly, there was no special designator for the Japanese Navy translations produced by OP-20-G.

[72] From: Naval Inspector General, To: Secretary of the Navy. Serial 001971, “Survey of OP-20-G Section of Naval Communications Division of the Bureau of Naval Personnel which Procures Uniformed Naval Personnel.” 13 July 1945. RG 457, HCC, Box 1286. Item 23 (d)

[73] Rowlett, 253-4

[74] Non-notated and unsigned memorandum on diplomatic cryptanalysis for 1940, 31 January 1941, HW 14/11, “Government Code and Cypher School: Directorate: Second World War Policy Papers,” as cited in John Ferris, “The Road to Bletchley Park: The British Experience with Signals Intelligence.” Intelligence and National Security (Vol. 17, No. 1, Spring 2002), 53-84.

[75] Peter Calvocoressi, Top Secret Ultra (New York: Pantheon, 1980), 61-66.

[76] J.E.S. Cooper, “Lecture on SIGINT History,” September 1961. Author’s copy. Also available at PRO HW 3, Naval Intelligence Division and Successors: History of UK Signals Intelligence, 1914-1925.

Continue to Part 8