Introduction/Synopsis
Big Data is driving virtually every industry in today’s Digital Age, including, among all others, the work of the Intelligence community. From human intelligence (HUMINT) to open source intelligence (OSINT), “strategic intelligence as a professional discipline and force multiplier” has evolved from centering on “qualitative subject-matter content analyzed by human specialists” to leveraging “the increasingly massive collection and machine analysis of quantifiable, if not necessarily quantitative, data.”[footnoteRef:2] However, as Richard Best, Specialist in National Defense, Foreign Affairs, Defense and Trade Division, testified before Congress in 2006, OSINT, which is primarily derived from old and new media publications, including social media, lacks a systematic manner in which such data can be incorporated into or used to supplement classified information.[footnoteRef:3] As Best notes, “a consensus now exists that OSINT must be systematically collected and should constitute an essential component of analytical products.”[footnoteRef:4] The main challenge is that among the community itself, a systematic approach has not yet been adopted and the Director of National Intelligence has not taken sufficient steps to exploit OSINT to its maximum potential. How to better incorporate OSINT into strategic decision making within the intelligence community is a major issue that has to be discussed in more detail so that U.S. intelligence can be better fortified by making the utmost use of all available information. [2: Kevjn Lim, "Big data and strategic intelligence." Intelligence and National Security 31, no. 4 (2016), 620.] [3: Richard Best, Open Source Intelligence (OSINT): Issues for Congress, 2007, 2.] [4: Ibid 2.]

Background/Problems

While OSINT and HUMINT are recognized as the “most democratic intelligence disciplines,”[footnoteRef:5] they are also the most common. The fact that virtually every intelligence agency collects OSINT, however, means that there one of the major problems surrounding OSINT is tied to “duplicative efforts, wasted resources, and often budget competition that promoted information hoarding rather than information sharing.”[footnoteRef:6] There is also another challenge with respect to OSINT. As OSINT is the source of first resort (since it is so readily and broadly available), OSINT has a long history in intelligence gathering. The collection of this data, though, must be legal and lawful—and with so much data now being posted and shared via social media, a gray line exists as to whether such information is public or private and whether intelligence agencies should have lawful access to social media...…their private information involves a degree of cyber HUMINT, which could lead to overlap and role confusion among intelligence officers; 3) to process Big Data effectively, OSINT officers must also work closely with IT developers to find effective ways of processing OSINT and distilling from the pool of Big Data the most essential sequences of information—which is easier said than done. As Krasner and Zegart show, “the national security landscape for the foreseeable future will be marked by unprecedented uncertainty. New threats are emerging and old threats are evolving at speeds unknown in earlier eras.”[footnoteRef:10] To get the most out of OSINT, new approaches and methods must constantly be employed, and this could absorb a great deal of time, energy and resources. [10: Amy Zegart and Stephen D. Krasner, eds., “Pragmatic Engagement amidst Global Uncertainty: Three Major Challenges,” Hoover Institution, 2015, 12]

Conclusion: Optimal Solution

The best solution going forward is to combine OSINT with HUMINT when searching social media for information and to develop an approach to OSINT that the intelligence community overall can agree to share. IT developers should also be sought to help develop a better method of processing Big Data so as to facilitate OSINT collection.…