Posts
There Are Spying Eyes
Everywhere—and Now They
Share a Brain
Security cameras. License plate readers. Smartphone trackers. Drones. We’re being watched 24/7. What happens when all those data streams fuse into one?
4 February, 2021
ONE AFTERNOON IN the fall of 2019, in a grand old office building near the Arc de Triomphe, I was buzzed through an unmarked door into a showroom for the future of surveillance. The space on the other side was dark and sleek, with a look somewhere between an Apple Store and a doomsday bunker. Along one wall, a grid of electronic devices glinted in the moody downlighting—automated license plate readers, Wi-Fi-enabled locks, boxy data processing units. I was here to meet Giovanni Gaccione, who runs the public safety division of a security technology company called Genetec. Headquartered in Montreal, the firm operates four of these “Experience Centers” around the world, where it peddles intelligence products to government officials. Genetec’s main sell here was software, and Gaccione had agreed to show me how it worked.
He led me first to a large monitor running a demo version of Citigraf, his division’s flagship product. The screen displayed a map of the East Side of Chicago. Around the edges were thumbnail-size video streams from neighborhood CCTV cameras. In one feed, a woman appeared to be unloading luggage from a car to the sidewalk. An alert popped up above her head: “ILLEGAL PARKING.” The map itself was scattered with color-coded icons—a house on fire, a gun, a pair of wrestling stick figures—each of which, Gaccione explained, corresponded to an unfolding emergency. He selected the stick figures, which denoted an assault, and a readout appeared onscreen with a few scant details drawn from the 911 dispatch center. At the bottom was a button marked “INVESTIGATE,” just begging to be clicked.
Citigraf was conceived in 2016, when the Chicago Police Department hired Genetec to solve a surveillance conundrum. Like other large law enforcement organizations around the country, the department had built up such an impressive arsenal of technologies for keeping tabs on citizens that it had reached the point of surveillance overload. To get a clear picture of an emergency in progress, officers often had to bushwhack through dozens of byzantine databases and feeds from far-flung sensors, including gunshot detectors, license plate readers, and public and private security cameras. This process of braiding together strands of information—“multi-intelligence fusion” is the technical term—was becoming too difficult. As one Chicago official put it, echoing a well-worn aphorism in surveillance circles, the city was “data-rich but information-poor.” What investigators needed was a tool that could cut a clean line through the labyrinth. What they needed was automated fusion.
Gaccione now demonstrated the concept in practice. He clicked “INVESTIGATE,” and Citigraf got to work on the reported assault. The software runs on what Genetec calls a “correlation engine,” a suite of algorithms that trawl through a city’s historical police records and live sensor feeds, looking for patterns and connections. Seconds later, a long list of possible leads appeared onscreen, including a lineup of individuals previously arrested in the neighborhood for violent crimes, the home addresses of parolees living nearby, a catalog of similar recent 911 calls, photographs and license plate numbers of vehicles that had been detected speeding away from the scene, and video feeds from any cameras that might have picked up evidence of the crime itself, including those mounted on passing buses and trains. More than enough information, in other words, for an officer to respond to that original 911 call with a nearly telepathic sense of what has just unfolded.
Gaccione turned to a second console, this one loaded with a program called Valcri. Where Citigraf is designed for relaying early leads to patrol officers rushing to the scene of a crime, Valcri is for the detectives working long cases at the precinct. Originally developed to root out sex-trafficking rings, its fusion algorithms hunt for subtler, more elaborate patterns that might stretch across years of unstructured data. Gaccione told me about one counterterrorism unit, which he wouldn’t name, that had used the system to build a detailed profile of “a middle-aged unemployed individual with signs of radicalization,” using “various databases, CCTV, phone records, banking transactions, and other surveillance methods.” If done manually, he estimated, this kind of investigatory grunt work would take a couple of weeks. In this instance, it took “less than a day.”
The market for fusion technology has been enjoying a quiet boom in recent years. Genetec says that Citigraf is deployed in “many cities.” A growing number of established tech giants, including Cisco, Microsoft, and Motorola, sell fusion systems globally, often in the guise of “smart city” modernization packages. (Cisco sometimes even sweetens the pot with no-interest financing.) Palantir, which bills itself as a “data integration” firm, reportedly counts among its clients the Central Intelligence Agency, Immigration and Customs Enforcement, and the Centers for Disease Control and Prevention. Anduril has built a “virtual wall” along parts of the border with Mexico, using fusion software to link together a network of surveillance towers. Last fall, the four-year-old company won a flexible contract, capped at $950 million, to contribute elements of the technology to the US military’s Advanced Battle Management System.
For all these customers, a central appeal of fusion is that it can scale to new sources of data. You can add fuel to your “correlation engine” by, say, hooking up a new network of sensors or acquiring a privately owned library of smartphone location data. (The Pentagon’s Special Operations Command was recently revealed to be a buyer of many such libraries, including those from a Muslim prayer app with tens of millions of users.) Organizations with their own coders can develop capabilities in-house. In New York, for instance, the police department’s analytics division created a custom plug-in for its fusion system. The feature, called Patternizr, draws on more than a decade’s worth of departmental data to match property crimes that could be related to each other. When a new report comes in, all the investigator has to do is click “Patternize,” and the system will return a list of previous incidents, scored and ranked by similarity.
Mind-bending new breakthroughs in sensor technology get a lot of buzzy press: A laser that can covertly identify you from two football fields away by measuring your heartbeat. A hack that makes your smartphone spy on anything nearby with a Bluetooth connection, from your Fitbit to your smart refrigerator. A computer vision system that will let the authorities know if you suddenly break into a run within sight of a CCTV camera. But it’s a mistake to focus our dread on each of these tools individually. In many places across the world, they’re all inputs for a system that, with each new plug-in, reaches a little closer to omniscience.
That idea—of an ever-expanding, all-knowing surveillance platform—used to be a technologist’s fantasy, like the hoverbike or the jetpack. To understand how this particular hoverbike will finally be built, I began by calling up the people who designed the prototype.
THE DEPARTMENT OF Defense was among the first organizations to face large-scale surveillance overload. By the decade after September 11, its arsenal of spy technologies had grown to galactic proportions. The department had experimented with computerized fusion since at least the 1970s, but the most advanced systems still couldn’t handle more than two or three data inputs. A modern intelligence unit had to contend with hundreds. According to Erik Lin-Greenberg, who ran an elite fusion team for the Air Force from 2010 to 2013, the old ways still ruled. Each human analyst was typically responsible for a single data stream. They compared their findings in chats and phone calls, or sometimes by yelling to one another across the room. In one case, Lin-Greenberg said, another team in his squadron identified an IED just in time to halt a convoy less than 500 feet up the road.
One of the people who was supposed to help fix intractable problems like this was Dan Kaufman, the director of information innovation at the Defense Advanced Research Projects Agency, the Pentagon’s storied R&D hub. With his sunny manner and bowl of shimmering silver hair, Kaufman wasn’t cut from the camo-speckled cloth of the typical military-industrial denizen. In his previous life, he had run the video game developer DreamWorks Interactive, where he helped launch what would become the Medal of Honor series. Later, as a consultant, he had worked with the CIA’s venture capital fund, In-Q-Tel. At Darpa, Kaufman was known for championing complex computing projects with a distinct commercial flavor. He felt that the Pentagon’s fusion efforts were due for a shake-up.
In the winter of 2010, Kaufman was introduced to Ben Cutler, an experienced engineer and tech entrepreneur who was considering a tour of duty in government. Over the phone, Kaufman explained the problem to Cutler and outlined his vision for what to do about it: He wanted a software platform that could integrate all available intelligence in a single, consolidated interface and grow as new capabilities came online. For Cutler, who had spent the previous year working on a new operating system at Microsoft, the idea clicked right away. What the Pentagon needed, he realized, was an OS for surveillance.
Cutler was intrigued enough to write a pitch. The document, which he completed in a day, opens with a theatrical flourish: “A patrolling group of soldiers pursues a pickup into a village; it stops at a mosque.” At this point, in real life, the soldiers might have to wait for an old-fashioned fusion team to deliver its assessment. But in Cutler’s scenario, they would simply log their geographic coordinates and the pickup’s license plate number into a tablet. The operating system would then return a description of the neighborhood around the mosque (“known insurgent meeting area”), a profile of the imam (“has worked well with friendly forces”), and any records connecting the vehicle with known terrorist groups.
Weeks later, Cutler was offered the job. I asked him whether he believed at the time that he had the expertise to build what he had pitched. “No!” he said, laughing a bit wildly.
To be fair, nobody did. The project, officially called Insight, would depend on a science fiction novel’s worth of technical breakthroughs. A study commissioned the previous year by the National Geospatial-Intelligence Agency had concluded that many of the features that Darpa was now proposing were still far from feasible
Read the rest of the article HERE
