Unit X — How A Band Of Tech Mavericks Bridged The 'Valley Of Death' Between Innovation And The Pentagon

Unit X: How the Pentagon and Silicon Valley Are Transforming the Future of War. Raj M Shah and Christopher Kirchhoff. Scribner. 319 pages. Rs 999.

In late 2000, the Central Intelligence Agency (CIA) was focused on an escalating threat. Al-Qaeda’s attacks on United States (US) embassies in Kenya and Tanzania had left hundreds dead, and there was a growing concern that more attacks were imminent.

In response, the agency launched "Afghan Eyes," a 60-day trial run of Predator drones over Afghanistan to track down Osama bin Laden, the man believed to be behind the embassy bombings.

A few days later, a Predator drone, launched from a base in Uzbekistan, hovered over Tarnak Farm, a large compound outside Kandahar, Afghanistan.

Inside the Ground Control Station at CIA headquarters in Langley, the drone's pilot, Scott Swanson, sat in front of his screens, focused on the feed coming in from the drone. His sensor operator, Master Sergeant Jeff Guay, was beside him, working the camera, slowly scanning the large area below. They had been looking for activity for hours.

Then, a figure appeared in the courtyard: A tall man in white robes, flanked by armed men who carried themselves with the deference of devoted followers and the discipline of loyal bodyguards.

The moment was unmistakable. Swanson later recalled, “Jeff and I immediately knew we had bin Laden in our sights.”

There he was — the man the US had been hunting, captured live on their screen.

They tracked him, following every move as he walked across the compound and disappeared into a building. But as the feed continued to roll, reality set in.

The Predator drone, while a marvel of surveillance, was unarmed. There was nothing more they could do. “Watching was all we could do,” Swanson wrote later.

A Predator drone would go on to fire a Hellfire missile for the first time next year. James "Snake" Clark, a former F-4 pilot, figured out how to mount the missile on the drone in just 61 days. Bureaucratic hurdles had slowed him down. Had they achieved the capability a few weeks sooner, the course of history might have been altered.

When Raj M Shah, an entrepreneur and a former F-16 pilot, and Christopher Kirchhoff, Special Assistant to the Chairman of the Joint Chiefs of Staff, met Clark in 2016, they had just joined the Defense Innovation Unit Experimental, also known as DIUx, which had been started in 2015 by Ash Carter, who had recently become the Secretary of Defense.

DIUx

DIUx emerged from the need to bridge the gap between Silicon Valley's innovative, fast-paced technology ("tech") development and the US military's need for tech solutions. The military was not leveraging commercial innovation to solve its problems, and the Pentagon's slow-moving bureaucracy, rigid procurement processes, and risk-averse culture wasn't helping.

The Silicon Valley and the US military were deeply sceptical of each other. There were doubts in the Silicon Valley about the government's reliability as a customer, rooted in a long history of frustration with the bureaucracy.

On the other hand, the military leadership saw Silicon Valley programmers as pampered and out of touch with the harsh realities faced by soldiers on the ground. They were outraged that tech companies were willing to sell to the Chinese Communist Party but not to the US Department of Defense.

Carter believed the growing gap was eroding the US military's superiority and needed to be bridged. However, with the entrenched bureaucracy in Washington holding sway over every decision, it was clear that this would not be an easy task for DIUx.

In their book Unit X: How the Pentagon and Silicon Valley Are Transforming the Future of War, Shah and Kirchhoff recount the two things Clark told them when they met him: “Shoot for the moon. And know everyone will come after you for doing so.”

Clark had seen the hurdles firsthand, and he wasn't wrong.

'Effed Up Beyond All Repair'

One of the book’s most striking moments unfolds at the Combined Air Operations Center (CAOC) in Al Udeid Air Base, Qatar.

In 2016, just months into his tenure at DIUx, Shah walks into CAOC, the nerve centre of US air ops across a vast region, stretching from North East Africa through the Middle East to Central and South Asia, an area larger than the entire continental US.

It was a place Shah describes as looking “like NASA Mission Control, with giant screens on the walls and a few hundred people sitting at rows of desks, tracking hundreds of fighter jets, tankers, drones, and AWACS planes.”

But the shine wears off quick. “On closer inspection,” Shah writes, “the CAOC turned out to be not so cutting edge. In fact, its technology was woefully obsolete.”

Systems held together with “duct tape and Band-Aids” were slowing everything down and, as Shah writes in the book, "putting lives at risk.”

The CAOC was juggling 26,000 strike sorties in 2016 — 70 a day — hammering ISIS across Iraq, Syria, and Afghanistan, with Mosul hit every eight minutes at peak. It took 18,000 tanker sorties and 100,000 refuelling to pump 200 million gallons of fuel into the sky that year.

When plans went sideways — bad weather, change in mission plans — spare tankers scrambled at $250,000 a pop, two to three times a day, racking up over $5 million a week in waste. The stakes were sky-high — troops on the ground, civilians, like the 700,000 trapped in Mosul, all counting on air support to beat the enemy.

Managing all of this were airmen equipped with whiteboards and Excel sheets. One called out numbers, another entered them into a laptop, and a third double-checked the calculations. Strip away the laptops, and the scene could have belonged to a Second World War-era London bunker, where members of the Women’s Auxiliary Air Force moved markers across a map to track air operations during the Battle of Britain.

It chewed up 60 man-hours daily. General Jeffrey Harrigian, who was running the show, was fed up. A $745 million tech overhaul of the entire setup at the base by Northrop Grumman was floundering spectacularly. The programme was in its eighth year, three years behind schedule, $371 million over budget, and the US Air Force was going to ask the Congress for $65 million in additional funding to keep the project running.

Former Google chief executive officer (CEO) Eric Schmidt, who was also at the base with Shah, recognised the situation as what the military would call "FUBAR" — effed up beyond all repair.

"Eric also knew how quickly a team in Silicon Valley could build a solution," Shah and Kirchhoff write, adding that the "team was already scoping out the app" by the time they returned to the US and went back to their Mountain View office.

Shah spotted an opportunity and seized it. DIUx would ensure that the CAOC got an app to do the job that was being done with whiteboards and Excel sheets.

Along with a handful of active-duty airmen who were skilled software engineers and help from a private sector firm, Shah and Kirchhoff were able to deliver a working product in just 132 days.

The tool could match a fighter with the perfect tanker in less than a minute, and, if anything changed, they could recalculate the entire schedule with just a press of a button.

The best part was that DIUx had managed to deliver this app for just $1.5 million. The project paid for itself in just three days of air operations, and it boosted efficiency by 10 per cent, saving a staggering 25 million gallons of jet fuel each year.

How DIUx pulled together the right team, sneaked private sector experts and their ever-beloved Macs into an active Air Force operations centre, and built the app in just over four months is a tale of imagination, grit, and a touch of mischief — all told in fascinating detail in the book.

Raj, ever the Star Wars fan — something this writer relates to all too well — saw DIUx as part of the Rebel Alliance. "When we delivered the tanker app, we felt like Luke Skywalker dropping a proton torpedo into an exhaust port of the Death Star. But the exhilaration was short-lived, and we now found ourselves at the remote edge of the galaxy on a planet covered in ice and snow," Shah and Kirchhoff write in the book.

DIUx set its sights on the full overhaul next and secured the project, ultimately delivering it to the US Air Force in 2022. They did it with help from a team they named "Kessel Run," a reference to a legendary hyperspace route in Star Wars — a dangerous smuggling corridor littered with black holes and gravitational anomalies.

In A New Hope, Han Solo famously brags that his ship, the Millennium Falcon, made the Kessel Run in "less than twelve parsecs," implying he found a shorter, more efficient route through treacherous space.

The name was a fitting choice for a team tasked with cutting through bureaucratic deadweight and modernising outdated military systems at lightning speed.

Revenge of the Paper Pushers

If the initial launch of DIUx was intended to emulate the Rebel Alliance's bold defiance against the Empire's rigidity, it instead mirrored the beleaguered state of the rebels at the onset of The Empire Strikes Back — beset by setbacks and struggling to gain momentum.

Shah and Kirchhoff had been tapped to run DIUx only after the first iteration of the initiative failed to take off in 2015 due to the Pentagon's bureaucratic red tape and inflexibility, and the lack of imagination on the part of those elected to run it, leaving Carter embarrassed.

"Upon arriving in Silicon Valley, the Pentagon team botched the initial setup so badly that DIUx couldn’t even get furniture or a working Internet connection. For six months the first team worked at folding card tables and used 4G hotspots bought at Best Buy," Shah and Kirchhoff write.

When Carter realised things were going off track, he sent Todd Park, President Barak Obama’s Chief Technology Advisor, to investigate. Park gave a blunt assessment of the situation upon his return — “It’s f*****.”

When it was first launched, Carter had flown to the Silicon Valley and presented DIUx as a solution to the persistent issues companies faced when trying to work with the Pentagon: slow contracts, endless red tape, and unwilling bureaucrats. But after the initial failure, he had to return to the Silicon Valley, admit the project had fallen short of what he had promised, and attempt a fresh approach.

The failure of the first iteration had only deepened the scepticism in the Silicon Valley. And it wasn't going to get any easier anytime soon, as Shah and Kirchhoff, now in charge, were looking at another crisis unfold just as they began to fix the broken initiative.

"For reasons unknown, our $30 million budget for the upcoming fiscal year, which started in only four months, was reduced—to zero. In Washington they call this getting “zeroized.” It is the strongest way for the first branch of government, Congress, to tell the second branch, the Executive, to go to hell," Shah and Kirchhoff write.

After sleuthing around, Shah and Kirchhoff found that the decision to zeroize the DIUx budget had been made by two Congressional staffers on the House Appropriations Committee, whom they referred to as Evelyn and Ed. The US Congress controls every dollar the Executive spends on its programmes, with only about 20 staffers wielding power over the Pentagon’s $700 billion budget. This gives each staffer influence over roughly $35 billion, an extraordinary concentration of authority.

When Shah and Kirchhoff managed to meet the two staffers who had zeroized DIUx's budget, they found out it was petty score-settling and not a grand policy decision that led to it. While one of the staffers had killed the budget because she worked for a Congressman from Indiana and DIUx was not spending any money in Indiana, the other held a grudge from years ago because Secretary of Defence Carter had once refused his request to use an Air Force Gulfstream to take a Congressional staff delegation overseas.

"It looked like Ed and Evelyn had us in checkmate. It was time to make some calls. In the hallway, we escalated the impasse to top budget officials at the Pentagon and alerted the Secretary’s office to the trouble we were now in. But for the moment, backstabbing, score-settling, and small-mindedness had won the day," they write.

It would eventually take more than a few calls to find the money DIUx needed.

"This was our first glimpse of the empire’s counterattack and the battles we’d have to fight to see our mission through," Shah and Kirchhoff write.

They would soon learn how relentless the empire could be.

A Shortcut Around the 'Valley of Death'

The Pentagon’s procurement system, much like India’s, was a bureaucratic labyrinth. The Federal Acquisition Regulations (FAR), a sprawling 1,300-page behemoth governing every purchase, didn’t just slow things down — it ground them to a halt. Deals that should have taken weeks dragged on for 12 to 18 months, sometimes longer.

For Silicon Valley startups, often cash-strapped, such delays weren’t just frustrating; they were fatal. They had little patience for the Pentagon’s slow-moving bureaucracy, where deals took years to close, payments took even longer, and too many innovations failed to reach the battlefield. Too many cutting-edge products faded away in the Pentagon’s now Grand Canyon-sized 'Valley of Death,' lost in red tape and bureaucratic inertia.

"No startup CEO trying to book revenue before their next venture capital raise can wait for the earth to circle the sun twice," Shah and Kirchhoff write, adding, "We needed a new way."

Unbeknownst to them, a hard-nosed employee had already found a hack—a shortcut that bypassed the 'Valley of Death.'

When Lauren Dailey, a 29-year-old acquisitions manager, was transferred to the fledgling DIUx at Moffett Field in 2015 — long before Shah and Kirchhoff arrived — she found six colleagues working at card tables, tethered to hotspot Wi-Fi.

“We quickly found that a startup and a customer would talk, but nothing would happen because there was no easy way to test out a prototype,” Lauren recalls.

The system was built for buying submarines and aircraft carriers from the primes of the defence-industrial complex, not tech products from Silicon Valley’s nimble tech firms.

As Shah and Kirchhoff write, "Lauren was a Pentagon employee with the mind-set of a Silicon Valley growth hacker." While others studied the problem, she dissected the 2016 National Defense Authorisation Act, unearthing a loophole: an expanded Other Transactions Authority (OTA). This relic from the space race could sidestep the FAR, turning successful pilots into production contracts without the usual year-long slog.

The OTA was introduced during the space race in the 1950s and 1960s to help the National Aeronautics and Space Administration (NASA) and the Pentagon bypass rigid federal procurement rules.

“How come nobody else ever thought of this?” Shah, her new boss, asked. “I guess they didn’t look,” she shrugged, handing him a 20-page white paper.

Lauren’s hack would unlock $70 billion in tech buys over the next few years. For DIUx 2.0, it was a lifeline, delivering warfighter solutions before the fiscal clock ran out.

Disrupting the Battlefield With Commercial Innovation

Silicon Valley’s success is often credited to the spirit of free-market entrepreneurship — visionary founders tinkering in suburban California garages, building tech giants through sheer grit.

But the reality is more complex. In the post-war era, the US government played a crucial role, pouring billions into research and development (R&D) as part of its Cold War competition with the Soviet Union.

During the Cold War, the Pentagon and Silicon Valley moved in lockstep, their ambitions intertwined. The Valley wasn’t just a hub for consumer technology; it was an extension of the national security state.

Lockheed’s Missile and Space Division was the region’s largest employer until the 1980s, while companies like Fairchild Semiconductor supplied the silicon chips that powered Apollo rockets and intercontinental ballistic missiles.

Innovation thrived, but not in isolation — it was fuelled by defence dollars and a government in Washington, DC, eager to dominate the technological frontier.

But by the 1990s, the balance of power had shifted. The Cold War ended, defence budgets shrank, and the cutting edge of technology moved from government labs to venture-backed startups.

As Ash Carter observed in 2001, “tomorrow’s defence innovations will largely be derivatives of technology developed and marketed by commercial companies for commercial motives.” The Pentagon no longer set the pace — Silicon Valley did.

But while technology surged ahead, the Pentagon and its bureaucrats remained shackled to an older way of doing things. Procurement officials, trained to minimise risk, stuck to a rigid and bureaucratic system that favoured established defence contractors — the “primes” like Lockheed Martin and Raytheon — over the chaotic, fast-moving world of startups, where deals happened over handshakes and paper-work follows later, Shah and Kirchhoff write.

Large defence contracts took years to finalise, a timeframe utterly incompatible with a tech industry that thrived on speed and iteration.

The Pentagon still controlled vast resources, but it was losing the one thing that mattered most in modern war: agility. By 2019, US defence spending accounted for just over 3 per cent of global R&D. That same year, the combined market capitalisation of Amazon, Apple, Google, and Microsoft surpassed the entire US defence sector. While civilians used their smartphones to summon taxis and pay bills, many US military units were “running software programs older than the officers using them.”

Yet the problem wasn’t just slow bureaucracy; it was a cultural divide that had been widening for decades.

By the 2010s, the engineers shaping the Silicon Valley had come of age in the shadow of the Iraq War and its disastrous consequences. They had seen the justifications for war — claims of weapons of mass destruction — unravel into a quagmire of occupation, insurgency, and civilian casualties. Unlike the Cold War-era engineers who had eagerly built missile guidance systems for the Apollo programme and the Pentagon, this new generation of tech workers was sceptical of the military.

This mistrust only deepened after Edward Snowden’s 2013 revelations, which exposed the extent of the National Security Agency’s (NSA's) mass surveillance programmes.

Tech companies, once viewed as symbols of innovation and progress, were implicated in secret government data collection, leading to widespread backlash. Silicon Valley engineers, many of whom had joined the industry to build tools for connectivity and empowerment, suddenly found themselves linked to a surveillance apparatus they had neither consented to nor fully understood.

The result was an industry-wide reckoning: a growing unwillingness to collaborate with the US government, particularly in matters of defence and intelligence.

By the time Defense Secretary Ash Carter visited Silicon Valley in 2015, hoping to bridge the divide, the rift had hardened. It had been decades since a sitting defense secretary had made such an effort, but the reception was cold. Google, a company whose motto had once been, "Don't Be Evil," outright refused to let him onto its campus — a symbolic rejection of the military-industrial ties that had once defined the region.

The Pentagon, long accustomed to being the primary driver of technological innovation, now found itself struggling to even get a seat at the table in America’s most influential tech hub.The Pentagon had become an outsider in a world it once helped create.

Unit X tells the story of what happened next — the scramble to reconnect.

As historian Margaret O’Mara outlines in The Code: Silicon Valley and the Remaking of America, Cold War-era government R&D spending provided the foundational “blueprint” for Silicon Valley’s success, leading to a model where public investment accelerated private ingenuity. This dynamic, however, has undergone a striking reversal.

Today, as Shah and Kirchhoff argue in their book, it is commercial innovation that is increasingly transforming warfare. Nowhere is this shift more evident than in Ukraine, where Starlink internet terminals became a critical asset when Russian jamming rendered traditional military communications ineffective.

The episode underscores the growing dependence of state actors on private technological infrastructure — an issue now thrust into the spotlight as Elon Musk, amid a controversy of his own making, insists that he would never deactivate Starlink for Ukraine, even if he personally opposes their policies.

There's no doubt that conventional military tools — tanks and fighters and submarines and frigates would continue to dominate the battlefield, and Shah and Kirchhoff do not argue otherwise. But commercial innovation is increasingly shaping the battlefield in ways traditional defence planners did not anticipate, or were't aware could happen.

From artificial intelligence (AI)-driven intelligence analysis to autonomous drones and satellite communications, private-sector technology is filling critical gaps in military capability. In short, technology will enable war-fighters to deploy weapons systems with greater efficiency.

Shah and Kirchhoff highlight how software-defined warfare — where algorithms, connectivity, and rapid iteration matter as much as firepower — is redefining strategic advantage.

Unit X is ultimately a story of power — not just military power, but the power to shape the future of war. It reveals a Pentagon locked in a defining struggle, not against foreign adversaries, but against its own pace of change.

In a world where wars are no longer won by sheer force but by the speed of adaptation, the book shows how institutions built for the long arc of history are now being forced to think in the rapid cycles of software updates and venture capital.

It is about the collision of two cultures: one that values stability, control, and hierarchy, and another that thrives on disruption, iteration, and risk. The outcome of this collision will determine whether the US can maintain its military edge or if it will be outpaced by the adversary which is believed to have fewer constraints and greater agility — China.