절대 중국산 전자제품 사용하면 안되는 이유..."해킹 칩 숨겨져 있어" The Big Hack: How China Used a Tiny Chip to Infiltrate U.S. Companies
The Big Hack: How China Used a Tiny Chip to Infiltrate U.S. Companies
The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.
In 2015, Amazon.com Inc. began quietly evaluating a startup called Elemental Technologies, a potential acquisition to help with a major expansion of its streaming video service, known today as Amazon Prime Video. Based in Portland, Ore., Elemental made software for compressing massive video files and formatting them for different devices. Its technology had helped stream the Olympic Games online, communicate with the International Space Station, and funnel drone footage to the Central Intelligence Agency. Elemental’s national security contracts weren’t the main reason for the proposed acquisition, but they fit nicely with Amazon’s government businesses, such as the highly secure cloud that Amazon Web Services (AWS) was building for the CIA.
절대 중국산 전자제품 사용하면 안되는 이유..."해킹 칩 숨겨져 있어" 중국은 미국 회사에 침투하기 위해 어떻게 초소형칩을 사용했나 블룸버그 정부 및 기업 소식통과의 광범위한 인터뷰에 따르면, 중국 스파이들의 미국의 기술 공급망 손상이 아마존과 애플을 포함한 거의 30개의 미국 회사들에 이르렀다고 한다. 2015년 아마존은 현재 아마존 프라임비디오로 알려진 스트리밍 비디오 서비스의 대규모 확장을 지원할 수 있는 잠재적인 인수인 엘리멘탈 테크놀러지(Elemental Technologies)라는 신생기업을 조용히 평가하기 시작했다. 오레곤 포틀랜드(Ore, Portland)에 기반을 둔 엘리멘탈은 대용량 비디오 파일을 압축하고 다른 장치에 맞게 포맷하기 위한 소프트웨어를 만들었다. 그것의 기술은 올림픽 게임을 온라인으로 스트리밍하고 국제우주정거장과 의사소통하는 것을 도왔다. 그 과정에 정통한 한 사람에 따르면, 인수를 감독하던 AWS는 실사를 돕기 위해 제3의 회사를 고용하여 보안을 면밀히 조사했다고 한다. 첫 번째 통과에서는 문제가 발견되어 AWS가 엘리멘탈의 주요 제품, 즉 고객이 비디오 압축을 처리하기 위해 네트워크에 설치한 값비싼 서버를 자세히 살펴보게 되었다. 서버 마더보드에서 쌀 한 알 크기만한 작은 마이크로칩을 발견했다 이는 IASB와 FASB의 원래 디자인의 일부가 아니었다. 아마존은 이 발견을 미국 당국에 보고하면서, 정보 기관들을 오싹하게 했다. 소자의 서버는 국방부 데이터 센터, CIA의 드론 운영, 해군 전함 탑재 네트워크에서 찾을 수 있다. 그리고 엘리멘탈은 단지 수백 개 회사 중 하나였을 뿐이었다 황기철 콘페이퍼 에디터 큐레이터 Ki Cheol Hwang, conpaper editor, curator |
edited by kcontents
To help with due diligence, AWS, which was overseeing the prospective acquisition, hired a third-party company to scrutinize Elemental’s security, according to one person familiar with the process. The first pass uncovered troubling issues, prompting AWS to take a closer look at Elemental’s main product: the expensive servers that customers installed in their networks to handle the video compression. These servers were assembled for Elemental by Super Micro Computer Inc., a San Jose-based company (commonly known as Supermicro) that’s also one of the world’s biggest suppliers of server motherboards, the fiberglass-mounted clusters of chips and capacitors that act as the neurons of data centers large and small. In late spring of 2015, Elemental’s staff boxed up several servers and sent them to Ontario, Canada, for the third-party security company to test, the person says.
Featured in Bloomberg Businessweek, Oct. 8, 2018. Subscribe now.PHOTOGRAPHER: VICTOR
PRADO FOR BLOOMBERG BUSINESSWEEK
Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design. Amazon reported the discovery to U.S. authorities, sending a shudder through the intelligence community. Elemental’s servers could be found in Department of Defense data centers, the CIA’s drone operations, and the onboard networks of Navy warships. And Elemental was just one of hundreds of Supermicro customers.
During the ensuing top-secret probe, which remains open more than three years later, investigators determined that the chips allowed the attackers to create a stealth doorway into any network that included the altered machines. Multiple people familiar with the matter say investigators found that the chips had been inserted at factories run by manufacturing subcontractors in China.
This attack was something graver than the software-based incidents the world has grown accustomed to seeing. Hardware hacks are more difficult to pull off and potentially more devastating, promising the kind of long-term, stealth access that spy agencies are willing to invest millions of dollars and many years to get.
“Having a well-done, nation-state-level hardware implant surface would be like witnessing a unicorn jumping over a rainbow”
There are two ways for spies to alter the guts of computer equipment. One, known as interdiction, consists of manipulating devices as they’re in transit from manufacturer to customer. This approach is favored by U.S. spy agencies, according to documents leaked by former National Security Agency contractor Edward Snowden. The other method involves seeding changes from the very beginning.
One country in particular has an advantage executing this kind of attack: China, which by some estimates makes 75 percent of the world’s mobile phones and 90 percent of its PCs. Still, to actually accomplish a seeding attack would mean developing a deep understanding of a product’s design, manipulating components at the factory, and ensuring that the doctored devices made it through the global logistics chain to the desired location—a feat akin to throwing a stick in the Yangtze River upstream from Shanghai and ensuring that it washes ashore in Seattle. “Having a well-done, nation-state-level hardware implant surface would be like witnessing a unicorn jumping over a rainbow,” says Joe Grand, a hardware hacker and the founder of Grand Idea Studio Inc. “Hardware is just so far off the radar, it’s almost treated like black magic.”
But that’s just what U.S. investigators found: The chips had been inserted during the manufacturing process, two officials say, by operatives from a unit of the People’s Liberation Army. In Supermicro, China’s spies appear to have found a perfect conduit for what U.S. officials now describe as the most significant supply chain attack known to have been carried out against American companies.
One official says investigators found that it eventually affected almost 30 companies, including a major bank, government contractors, and the world’s most valuable company, Apple Inc. Apple was an important Supermicro customer and had planned to order more than 30,000 of its servers in two years for a new global network of data centers. Three senior insiders at Apple say that in the summer of 2015, it, too, found malicious chips on Supermicro motherboards. Apple severed ties with Supermicro the following year, for what it described as unrelated reasons.
In emailed statements, Amazon (which announced its acquisition of Elemental in September 2015), Apple, and Supermicro disputed summaries of Bloomberg Businessweek’s reporting. “It’s untrue that AWS knew about a supply chain compromise, an issue with malicious chips, or hardware modifications when acquiring Elemental,” Amazon wrote. “On this we can be very clear: Apple has never found malicious chips, ‘hardware manipulations’ or vulnerabilities purposely planted in any server,” Apple wrote. “We remain unaware of any such investigation,” wrote a spokesman for Supermicro, Perry Hayes. The Chinese government didn’t directly address questions about manipulation of Supermicro servers, issuing a statement that read, in part, “Supply chain safety in cyberspace is an issue of common concern, and China is also a victim.” The FBI and the Office of the Director of National Intelligence, representing the CIA and NSA, declined to comment.
The companies’ denials are countered by six current and former senior national security officials, who—in conversations that began during the Obama administration and continued under the Trump administration—detailed the discovery of the chips and the government’s investigation. One of those officials and two people inside AWS provided extensive information on how the attack played out at Elemental and Amazon; the official and one of the insiders also described Amazon’s cooperation with the government investigation. In addition to the three Apple insiders, four of the six U.S. officials confirmed that Apple was a victim. In all, 17 people confirmed the manipulation of Supermicro’s hardware and other elements of the attacks. The sources were granted anonymity because of the sensitive, and in some cases classified, nature of the information.
One government official says China’s goal was long-term access to high-value corporate secrets and sensitive government networks. No consumer data is known to have been stolen.
The ramifications of the attack continue to play out. The Trump administration has made computer and networking hardware, including motherboards, a focus of its latest round of trade sanctions against China, and White House officials have made it clear they think companies will begin shifting their supply chains to other countries as a result. Such a shift might assuage officials who have been warning for years about the security of the supply chain—even though they’ve never disclosed a major reason for their concerns.
Back in 2006, three engineers in Oregon had a clever idea. Demand for mobile video was about to explode, and they predicted that broadcasters would be desperate to transform programs designed to fit TV screens into the various formats needed for viewing on smartphones, laptops, and other devices. To meet the anticipated demand, the engineers started Elemental Technologies, assembling what one former adviser to the company calls a genius team to write code that would adapt the superfast graphics chips being produced for high-end video-gaming machines. The resulting software dramatically reduced the time it took to process large video files. Elemental then loaded the software onto custom-built servers emblazoned with its leprechaun-green logos.
Elemental servers sold for as much as $100,000 each, at profit margins of as high as 70 percent, according to a former adviser to the company. Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not.
Elemental also started working with American spy agencies. In 2009 the company announced a development partnership with In-Q-Tel Inc., the CIA’s investment arm, a deal that paved the way for Elemental servers to be used in national security missions across the U.S. government. Public documents, including the company’s own promotional materials, show that the servers have been used inside Department of Defense data centers to process drone and surveillance-camera footage, on Navy warships to transmit feeds of airborne missions, and inside government buildings to enable secure videoconferencing. NASA, both houses of Congress, and the Department of Homeland Security have also been customers. This portfolio made Elemental a target for foreign adversaries.
Supermicro had been an obvious choice to build Elemental’s servers. Headquartered north of San Jose’s airport, up a smoggy stretch of Interstate 880, the company was founded by Charles Liang, a Taiwanese engineer who attended graduate school in Texas and then moved west to start Supermicro with his wife in 1993. Silicon Valley was then embracing outsourcing, forging a pathway from Taiwanese, and later Chinese, factories to American consumers, and Liang added a comforting advantage: Supermicro’s motherboards would be engineered mostly in San Jose, close to the company’s biggest clients, even if the products were manufactured overseas.
Today, Supermicro sells more server motherboards than almost anyone else. It also dominates the $1 billion market for boards used in special-purpose computers, from MRI machines to weapons systems. Its motherboards can be found in made-to-order server setups at banks, hedge funds, cloud computing providers, and web-hosting services, among other places. Supermicro has assembly facilities in California, the Netherlands, and Taiwan, but its motherboards—its core product—are nearly all manufactured by contractors in China.
The company’s pitch to customers hinges on unmatched customization, made possible by hundreds of full-time engineers and a catalog encompassing more than 600 designs. The majority of its workforce in San Jose is Taiwanese or Chinese, and Mandarin is the preferred language, with hanzi filling the whiteboards, according to six former employees. Chinese pastries are delivered every week, and many routine calls are done twice, once for English-only workers and again in Mandarin. The latter are more productive, according to people who’ve been on both. These overseas ties, especially the widespread use of Mandarin, would have made it easier for China to gain an understanding of Supermicro’s operations and potentially to infiltrate the company. (A U.S. official says the government’s probe is still examining whether spies were planted inside Supermicro or other American companies to aid the attack.)
With more than 900 customers in 100 countries by 2015, Supermicro offered inroads to a bountiful collection of sensitive targets. “Think of Supermicro as the Microsoft of the hardware world,” says a former U.S. intelligence official who’s studied Supermicro and its business model. “Attacking Supermicro motherboards is like attacking Windows. It’s like attacking the whole world.”
The security of the global technology supply chain had been compromised, even if consumers and most companies didn’t know it yet
Well before evidence of the attack surfaced inside the networks of U.S. companies, American intelligence sources were reporting that China’s spies had plans to introduce malicious microchips into the supply chain. The sources weren’t specific, according to a person familiar with the information they provided, and millions of motherboards are shipped into the U.S. annually. But in the first half of 2014, a different person briefed on high-level discussions says, intelligence officials went to the White House with something more concrete: China’s military was preparing to insert the chips into Supermicro motherboards bound for U.S. companies.
The specificity of the information was remarkable, but so were the challenges it posed. Issuing a broad warning to Supermicro’s customers could have crippled the company, a major American hardware maker, and it wasn’t clear from the intelligence whom the operation was targeting or what its ultimate aims were. Plus, without confirmation that anyone had been attacked, the FBI was limited in how it could respond. The White House requested periodic updates as information came in, the person familiar with the discussions says.
Apple made its discovery of suspicious chips inside Supermicro servers around May 2015, after detecting odd network activity and firmware problems, according to a person familiar with the timeline. Two of the senior Apple insiders say the company reported the incident to the FBI but kept details about what it had detected tightly held, even internally. Government investigators were still chasing clues on their own when Amazon made its discovery and gave them access to sabotaged hardware, according to one U.S. official. This created an invaluable opportunity for intelligence agencies and the FBI—by then running a full investigation led by its cyber- and counterintelligence teams—to see what the chips looked like and how they worked.
The chips on Elemental servers were designed to be as inconspicuous as possible, according to one person who saw a detailed report prepared for Amazon by its third-party security contractor, as well as a second person who saw digital photos and X-ray images of the chips incorporated into a later report prepared by Amazon’s security team. Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment. Depending on the board model, the chips varied slightly in size, suggesting that the attackers had supplied different factories with different batches.
Officials familiar with the investigation say the primary role of implants such as these is to open doors that other attackers can go through. “Hardware attacks are about access,” as one former senior official puts it. In simplified terms, the implants on Supermicro hardware manipulated the core operating instructions that tell the server what to do as data move across a motherboard, two people familiar with the chips’ operation say. This happened at a crucial moment, as small bits of the operating system were being stored in the board’s temporary memory en route to the server’s central processor, the CPU. The implant was placed on the board in a way that allowed it to effectively edit this information queue, injecting its own code or altering the order of the instructions the CPU was meant to follow. Deviously small changes could create disastrous effects.
Since the implants were small, the amount of code they contained was small as well. But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off.
This system could let the attackers alter how the device functioned, line by line, however they wanted, leaving no one the wiser. To understand the power that would give them, take this hypothetical example: Somewhere in the Linux operating system, which runs in many servers, is code that authorizes a user by verifying a typed password against a stored encrypted one. An implanted chip can alter part of that code so the server won’t check for a password—and presto! A secure machine is open to any and all users. A chip can also steal encryption keys for secure communications, block security updates that would neutralize the attack, and open up new pathways to the internet. Should some anomaly be noticed, it would likely be cast as an unexplained oddity. “The hardware opens whatever door it wants,” says Joe FitzPatrick, founder of Hardware Security Resources LLC, a company that trains cybersecurity professionals in hardware hacking techniques.
U.S. officials had caught China experimenting with hardware tampering before, but they’d never seen anything of this scale and ambition. The security of the global technology supply chain had been compromised, even if consumers and most companies didn’t know it yet. What remained for investigators to learn was how the attackers had so thoroughly infiltrated Supermicro’s production process—and how many doors they’d opened into American targets.
Unlike software-based hacks, hardware manipulation creates a real-world trail. Components leave a wake of shipping manifests and invoices. Boards have serial numbers that trace to specific factories. To track the corrupted chips to their source, U.S. intelligence agencies began following Supermicro’s serpentine supply chain in reverse, a person briefed on evidence gathered during the probe says.
As recently as 2016, according to DigiTimes, a news site specializing in supply chain research, Supermicro had three primary manufacturers constructing its motherboards, two headquartered in Taiwan and one in Shanghai. When such suppliers are choked with big orders, they sometimes parcel out work to subcontractors. In order to get further down the trail, U.S. spy agencies drew on the prodigious tools at their disposal. They sifted through communications intercepts, tapped informants in Taiwan and China, even tracked key individuals through their phones, according to the person briefed on evidence gathered during the probe. Eventually, that person says, they traced the malicious chips to four subcontracting factories that had been building Supermicro motherboards for at least two years.
As the agents monitored interactions among Chinese officials, motherboard manufacturers, and middlemen, they glimpsed how the seeding process worked. In some cases, plant managers were approached by people who claimed to represent Supermicro or who held positions suggesting a connection to the government. The middlemen would request changes to the motherboards’ original designs, initially offering bribes in conjunction with their unusual requests. If that didn’t work, they threatened factory managers with inspections that could shut down their plants. Once arrangements were in place, the middlemen would organize delivery of the chips to the factories.
The investigators concluded that this intricate scheme was the work of a People’s Liberation Army unit specializing in hardware attacks, according to two people briefed on its activities. The existence of this group has never been revealed before, but one official says, “We’ve been tracking these guys for longer than we’d like to admit.” The unit is believed to focus on high-priority targets, including advanced commercial technology and the computers of rival militaries. In past attacks, it targeted the designs for high-performance computer chips and computing systems of large U.S. internet providers.
Provided details of Businessweek’s reporting, China’s Ministry of Foreign Affairs sent a statement that said “China is a resolute defender of cybersecurity.” The ministry added that in 2011, China proposed international guarantees on hardware security along with other members of the Shanghai Cooperation Organization, a regional security body. The statement concluded, “We hope parties make less gratuitous accusations and suspicions but conduct more constructive talk and collaboration so that we can work together in building a peaceful, safe, open, cooperative and orderly cyberspace.”
The Supermicro attack was on another order entirely from earlier episodes attributed to the PLA. It threatened to have reached a dizzying array of end users, with some vital ones in the mix. Apple, for its part, has used Supermicro hardware in its data centers sporadically for years, but the relationship intensified after 2013, when Apple acquired a startup called Topsy Labs, which created superfast technology for indexing and searching vast troves of internet content. By 2014, the startup was put to work building small data centers in or near major global cities. This project, known internally as Ledbelly, was designed to make the search function for Apple’s voice assistant, Siri, faster, according to the three senior Apple insiders.
Documents seen by Businessweek show that in 2014, Apple planned to order more than 6,000 Supermicro servers for installation in 17 locations, including Amsterdam, Chicago, Hong Kong, Los Angeles, New York, San Jose, Singapore, and Tokyo, plus 4,000 servers for its existing North Carolina and Oregon data centers. Those orders were supposed to double, to 20,000, by 2015. Ledbelly made Apple an important Supermicro customer at the exact same time the PLA was found to be manipulating the vendor’s hardware.
Project delays and early performance problems meant that around 7,000 Supermicro servers were humming in Apple’s network by the time the company’s security team found the added chips. Because Apple didn’t, according to a U.S. official, provide government investigators with access to its facilities or the tampered hardware, the extent of the attack there remained outside their view.
https://www.bloomberg.com/news/features/2018-10-04/the-big-hack-how-china-used-a-tiny-chip-to-infiltrate-america-s-top-companies?srnd=businessweek-v2
kcontents
