Forescout Research Labs concludes Project Memoria – Lessons Learned after 18 months of vulnerability research
What is Project Memoria?
Project Memoria is the largest study on the security of TCP/IP stacks. The idea for this project emerged in May 2020 while collaborating with JSOF on Ripple20. Our researchers understood that the problem with TCP/IP stacks was much deeper and more widespread than initial research had suggested. We hypothesized that similar issues to those identified in Ripple20 could be present in other stacks as well.
The will to validate this hypothesis led to our first study, AMNESIA:33, where we confirmed that many open-source TCP/IP stacks shared several similar vulnerabilities. Building on top of the knowledge gathered with AMNESIA:33, we extended the project by first looking into similar bugs in different stacks (NUMBER:JACK and NAME:WRECK) and then looking into security flaws of very specific stacks (INFRA:HALT and NUCLEUS:13).
Results and Lessons Learned
There are a total of 97 vulnerabilities under Project Memoria: 19 in Ripple20 found by JSOF, 33 in AMNESIA:33, 9 in NUMBER:JACK, 9 in NAME:WRECK found together with JSOF, 14 in INFRA:HALT found together with JFrog, and 13 in NUCLEUS:13 found together with Medigate. These vulnerabilities affect 14 TCP/IP stacks.
The figure below summarizes the main achievements of Project Memoria:
- Legacy software enables the connected world. TCP/IP stacks have been around for a long time, and they have a variety of decades-old vulnerabilities. The average age of the stacks we analyzed is almost 20 years. Although these stacks are still actively developed, it is common that some vulnerabilities that have been patched by the stack vendor do not make all the way down the supply chain to all the affected devices. One of the main reasons for that is silent patching.
- Silent patching is a terrible idea.Silent patching means fixing a vulnerability without public documentation. This has always been common practice among software vendors and is slowly changing, with some vendors becoming more open to disclosing issues that are internally discovered or that affect older versions of their software. In Project Memoria, we first encountered silent patching with CVE-2016-20009 in NAME:WRECK. After our publication, as downstream vendors became aware of this 5-year old patched issue, there have been several vendor advisories listing critical vulnerable devices, such as ABB controllers, BD Alaris infusion pumps, GE healthcare devices, Rockwell PLCs, and Siemens gas turbines. Project Memoria shows that silently patched vulnerabilities tend to be rediscovered and that they exist in very critical supply-chain software, which means that there are millions of devices out there that have been vulnerable for a long time without even their vendors knowing about it because other vendors chose to remain silent.
- Vulnerabilities are predictable. We have seen the same mistakes happening repeatedly and shared with the community a set of 11 anti-patterns, or ‘what developers should not do’. These anti-patterns appeared in other vulnerabilities found as a direct or indirect result of the work in this project, such as CVE-2020-17528 and CVE-2020-17529 on NuttX (which are essentially the same as two CVEs in AMNESIA:33 but tracked under a different number) and CVE-2021-25663 and CVE-2021-25664 on Nucleus NET (found by Siemens and similar to IPv6 issues found in AMNESIA:33). By now, we are confident that continuing the project with other stacks would yield many other CVEs. Therefore, we invite the security community to continue the work by using (and improving) the artifacts we have shared, which include a fingerprinting script, a static analysis tool, exploit PoCs and traffic samples, which can be requested by sending an email to [email protected].
- Vendors are often unresponsive. Identifying vulnerable vendors and devices has been the greatest challenge in Project Memoria. In the different phases of the project, we identified a total of 422 device vendors that could be using the vulnerable stacks. We currently track 81 vendors that have issued public advisories: 36 have confirmed to be affected, 41 have confirmed not to be affected and 4 are still investigating. Of the 36 affected, 10 will not provide patches and 3 are still working on them. This means that only 19% of potentially affected vendors have provided some public response and only 5.5% have actually patched the vulnerabilities. So far, the vendor with the longest time to publish patches was Schneider Electric: their patches for AMNESIA:33 were published on 12/October/2021, 308 days after the public disclosure on 08/Dec/2020.
- Hundreds of products are impacted. The TCP/IP stacks analyzed under Project Memoria are used in a wide variety of connected devices, so the potential impact of the same vulnerability spans several industries. Some examples of products impacted include motion control and track technology used in manufacturing; high power transmission devices and RTUs used by electrical utilities; printers and switches used in corporate organizations; infusion pumps and patient information systems used in healthcare; as well as industrial controllers from major manufacturers used for several different functions such as building automation and electric vehicle charging stations.
Mitigation – how Forescout helps
The Forescout Platform enables four important measures to mitigate the impact of vulnerabilities such as the ones on Project Memoria:
- Identification of vulnerable devices. Forescout uses the Security Policy Templates (SPTs) module to identify and group potentially vulnerable devices. Besides flagging known vulnerable device models, devices in scope for the policies are scanned using active and passive techniques for several protocols (such as HTTP, FTP, Telnet, ICMP, and TCP) and can be classified as vulnerable with different confidence levels.
- Network segmentation. Forescout provides network flow mapping of existing communications, which helps to identify unintended communications and enforce appropriate segmentation controls. Once vulnerable devices have been identified, they can be logically grouped to decrease the communications allowed to or from them, thereby limiting the likelihood of compromise and the blast radius if a compromise occurs.
- Detection of active exploits. The platform can detect real-time exploitation attempts against vulnerable devices, based on characteristics such as exploit signatures and malformed packets.
- Security Orchestration and Automation. Raised alerts can be forwarded to SIEM/SOAR systems for further analysis or be used to enable immediate action, such as assigning a device to a VLAN, instructing the switch to block and isolate a device from the network or using a virtual firewall to restrict specific traffic.
Conclusion
The main conclusion of Project Memoria is that supply-chain vulnerabilities are here to stay but mitigation is possible, as long as both the security community and individual organizations take action.
When we published AMNESIA:33, we discussed opaque supply chains and the lack of Software Bills of Materials (SBOMs) as the biggest challenges to identifying vulnerable devices. While our project progressed, other researchers have done very important work on vulnerabilities affecting other important supply-chain components, such as DNS forwarders (DNSpooq) and RTOSes (BadAlloc). At the same time, attackers have realized that compromising supply chains is an effective way of targeting organizations. The past year has seen devastating attacks leveraging, for instance, SolarWinds and Kaseya to infiltrate hundreds of organizations. The cybersecurity community has reached a point where both industry and government recognize the complexity of software supply chains and the importance of SBOMs to fix the supply chain vulnerability problem. Luckily, there is important progress being done on that. We are proud to have been a small part of what made progress possible, by highlighting and bringing awareness to an important topic.
For individual organizations, mitigation measures such as device visibility, segmentation, and exploit detection help with supply-chain vulnerabilities and other kinds of threats. However, those measures require mature cybersecurity tools that can handle a growing and changing device landscape, instead of focusing on individual parts of the network. The same vulnerabilities in the project affected devices as diverse as medical equipment and building automation controllers. A security tool that has visibility on only one of those groups could allow for large blind spots on the heterogeneous networks of modern organizations. At the same time, one of the most important takeaways from the project is that simply identifying vulnerable devices is not enough if no further action can be taken. Organizations must adopt security tools that allow for detection of threats and automated or orchestrated response.
Concluding Project Memoria does not mean that our work is done, either for TCP/IP stacks or for other foundational components of the connected device ecosystem. As we did in previous studies, we invite other researchers and device vendors to continue this work and to collaborate with us in future research.
Disclaimer: In this blog, we account for both vulnerabilities clearly branded as Project Memoria (AMNESIA:33, NUMBER:JACK, NAME:WRECK, INFRA:HALT and NUCLEUS:13) as well as vulnerabilities strictly connected to Project Memoria, such as Ripple20.