On Thanksgiving Day 2023, Cloudflare detected a threat actor on their self-hosted Atlassian server. Their security team responded by removing access and initiating an investigation. CrowdStrike's Forensic team was brought in for an independent analysis, and the analysis was completed in early February 2024. It was found that no Cloudflare customer data or systems were compromised due to security measures like access controls, firewall rules, and Zero Trust tools.
Cloudflare elaborated on the attack timeline and the investigation details in a blog post. The threat actor engaged in intelligence gathering from November 14 to 17, obtaining access to internal wikis and bug databases. They returned on November 22, gaining persistent access to Atlassian servers and attempting to access source code management systems.
They accomplished this by utilizing one access token and three service account credentials that were acquired and remained unchanged following the Okta compromise in October 2023. All unauthorized access was terminated by November 24, with the last evidence of threat activity detected that day.
After removing the threat actor from their environment on November 24, the Cloudflare security team gathered personnel from various departments within the company to investigate the breach thoroughly. Their goal was to confirm the complete denial of access to Cloudflare systems by the threat actor and to comprehensively assess the extent of their unauthorized access.
Starting from November 27, a significant portion of Cloudflare's technical staff, both within and outside the security team, shifted their focus to a unified project named Code Red. This initiative aimed to fortify, validate, and address any vulnerabilities within the Cloudflare environment to ensure resilience against future intrusions. Simultaneously, they continued meticulous examination of every system, account, and log to verify the absence of persistent access by the threat actor and to gain an understanding of the attacker’s interactions with Cloudflare systems, including attempted accesses.
The threat actor primarily targeted Cloudflare's Atlassian environment in search of sensitive information regarding the global network's architecture, security, and management. To prevent potential future breaches and to mitigate any overlooked vulnerabilities, a comprehensive security protocol overhaul was initiated. This included rotating over 5,000 production credentials, segmenting test and staging systems, performing forensic analysis on nearly 4,900 systems, and rebooting all machines globally.
Additionally, proactive measures were taken to secure systems in the São Paulo data center, including returning equipment to manufacturers for inspection. Further steps involved updating software packages, identifying and removing unused accounts, and scrutinizing data for any potential security risks. The "Code Red" initiative, involving collaboration from various teams within the company, concluded on January 5, but ongoing efforts persist in enhancing credential management, software security, vulnerability management, and implementing additional security measures.
The blog post detailing the incident included a link to engage in a discussion on Hacker News. Within a discussion thread on Hacker News, tech community members commended Cloudflare for their reliability. Some of the members also inquired about Cloudflare's stance on remaining with Okta as well.
The blog post also contains details known as Indicators of Compromise (IOCs). These IOCs can help organizations affected by the Okta breach to check their logs and ensure that the same threat actor didn't access their systems.
Cloudflare recognized the collective efforts made over the Thanksgiving holiday to carry out the initial analysis and prevent further access by the threat actor. Additionally, Cloudflare expressed gratitude to CrowdStrike for their prompt availability in conducting an independent assessment.