더시드위키
최근 변경최근 토론
특수 기능

2025년 11월 18일 Cloudflare Global Network 오류(r30 Blame)

보기RAWBlame되돌리기비교
r30
r2
Aidenk
1[[분류:Aidenk]]
r3
Aidenk
2[include(틀:사건사고)]
r5
Aidenk
3[목차]
r2
Aidenk
4== 개요 ==
52025년 11월 18일 Cloudflare Global Network 오류로 인하여 전세계의 Cloudflare을 쓰는 웹 및 앱이 사용 불능이 된 사태.
r24
Aidenk
6
7== 이유 ==
r27
Aidenk
8[[https://blog.cloudflare.com/18-november-2025-outage/|원본.]]
r24
Aidenk
9
r28
Aidenk
10[[파일:클플 오류1.png]]
11
122025년 11월 18일 11:20 UTC에(이 블로그의 모든 시간은 UTC 기준), Cloudflare 네트워크에서 핵심 트래픽 전달에 심각한 장애가 발생하기 시작했습니다. 이로 인해 인터넷 사용자가 고객 사이트에 접속하려고 하면, Cloudflare 네트워크 내부에서 문제가 발생했다는 오류 페이지가 표시되었습니다.
13
14[[파일:클플 오류2.png]]
15
16'''이 문제는 사이버 공격이나 악의적 활동과는 직접적·간접적으로도 관련이 없었습니다.''' 대신, 데이터베이스 시스템 중 하나의 권한 변경으로 인해, Bot Management 시스템에서 사용하는 “feature file”에 여러 항목이 출력되면서 문제가 발생했습니다. 그 결과, 해당 feature file의 크기가 두 배로 늘어났고, 이 커진 파일이 네트워크를 구성하는 모든 머신에 전파되었습니다.
17
18이 머신들에서 트래픽을 라우팅하는 소프트웨어는 Bot Management 시스템을 최신 위협에 맞게 유지하기 위해 이 feature file을 읽습니다. 그런데 이 소프트웨어에는 파일 크기에 제한이 있었고, 두 배로 커진 파일 크기가 그 한계를 초과하면서 소프트웨어가 실패하게 된 것입니다.
19
20처음에는 우리가 관찰한 증상이 대규모 DDoS 공격 때문이라고 잘못 의심했지만, 이후 핵심 원인을 정확히 파악하여 예상보다 커진 feature file의 전파를 중단하고 이전 버전으로 교체할 수 있었습니다. 14:30경에는 핵심 트래픽이 대부분 정상적으로 흐르기 시작했고, 그 후 몇 시간 동안 네트워크 각 부분에 급격히 몰린 트래픽을 완화하는 작업을 진행했습니다. 17:06에는 Cloudflare의 모든 시스템이 정상적으로 작동했습니다.
21
22고객과 인터넷 전반에 끼친 영향에 대해 사과드립니다. Cloudflare가 인터넷 생태계에서 중요한 역할을 하는 만큼, 시스템 장애는 용납될 수 없습니다. 네트워크가 일정 시간 동안 트래픽을 라우팅할 수 없었던 상황은 팀의 모든 구성원에게 깊은 고통이었습니다. 오늘 여러분을 실망시킨 것을 알고 있습니다.
23
24이 글은 정확히 어떤 일이 일어났고, 어떤 시스템과 프로세스가 실패했는지를 자세히 설명하는 내용입니다. 또한 이번과 같은 장애가 다시 발생하지 않도록 하기 위한 계획의 시작점이기도 하지만, 끝은 아닙니다.
25
r30
Aidenk
26'''{{{+5 장애 상황}}}'''
r28
Aidenk
27아래 차트는 Cloudflare 네트워크에서 발생한 5xx HTTP 상태 코드의 양을 보여줍니다. 정상적으로는 이 수치가 매우 낮아야 하는데, 장애가 시작되기 전까지는 정상적으로 낮은 상태였습니다.
28
29[[파일:클플 오류3.png]]
30
31
32
33
r2
Aidenk
34== 에러 상황 ==
r1
Aidenk
35{{{#!wiki style="border: 1px solid gray; border-top: 5px solid crimson; padding: 12px"
36'''{{{+1 Cloudflare Global Network experiencing issues}}}'''
37
r20
Aidenk
38'''Resolved''' - This incident has been resolved.
r21
Aidenk
39{{{-2 {{{#gray Nov 18, 2025 - 19:28 UTC}}}}}}
r20
Aidenk
40
41'''Update''' - Cloudflare services are currently operating normally. We are no longer observing elevated errors or latency across the network.
42
43Our engineering teams continue to closely monitor the platform and perform a deeper investigation into the earlier disruption, but no configuration changes are being made at this time.
44
45At this point, it is considered safe to re-enable any Cloudflare services that were temporarily disabled during the incident. We will provide a final update once our investigation is complete.
46{{{-2 {{{#gray Nov 18, 2025 - 17:44 UTC}}}}}}
47
48'''Update''' - We continue to monitor the system through recovery and we are seeing errors and latency return to normal levels. A full post-incident investigation and details about the incident will be made available asap.
49{{{-2 {{{#gray Nov 18, 2025 - 17:14 UTC}}}}}}
50
51'''Update''' - We continue to see errors drop as we work through services globally and clearing remaining errors and latency.
52{{{-2 {{{#gray Nov 18, 2025 - 16:46 UTC}}}}}}
53
54'''Update''' - We continue to see errors and latency improve but still have reports of intermittent errors. The team continues to monitor the situation as it improves, and looking for ways to accelerate full recovery.
55{{{-2 {{{#gray Nov 18, 2025 - 16:27 UTC}}}}}}
56
r14
Aidenk
57'''Update''' - Bot scores will be impacted intermittently while we undergo global recovery. We will update once we believe bot scores are fully recovered.
58{{{-2 {{{#gray Nov 18, 2025 - 16:04 UTC}}}}}}
59
r7
Aidenk
60'''Update''' - The team is continuing to focus on restoring service post-fix. We are mitigating several issues that remain post-deployment.
61{{{-2 {{{#gray Nov 18, 2025 - 15:40 UTC}}}}}}
62
r1
Aidenk
63'''Update''' - We are continuing to monitor for any further issues.
64{{{-2 {{{#gray Nov 18, 2025 - 15:23 UTC}}}}}}
65
66'''Update''' - Some customers may be still experiencing issues logging into or using the Cloudflare dashboard. We are working on a fix to resolve this, and continuing to monitor for any further issues.
67{{{-2 {{{#gray Nov 18, 2025 - 14:57 UTC}}}}}}
68
69'''Monitoring''' - A fix has been implemented and we believe the incident is now resolved. We are continuing to monitor for errors to ensure all services are back to normal.
70{{{-2 {{{#gray Nov 18, 2025 - 14:42 UTC}}}}}}
71
72'''Update''' - We've deployed a change which has restored dashboard services. We are still working to remediate broad application services impact
73{{{-2 {{{#gray Nov 18, 2025 - 14:34 UTC}}}}}}
74
75'''Update''' - We are continuing to work on a fix for this issue.
76{{{-2 {{{#gray Nov 18, 2025 - 14:22 UTC}}}}}}
77
78'''Update''' - We are continuing working on restoring service for application services customers.
79{{{-2 {{{#gray Nov 18, 2025 - 13:58 UTC}}}}}}
80
81'''Update''' - We are continuing working on restoring service for application services customers.
82{{{-2 {{{#gray Nov 18, 2025 - 13:35 UTC}}}}}}
83
84'''Update''' - We have made changes that have allowed Cloudflare Access and WARP to recover. Error levels for Access and WARP users have returned to pre-incident rates.
85We have re-enabled WARP access in London.
86
87We are continuing to work towards restoring other services.
88{{{-2 {{{#gray Nov 18, 2025 - 13:13 UTC}}}}}}
89
90'''Identified''' - The issue has been identified and a fix is being implemented.
91{{{-2 {{{#gray Nov 18, 2025 - 13:09 UTC}}}}}}
92
93'''Update''' - During our attempts to remediate, we have disabled WARP access in London. Users in London trying to access the Internet via WARP will see a failure to connect.
94{{{-2 {{{#gray Nov 18, 2025 - 13:04 UTC}}}}}}
95
96'''Update''' - We are continuing to investigate this issue.
97{{{-2 {{{#gray Nov 18, 2025 - 12:53 UTC}}}}}}
98
99'''Update''' - We are continuing to investigate this issue.
100{{{-2 {{{#gray Nov 18, 2025 - 12:37 UTC}}}}}}
101
102'''Update''' - We are seeing services recover, but customers may continue to observe higher-than-normal error rates as we continue remediation efforts.
103{{{-2 {{{#gray Nov 18, 2025 - 12:21 UTC}}}}}}
104
105'''Update''' - We are continuing to investigate this issue.
106{{{-2 {{{#gray Nov 18, 2025 - 12:03 UTC}}}}}}
107
108'''Investigating''' - Cloudflare is experiencing an internal service degradation. Some services may be intermittently impacted. We are focused on restoring service. We will update as we are able to remediate. More updates to follow shortly.
109{{{-2 {{{#gray Nov 18, 2025 - 11:48 UTC}}}}}}}}}
110
111{{{#!wiki style="border: 1px solid gray; border-top: 5px solid navy; padding: 12px"
112'''{{{+1 SYD (Sydney) on 2025-11-18}}}'''
113
114'''In progress''' - Scheduled maintenance is currently in progress. We will provide updates as necessary.
115{{{-2 {{{#gray Nov 18, 2025 - 15:01 UTC}}}}}}
116
117'''Scheduled''' - We will be performing scheduled maintenance in SYD (Sydney) datacenter on 2025-11-18 between 15:00 and 19:00 UTC.
118
119Traffic might be re-routed from this location, hence there is a possibility of a slight increase in latency during this maintenance window for end-users in the affected region. For PNI / CNI customers connecting with us in this location, please make sure you are expecting this traffic to fail over elsewhere during this maintenance window as network interfaces in this datacentre may become temporarily unavailable.
120
121You can now subscribe to these notifications via Cloudflare dashboard and receive these updates directly via email, PagerDuty and webhooks (based on your plan): https://developers.cloudflare.com/notifications/notification-available/#cloudflare-status.
122{{{-2 {{{#gray Nov 18, 2025 15:00-19:00 UTC}}}}}}}}}
123
124{{{#!wiki style="border: 1px solid gray; border-top: 5px solid navy; padding: 12px"
125'''{{{+1 PPT (Tahiti) on 2025-11-18}}}'''
126
127'''In progress''' - Scheduled maintenance is currently in progress. We will provide updates as necessary.
128{{{-2 {{{#gray Nov 18, 2025 - 12:00 UTC}}}}}}
129
130'''Scheduled''' - We will be performing scheduled maintenance in PPT (Tahiti) datacenter on 2025-11-18 between 12:00 and 16:00 UTC.
131
132Traffic might be re-routed from this location, hence there is a possibility of a slight increase in latency during this maintenance window for end-users in the affected region. For PNI / CNI customers connecting with us in this location, please make sure you are expecting this traffic to fail over elsewhere during this maintenance window as network interfaces in this datacentre may become temporarily unavailable.
133
134You can now subscribe to these notifications via Cloudflare dashboard and receive these updates directly via email, PagerDuty and webhooks (based on your plan): https://developers.cloudflare.com/notifications/notification-available/#cloudflare-status.
135{{{-2 {{{#gray Nov 18, 2025 12:00-16:00 UTC}}}}}}}}}
136
137{{{#!wiki style="border: 1px solid gray; border-top: 5px solid navy; padding: 12px"
138'''{{{+1 ATL (Atlanta) on 2025-11-18}}}'''
139
140'''In progress''' - Scheduled maintenance is currently in progress. We will provide updates as necessary.
141{{{-2 {{{#gray Nov 18, 2025 - 07:02 UTC}}}}}}
142
143'''Scheduled''' - We will be performing scheduled maintenance in ATL (Atlanta) datacenter between 2025-11-18 07:00 and 2025-11-19 22:00 UTC.
144
145Traffic might be re-routed from this location, hence there is a possibility of a slight increase in latency during this maintenance window for end-users in the affected region. For PNI / CNI customers connecting with us in this location, please make sure you are expecting this traffic to fail over elsewhere during this maintenance window as network interfaces in this datacentre may become temporarily unavailable.
146
147You can now subscribe to these notifications via Cloudflare dashboard and receive these updates directly via email, PagerDuty and webhooks (based on your plan): https://developers.cloudflare.com/notifications/notification-available/#cloudflare-status.
148{{{-2 {{{#gray Nov 18, 2025 07:00 - Nov 19, 2025 22:00 UTC}}}}}}}}}
149
150{{{#!wiki style="border: 1px solid gray; border-top: 5px solid orange; padding: 12px"
151'''{{{+1 Support Portal Availability Issues}}}'''
152
153'''Update''' - We are continuing to investigate this issue.
r7
Aidenk
154{{{-2 {{{#gray Nov 18, 2025 - 15:16 UTC}}}}}}
r1
Aidenk
155
r7
Aidenk
156'''Investigating''' - Our support portal is currently experiencing issues, and as such customers might encounter errors viewing or responding to support cases. Responses on customer inquiries are not affected, and customers can still reach us via live chat (Business and Enterprise) through the Cloudflare Dashboard, or via the emergency telephone line (Enterprise).
157We are working to understand the full impact and mitigate this problem.
r22
Aidenk
158{{{-2 {{{#gray Nov 18, 2025 - 11:17 UTC}}}}}}}}}
r7
Aidenk
159
r2
Aidenk
160== 피해 상황 ==
r4
Aidenk
161=== 위키 ===
r10
Aidenk
162 * [[나무위키]]: 간혈적으로 편집, 삭제, 이동, 댓글 작성, ip에 한하여 게시글 작성 불가.
r9
Aidenk
163 * [[더시드위키]]: 접속불가.
164 * [[더시드?위키]]: 편집, 삭제, 이동, 댓글 작성 불가.
165 * [[알파위키]]: 접속 불가.
166 * [[위브위키]]: 접속 불가.
167 * [[테스트위키]](the tree): 접속 불가.
r4
Aidenk
168
169=== 커뮤니티 ===
r11
Aidenk
170 * 아카라이브: 로그인 불가.
r12
Aidenk
171 * X: 전반적인 시스템 이용 불가.
r19
Aidenk
172
r6
Aidenk
173=== 게임 ===
r13
Aidenk
174 * 리그 오브 레전드: 접속 불가.
175 * Geometry Dash: 온라인 서버 이용 불가.
r15
Aidenk
176 * 던전앤파이터: 접속 불가.
r19
Aidenk
177
r4
Aidenk
178=== 정부 기관 ===
179
180=== 대기업 ===
r16
Aidenk
181 * 맥도날드: 키오스크 이용 불가.
r18
Aidenk
182 * Chat GPT: 웹사이트에 한하여 이용 불가.
183 * Canva: 웹사이트, 앱 이용 불가.
r23
Aidenk
184 * Spotify
r19
Aidenk
185
r6
Aidenk
186=== 기타 ===
r19
Aidenk
187 * 불법 OTT 사이트: 접속 불가.
188
189== 대응 ==