At 01:35 a.m. on July 2 (local time), a severe outage occurred on the mobile network of KDDI, the second largest telecom operator in Japan. This incident interrupted call, SMS, and data services of up to 39.15 million users across the country. On top of that, it also hit some public services, such as weather forecast, banking, and transportation. This incident, more severe than similar incidents in 2019 and 2020, reaffirms the importance of building robust core networks to mitigate potential network failures.
The network services are finally fully restored after 86 hours of disruption, making it the most severe network incident in the telecommunication industry. KDDI has not given a detailed explanation for why this incident happens.
A Communication Blackout
In the early morning of July 2, KDDI engineers replaced old core routers with new ones during routine maintenance. However, the new routers encountered an unknown failure and then cannot properly route voice traffic to switching nodes of the VoLTE network. Consequently, VoLTE services are interrupted for 15 minutes.
KDDI engineers immediately rolled back the replacement. After the rollback, a large number of terminals initiated registration with the VoLTE network. This led to a signaling storm, making a large number of users fail to access VoLTE services.
At the same time, user data in databases of the core network becomes inconsistent, causing call service failures.
This disruption gave rise to a cascade effect, knocking out the whole data and voice services. KDDI was forced to disable network access as network congestion grew.
Lesson: Core network reliability is pivotal.
This incident was initially caused by just the replacement of core routers by KDDI. However, this severely affected the core network. The core network is the brain of a telecom network; and a minor fault may cause a national core network outage if there is no reliability mechanism available, affecting social and economic running. Similar incidents have occurred for several times, indicating that insufficient attention has been paid on these issues in the industry.
This time, KDDI’s network crisis pushes the core network to the center of attention for the global mobile communications industry. Operators and equipment vendors are on alert, and their cautiousness toward core networks must be resumed along with their services.
Looking back at the development of core networks, we see a clear trend toward cloudification, which is leading the telecom industry toward a broader horizon, but this has also thrown the industry under the heavy pressure of reliability and stability. Since operators shifted their eyes toward NFV and started to cloudify their core networks in 2015, many similar incidents occurred. As 5G gears up for scaled commercialization, NFV is being deployed in an ever-increasing pace to expand and optimize the existing core networks, aiming to upgrade virtual resources to pooled resources and reshape the cloud-ready networks as cloud native networks. Such a pace brings more stringent requirements on core networks.
Moreover, 5G will have a far-reaching effect on economic and social running. It is finding its way to the production and operation phases in various industries, stimulating business growth and unleashing infinite potential. As a key driving force behind this, 5G core network must stay flexible as a service. Moreover, it needs to leverage key technologies such as cloudification and virtualization to offer differentiated network capabilities and services to individuals and industries. In this case, if the core network goes down, a signaling storm is more likely to occur, throwing the operator into the KDDI situation. Reliability is at the root of core network services. For all parties in the industry, especially the small- and medium-sized vendors that lack prowess in core network reliability need to pay much more attention on this. Prevention is better than cure; what the industry need is to fortify the reliability of core networks, instead of only rectifying network problems.
In addition to high flexibility, 5G core networks must provide rock-solid reliability. The core network is central to 5G network construction and service running. 2020 saw the debut of 5G SA commercialization, on which core network devices run with the functionality stipulated in 3GPP R15. From this year, 5G core networks will be further upgraded as stipulated in R16 and R17, to truly underpin eMBB, URLLC, and mMTC services, which are the anchors for operators to navigate through new 5G markets. To achieve this, telecom networks must adapt to the ever-diversifying and differentiated services of different industries, such as industrial manufacturing, transportation, electric power, and finance. More importantly, the networks must satisfy extremely strict latency and reliability requirements from some industries. The electric power industry, for example, demands a reliability of 99.9999% for differential protection. In addition to the flexible system architecture of the 5G core network, the 5G core network must be highly stable and reliable to enable thousands of industries. There are many 5G core network equipment vendors emerging, but how many of them really place a high value on reliability investment?
The Way Ahead
5G will connect everything. And a single point of fault on a core network may take down trillions of devices, dropping them like dominoes. We can never place too much attention on core network reliability and stability. To secure core networks, operators must team up with equipment vendors to consistently solidify core networks along with their service innovation. On the other hand, the standard organizations, industry customers, and analyst agencies should also weigh more on this in their works.
