On Christmas Day, the zkSync network, a prominent Ethereum scaling network, encountered a significant outage lasting over five hours, leaving users and developers concerned. The incident was attributed to a bug in the network’s automated safety protocols, disrupting the functionality of zkSync Era. However, the zkSync team promptly addressed the issue, providing insights into the root cause and the subsequent corrective actions.
Automated safety protocols bug causes network disruption
The unexpected disruption, starting around 5:50 UTC on Christmas Day, resulted from an automated safety protocols bug triggered by a server glitch. At 7:36 am UTC, the zkSync developers said that the bug, although subtle, significantly impacted the network’s operations. The bug was specifically related to state updates.
Again, the team added that the protocol’s computed state was entirely accurate; however, an operator-related bug emerged during the computation process. This subtle error led the network into defensive mode, causing a temporary outage.
Despite the holiday season, the zkSync team worked diligently to rectify the issue, implementing a fix and promptly publishing an updated Node software version.
Resolution and future measures
The developers successfully resolved the issue, and the blockchain network was declared fully operational at 10:52 am UTC, approximately five hours after the initial incident. The team reassured users that the problem was isolated to the network operator and not an inherent flaw in the protocol itself. They emphasized that the computed state by the protocol remained accurate throughout the episode.
To prevent similar incidents in the future, the team has undertaken measures to enhance the robustness of the automated safety protocols and the overall network infrastructure.
It is important to know that this year, zkSync has joined the list of blockchain networks that have experienced network disruptions with its Christmas Day outage. The incident, caused by a subtle bug in the automated safety protocols, showcased the importance of rigorous testing and constant vigilance in the rapidly evolving blockchain space. As zkSync resumes its operations, the focus now shifts to ongoing efforts to bolster the network’s resilience and ensure a smoother experience for its users in the future.
