Users and Cardano stake pool operators (SPO) have reported that over the weekend, at least half of the network’s nodes briefly went offline.
An anomaly caused 50% of Cardano nodes to disconnect and restart, according to a message made on the Input-Output Global Telegram SPO on January 22. Input Output Global is the financial engineering and research team behind the Cardano blockchain.
According to the post explanation
“This appears to have been triggered by a temporary anomaly triggering two reactions in the node, some detached from a peer, and others threw an exception and resumed.”
The Cardano network temporarily declined, but it quickly rebounded on its own. The post explains that “such temporary difficulties” were taken into account in the consensus and node design, and “the systems operated exactly as predicted.”
Block production allegedly continued throughout the anomaly, which occurred between blocks 8300569 and 8300570, but was temporarily slowed. According to reports, the impact was minimal and was comparable to delays experienced during routine operations.
Cardano impacted nodes
Depending on the selected SPO, the majority of nodes automatically recovered.
The cause of the anomaly that led to the node disconnections and restarts is still being looked into as of the time of writing.
The official notification added,
“We’re now researching the root reason for this aberrant activity and deploying more logging measures alongside our regular monitoring methods.”
More than half of the listed nodes were reportedly impacted, according to Tom Stokes, a Cardano SPO and co-founder of Node Shark, in a post on January 22.
He also provided a graph that showed the network sync for more than 300 reporting nodes dropping from 100% to just about 40%.
The network sync bounced back to about 87% following the decrease, but it took some time before it reached its previous level of 100%, according to Stokes chart.
Similar problems were reported to Stoke by another SPO on January 22; however, this SPO stated that “Some SPOs noticed no impact.”
They also added that relays and BPs restarted for others. SPOs, developers, and IOG are currently debugging in Discord.
