In the early hours of Tuesday, MainOne experienced an administrative error in the configuration of our edge router during a planned network upgrade which impacted traffic to our internet peering partners.
Specifically, the configuration on our Border Gateway Protocol (BGP) filters led to the inadvertent advertisement of peering partner prefixes through one of our upstream partners, China Telecoms. This leak majorly impacted traffic to some of our peering partners because the return traffic for the leaked prefixes was dropped by ChinaTel. Therefore, during this period, the services of some of our peering partners was not accessible to some of its customers who preferred to use the leaked routes.
These routes were immediately shut down by MainOne as soon as we discovered the error.
It is important to note the BGP technology that routes Internet traffic from systems around the globe is predicated on trust within peering partners and encourages route sharing. While some providers have implemented some filtering on their networks to prevent the re-advertisement of routes that should not be propagated, implementation across networks differ, as evidenced in this instance.
In a statement, Réseaux IP Européens Network Coordination Centre (RIPE NCC), a non-profit organization responsible for allocating and registering IP numbers to ISPs in Europe has weighed in on this issue and concluded it was purely an error in network reconfiguration. “These reconfigurations happen every day in global routing and mistakes can happen, especially since the configuration of routers is error-prone and often still requires manual input which is prone to fat fingers”.
In order to prevent a re-occurrence, we have reviewed our supervisory and configuration policies to ensure these kinds of errors are avoided in the future.
MainOne appreciates our partners for their patience and cooperation through this brief period of internet disruption and we assure you of our continued commitment to the high degree of trust that makes the internet work efficiently.