Network issues in the datacentre – RESOLVED
We need to inform you that we are currently experiencing issues affecting the Core Network in our datacenter, almost all our servers are currently not reachable.
At the moment we are working with our datacentre staff and hope for the fastest possible resolution.
Estimated Service Restoration: Unfortunately, no ETA can be currently provided. We will update you as soon as any new information is available. Please check for the updates in the comments of this post.
We apologize for any inconvenience this may be causing you or your company. Your patience and understanding are highly appreciated.
During our investigation it was detected that two subnetworks are affected: 37.61.233.0/24 and 37.61.235.0/24
The subnetwork 37.61.233.0/24 is already fully available, and we continue to work with the datacenter to bring the second subnetwork 37.61.235.0/24 online as soon as possible.
We are glad to inform you that the issue has been resolved now.
All services are up and running.
Please find the full explanation of this incident below:
1. the DC had some issue with their core routers, details unknown yet;
2. before the issue, we had networks routed through BlackLotus (DDoS protection service). We get DDoS attacks from time to time, but normally didn’t even notice them;
3. after the issue:
3.1 we lost tunnel connections to BlackLotus, tunnel endpoint IPs on our side were inaccessible from outside networks. Thus, all announcements to BL have been lost;
3.2 announcements through DC were functional, except for two /24 prefixes;
3.3 in parallel, the DC noticed DDoS (which was being filtered by BL prior to the incident) and null-routed target IP.