Network Infrastructure

Our network infrastructure is based on Juniper EX/QFX/MX devices.

Technical overview

Internet Connectivity

Our AS30823 is connected to pretty large Tier-2 Carriers, which provide us with great peering as well as direct upstream to premium networks such as DTAG (Deutsche Telekom), Vodafone, Telefonica and Liberty Global. Beside of that, we operate private and public peerings.

We maintain multiple 100Gbit of internet connectivity, distributed on 10G, 40G and 100G uplinks. Our capacity policy is, to upgrade whenever a link reaches over 50% load during peak hours. This way, we are able to provide enough capacity even for unexpected peaks.

DDoS-Protection

Our multi-stage DDoS-Protection protects customers against common DDoS threats. We heavily rely on Anycast Routing to spread incoming traffic across network points of presence. Bad traffic is scrubbed over multiple stages, such as BGP Flowspec, Static ACL, flowShield as well as flowProxy DDoS-Filters. We operate a distributed ddos analysis infrastructure, analyzing traffic across network points of presence while communicating with other cluster mebers to exchange routing decisions in realtime.

Speaking of the capacity of our DDoS-Protection, we utilize BGP Flowspec to automatically generate and push rules to our upstreams. These have massive capacity, which allows to us block multiple Terabit of volumetric traffic, for example UDP Reflection. The second line of defense are our routers in each region. We maintain a set of filters (ACL), which discard for example unwanted traffic going in via peering or from customers, which allows to block attacks up to installed link capacities.

Once traffic passed ACL based filters, flowShield validates every packet against flexrules, common signatures and challenge response authentication. This is at the moment a installed capacity of about 2Tbit in all our regions. We aim to distribute traffic by utilizing bgp communities with our upstreams, for example to have traffic from APAC going in over specific links, while EMEA typically uses Peering with either our carriers or us directly. Whenever it makes sense, we try to get a PNI, allowing us to steer traffic any further.

With continous growing egress traffic, our ingress antiddos capacity grows. For example high traffic servers are part of the growth of our DDoS-Protection, as link capacity gets expanded :)

Monitoring

Our network is monitored for packetloss and latency on all relevant uplinks based on Prometheus. Beside of that, network equipment is monitored for load and errors. Measurements are reported to our monitoring, which alerts our oncall duty in case of anomalies.

DNS Caches

Internally reachable dns caching resolvers are available on 10.53.53.53 (ipv4) and fd53:: (ipv6) to all customers, without blacklisting restrictions. High availability and loadbalancing is done using ECMP towards different physical hosts.

Statistics

In order to keep track of our networks load as well as valueable insights, we heavily use sflow together with our own sflow collector called “goflow-es”, which is indeed based on GoFlow. Flow samples are exported to Elasticsearch concurrently, for later analysis or traffic accounting. Customers can use APIv3 to query sflow samples.

Points of Presence

We currently operate the following points of presence:

  • Skylink Eygelshoven, NL (EGH1)

  • Tornado Datacenter Langen, DE (FFM1)

  • Interwerk Frankfurt, DE (FFM2)

  • Interxion Frankfurt, DE (FFM3)

  • Equinix FR5 Frankfurt, DE (FFM5)

  • Maincubes Frankfurt, DE (FFM4)

  • Hetzner Helsinki, FI (HEL1)

All pops are interconnected using darkfiber or dedicated wavelenghts. Every pop is connected redundant over both diverse fiber or multiple upstreams.

Traffic Engineering

AS30823 allows bgp community based traffic engineering as well as passes through communities upstream.

BGP Communities set by AS30823

Internal AS30823 network (ibgp)

  • 30823:30823 - All Announcements from AS30823 and customers

  • 30823:10002 - Announcement originated at FFM2 - Interwerk Frankfurt, Germany

  • 30823:10003 - Announcement originated at FFM3 - Interxion Frankfurt, Germany

  • 30823:10004 - Announcement originated at HEL1 - Hetzner HEL1 Tuusula, Finland

  • 30823:10005 - Announcement originated at FFM4 - Maincubes Frankfurt, Germany

  • 30823:10006 - Announcement originated at EGH1 - Skylink Eygelshoven, Netherlands

External AS30823 network (ebgp)

Upstream
  • 30823:30821 - Route received from Edge Peer (Transit)

  • 30823:30822 - Route received from Core-Backbone

  • 30823:30823 - Route received from RETN Ltd.

  • 30823:30825 - Route received from Elisa Oy

  • 30823:30827 - Route received from GlobeDC

  • 30823:30826 - Route received from CDN77

Peering
  • 30823:1101 - Route received from DataIX Peer

  • 30823:1201 - Route received from GlobalIX Peer

BGP Communities accepted by AS30823

-> X stands for 1-3 prepends or 0 for do not annouce, e.g. 30823:38213 -> prepend 3 times with 30823 to Core-Backbone

General

  • 30823:30820 - Blackhole Prefix

  • 30823:30824 - Do not announce to upstream/peers

Upstream

  • 30823:3821X - Core-Backbone GmbH (AS33891)

  • 30823:3823X - RETN Ltd. (AS9002)

  • 30823:3824X - Elisa Oy (AS6667)

  • 30823:3825X - Skylink Datacenter B.V. (AS44592)

  • 30823:3826X - CDN77 - Datacamp Ltd. (AS60068)

Peering

At the moment, we dont peer at internet exchanges. Peering is solely private peering with physical crossconnects and dedicated ports. Please get in touch in case you want to establish a PNI with us.

Region specific BGP Communities

HEL1

  • 30823:20040 - Do not announce traffic via Frankfurt (This means that in HEL only traffic via Elisa (AS6667) is accepted. There is no longer any redundancy! We assume no responsibility for short-term failures and maintenance at AS6667)