This blog post is made to help me study for my upcoming CS4226 exams. Lets find out how the internet works!
Autonomous Systems (AS) #
An autonomous system is a network of interconnected routers that are identified by a unique AS number (ASN). They are controlled by a single administrative domain and use common routing protocols and policies.
The most commonly used protocol on the internet is the Border Gateway Protocol (BGP)
They are the fundamental units that make up the internet.
What is the architecture of the internet? #
The structure of the internet is a network of networks. At the center of all of this is a small number of well connected Tier 1 networks. These Tier 1 networks are the backbone of the internet.
Some content providers (EG: Google, Netflix, etc) connect their own private networks that connects themselves to the internet, often bypassing the Tier 1 and regional networks.
AS Topology #
The routers within an AS does not have to be connected to each other directly.
As you can see in the above diagram,
AS X is separated by
AS Y but they are still part of the same AS.
What does it look like? #
The current internet architecture looks something shown above. The redder the color, the more connected the AS is to the rest of the internet.
The top 3 ASes are as follows:
- Level 3
- Cogent Communications
Another interesting thing is that Singtel is actually within the top 20 at 16th place with AS number 7473. It also contains 5.39% of the world’s IP address.
You can see the rankings yourself here.
Internet Peering #
Internet peering is a voluntary interconnection of administratively separate Internet Networks for the purpose of exchanging traffic between the customers of each network.
Why will they peer with each other. #
There are various reasons why ASes would want to peer with each other.
Here are the list of some of the reasons:
- They have a customer-provider relationship
- They have discussed a peer-to-peer agreement
Note: Customers do not need to have AS Numbers.
Based on this relations, ASes can be classified further into 2 categories:
- Transit AS
- Non-Transit AS
Transit AS vs Non-Transit AS #
An AS is a transit AS when traffic from outside your AS is routed through your AS to another outside AS. In this case your AS is a transit AS. When setting up such a network, the network administration must all the eBGP peer to communicate with other eBGP peers.
An AS is a non-transit AS when traffic from outside your network is not allowed to connect to any network that is inside your current network. As a network admin, you will only allow traffic from inside your network to connect to the outside world and the traffic from the outside world to connect to your internal devices.
AS X wants to route traffic to
AS X will send the traffic to
AS Y and
AS Y will send the traffic to
AS Z. In this case,
AS Y is a transit AS. Assuming
AS X and
AS Z do not route traffic to anyone else, they are Non-Transit ASes.
Some examples of Transit ASes are Level 3, Arelion, Cogent Communications, etc. Some examples of Non-Transit ASes are content providers like Google, Netflix, Schools etc.
Other than just being a transit AS or a non-transit AS, you can also be a selective transit AS.
A selective transit AS is an AS that allows transit from a selective few while blocking transit from others.
Customer-Provider relationships #
In this relation, customers pay provider for access to the internet and to be reachable from anyone.
The provider routes the traffic of the customers to the internet. However, the customer does not help to route provider traffic through its networks. We can say that the provider provides transit service to the customer.
A customer can also have 2 providers, this is called multi-homing. When one network goes down, they can make use of the other network.
Peer-to-Peer relationships #
Peer to peer relationships are when 2 ASes have agreed to exchange traffic between each other. They provide transit between their respective customers but do not provide transit from one peer to a separate peer. Often, these agreements are free (settlement-free).
In the example above,
Peer 1 has a peering agreement with
Peer 2 and
Peer 3. However,
Peer 2 does not have a peering agreement with
If customer 1 wants to send data to customer 2, it will go through the peer network between
Peer 1 and
Peer 2. This will be similar between customer 1 and 3.
However, if customer 1 wants to send data to customer 3, it will not go through any peer connections. In this case he will not be able to send data to customer 3.
Peering provides shortcuts between different customers and result in less traffic to the providers between the ASes
Choosing the appropriate relations #
Many ASes have the dilemma of choosing between a customer-provider relationship or a peer-to-peer relationship.
Advantages of peer to peer
- Reduce upstream transit cost
- Improve end-to-end performance
- Might be the only way to connect customers (For Tier 1 ISPs)
Advantages of customer-provider
- Rather have them pay for the connection
- Peers are usually competition
- Peering relationships require periodic renegotiation
Peering agreements are usually confidential and one of the most contentious issues in the ISP world.
ISP / AS Tier list #
Tier 1 ISPs #
Tier 1 ISPs generally:
- Have access to the entire internet through settlement free peering
- Have no upstream provider
- Peer with other Tier 1 ISPs to form a full mesh
There are only 10-12 ASes that are Tier 1 ISPs.
Lower tier ISP #
Lower tier ISPs generally:
- Usually provide transit to their downstream customers but have at least 1 provider upstream.
- Have national or regional coverage
There is only a few thousand ASes which are considered lower tier ISPs.
Stub ASes #
Stub ASes generally:
- Do not provide transit services
- Only connect to upstream providers
This is similar to a customer with an AS number. A large majoring of all the ASes fall into this category.
Notable Peering Disputes #
Here are a list of notable disputes due to peering.
To see even more disputes, you can refer to the Anatomy of Internet Peering Disputes paper.
Internet Exchange Points (IXP) #
Internet exchange points are an open and neutral location where ASes can freely interconnect to exchange traffic. They are similar to an ethernet switch for ASes and are often found in neutral locations.
Why do we need IXP #
Imagine n different providers, each wanting to peer with each other. If they laid out cabling to connect to each other, they will have to lay out (n-1) * n cables. In the long run when there are more and more peers, the amount of cables will increase more and more.
This is where IXPs come in. Instead of laying cables to each other directly, they can now all lay cables to a neutral location where they are connected to each other (similar to an ethernet switch).
Goals of IXP #
The goals of an IXP include:
- Save upstream transit costs
- Keep local traffic local
- Better network performance and Quality of Service
- Better scalability (See above)
How to join an IXP #
Any network which has its own address space, AS number and transit agreements can join an IXP. Each member brings their own router to the IXP and connects it to the switch at the IXP.
Architecture of an IXP #
The diagram above shows a simplified architecture of an IXP. The IXP have a series of core routers which help to route traffic between each of the IXPs.
Choosing a location for an IXP #
Here are some of the criteria used to choose a location of an IXP
An example of good IXP locations are:
- Carrier Neutral Data Center
- Tech Parks
Bad locations include:
- ISP data center
- It is not neutral
- Government data center
- There might be a security risk
- Cable Landing Station
- Physical access might be difficult
How to Operate an IXP #
Operations of an IXP must be carried out by a neutral party. Usually they make use of a consortium model which consists of a management board with representative members of each of the members.
The costs are usually covered equally by all the participants and other services by the IXP.
You can see a list of real world IXPs here.