Skip to content

How do I find the network PPS quota for my Amazon EC2 instances?

3 minute read
0

I want to find the network packets per second (PPS) quota for my Amazon Elastic Compute Cloud (Amazon EC2) instance.

Short description

Use an iperf2 test to benchmark PPS based on your network characteristics and determine the PPS quota of your EC2 instance. For more information, see What is iPerf/iPerf3? on the iPerf website.

Note: TCP and UDP traffic, number of flows, packet size, new and existing connections, and security group rules affect the PPS of your instance.

Resolution

Complete the following steps:

  1. Use an Amazon Linux 2 (AL2) or Amazon 2023 (AL2023) Amazon Machine Image (AMI) to launch two instances in the same Availability Zone.
    Note: The instances must be the same instance type.

  2. To install iperf on your instance, run the following command on each instance:

    sudo yum install iperf

    If the tool installation works as expected, then proceed to step 6 to run the iPerf tool. If you encountered installation issues, then you must manually build and install the packages.

  3. To manually install the development tools and git, run the following commands on each instance:

    sudo yum groupinstall "Development Tools" -y
    sudo yum install git -y
  4. Run the following commands on each instance to change the directory to the /usr/local directory and copy the iPerf repository to your machine:

    cd /usr/local/
    sudo git clone https://git.code.sf.net/p/iperf2/code iperf2-code
  5. To build and install the package, run the following commands on each instance:

    cd /usr/local/iperf2-code
    sudo ./configure
    sudo make && make install
  6. Run the following command to use one instance as a iPerf server that runs in UDP mode for maximum performance:

    sudo /usr/local/bin/iperf -s -u

    Example output:

    ------------------------------------------------------------
    Server listening on UDP port 5001
    UDP buffer size:  208 KByte (default)
    ------------------------------------------------------------
  7. Make sure that the security group for the iPerf server instance allows connections on 5001/UDP.

  8. Run the following command to use the iPerf client instance to connect to the iPerf server instance with UDP:

    sudo /usr/local/bin/iperf -c private_ip_of_server_instance -u -i 1 -l 16 -b 20kpps -e -P64 -o /tmp/bw_test.txt && grep -i sum-64 /tmp/bw_test.txt

    Note: Replace private_ip_of_server_instance with the internal IP address of the iPerf server instance. For more information about the command, see IPERF on the iPerf website.
    Example output:

    [SUM-64] 0.00-1.00 sec 9.42 MBytes 79.1 Mbits/sec 617416/2 617416 pps
    [SUM-64] 1.00-2.00 sec 10.1 MBytes 84.3 Mbits/sec 658689/0 658692 pps
    [SUM-64] 2.00-3.00 sec 10.1 MBytes 84.7 Mbits/sec 661837/0 661838 pps
    [SUM-64] 3.00-4.00 sec 10.1 MBytes 84.6 Mbits/sec 661226/0 661226 pps
    [SUM-64] 6.00-7.00 sec 9.73 MBytes 81.7 Mbits/sec 637975/0 637975 pps
    [SUM-64] 7.00-8.00 sec 9.46 MBytes 79.4 Mbits/sec 620172/0 620172 pps
    [SUM-64] 8.00-9.00 sec 9.46 MBytes 79.4 Mbits/sec 620150/0 620151 pps

    The command writes the output to a log file under /tmp/bw_test.txt. The file shows a summary of each second that you ran the test and the PPS. Check the output for the maximum PPS for the instance type to identify your PPS quota.

Note: PPS allowance is separate from the overall bandwidth allowance. Even if an instance is under overall bandwidth allowance, the instance might exceed the PPS allowance if the mean packet size is too small. To determine whether your instance exceeds its PPS quota, check the pps_allowed_exceeded metric for the Elastic Network Adapter (ENA) driver.

Related information

iPerf2 user documentation on the iPerf website

AWS OFFICIALUpdated 8 months ago
6 Comments

Can you please provide an example of an output with its interpretation?

replied 3 years ago

I am seeing the following output for e.g: Stripped off the initial part which shows the ip address.

[ ID] Interval Transfer Bandwidth Write/Err PPS [ 12] 0.00-1.00 sec 83.1 KBytes 681 Kbits/sec 0/0 5316 pps [ 48] 0.00-1.00 sec 80.8 KBytes 662 Kbits/sec 0/0 6313 pps [ 16] 0.00-1.00 sec 78.1 KBytes 640 Kbits/sec 0/0 4995 pps ....

[SUM] 0.00-10.03 sec 57.9 MBytes 48.4 Mbits/sec 0/0 378161 pps [SUM-128] Sent 3791670 datagrams [ 8] Server Report: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 8] 0.00-9.76 sec 837 KBytes 703 Kbits/sec 0.020 ms 6786/60369 (11%) [ 8] 0.00-9.76 sec 2 datagrams received out-of-order [ 6] Server Report: [ 6] 0.00-10.02 sec 818 KBytes 669 Kbits/sec 0.018 ms 7630/59966 (13%) [ 6] 0.00-10.02 sec 165 datagrams received out-of-order .... [ 9] Server Report: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 9] 0.00-10.01 sec 848 KBytes 694 Kbits/sec 0.791 ms 4281/58556 (7.3%)

I see the total datagrams sent and the loss per thread. Where do I see the pps limit?

replied 3 years ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

AWS
EXPERT

replied 3 years ago

Also if we test this on a windows instance, with iperf3, although there is a packet loss observed in the iperf output, we don't see any allowance_exeeded metrics in Windows Performance Monitor - ENA packet reshaping reflecting this. https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/monitoring-network-performance-ena.html

replied 3 years ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

AWS
MODERATOR

replied 3 years ago

there's a typo.

cd /usr/local/iperf2-codesudo 
./configure

shouldn't be the following?

cd /usr/local/iperf2-code
sudo ./configure

replied 2 years ago