Bandwidth Management using a Queue Tree and PCQ

teejo · May 5, 2016, 3:27pm

Queue Types

This was a bit of a bastard to figure out. It took a lot of research and a bit
of experimentation to sort this out, so this will be a lengthy post.

This configuration can be used with multiple ISPs that have different
bandwidth capacities. See the ISP switch scripts, there’s a line in them to
adjust the queue parent as required.

Note that these are configuration scripts, and are not intended for the script
editor. You can paste them in to the terminal window.

The first thing that you will need to do is to create the appropriate queue
types. The queue types are used by the queue tree.

If you’re wondering why the 1mbit value shows 819KiB, it’s because I’m
accounting for bits (8 bits to a byte).
To get the desired value, use MB * 0.8. 1024Kb * 0.8 = 819KiB
This should make the shaping a bit more accurate, but don’t take my word for
it. I read it somewhere, and it stuck.

My max upload line speed is 1mbit. Although my slower account has a slower
upload speed, I don’t particularly care about it, so I’ve only created the one
max upload queue that will be assigned to the parent of the Queue Tree.

Create additional max upload queues if you want, and adjust the switch ISP
scripts to adjust the queue types.

# REQUIRED
/queue type
# Max Upload Queue
add kind=pcq name=max_upload_1m pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=819KiB

Below are the standard queues that will be assigned to clients. You don’t need
to use them all. Have as few or as many as you want, but you will need at
least one (either rate limited or not).

For the most part, from what I’ve noticed, is that even if you set a client’s
upload speed to say 16k, it will happily burst up to the available bandwidth
that is not being used by other queues. If you want to strictly restrict a
client, you want the rated queues in the next segment.

/queue type
# Upload Queues
add kind=pcq name=upload_512k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=409KiB
add kind=pcq name=upload_256k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=204KiB
add kind=pcq name=upload_128k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=102KiB
add kind=pcq name=upload_64k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=51KiB
add kind=pcq name=upload_32k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=26KiB
add kind=pcq name=upload_16k pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=12KiB

The rate limited queues below will completely restrict a client to the
specific bandwidth. Note that the value entry is different to the value
specified in the non-rated queues above. I don’t know why there is this
discrepancy.

I wasn’t aware of it initially and shot myself in the foot when I accidentally
limited the bandwidth for a connection to 16b instead of 16kb. Oops.
(In winbox you simply enter the digits with no suffix, but for rated you have
to add a k suffix)
You don’t have to add these queues at all if you don’t think you’ll need them.

/queue type
# Rate Limited Uploads
add kind=pcq name=upload_256k_rate pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-rate=204k
add kind=pcq name=upload_128k_rate pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-rate=102k
add kind=pcq name=upload_64k_rate pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-rate=51k pcq-src-address6-mask=64
add kind=pcq name=upload_32k_rate pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-rate=26k
add kind=pcq name=upload_16k_rate pcq-classifier=src-address \
    pcq-dst-address6-mask=64 pcq-rate=12k pcq-src-address6-mask=64

Below are the max download queues. There is one for each ISP that has a
different max bandwidth. The switch scripts change the queue in the parent of
the queue tree as required. You need at least one.

# REQUIRED
/queue type
# Max Download Queues
add kind=pcq name=max_download_10m pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=8192KiB
add kind=pcq name=max_download_6m pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=4915KiB

Below are the standard client download queues. Use what you need. You should
have at least one normal or one rate limited queue.

/queue type
# Download Queues
add kind=pcq name=download_4m pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=3276KiB
add kind=pcq name=download_2m pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=1638KiB
add kind=pcq name=download_1m pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=819KiB
add kind=pcq name=download_512k pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=409KiB
add kind=pcq name=download_256k pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=204KiB
add kind=pcq name=download_128k pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-src-address6-mask=64 pcq-total-limit=102KiB

These are the rate limited download queues. Use what you need. The rate
limiting works VERY well.

/queue type
# Rate Limited Downloads
add kind=pcq name=download_4m_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=3276k pcq-src-address6-mask=64
add kind=pcq name=download_2m_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=1638k pcq-src-address6-mask=64
add kind=pcq name=download_1m_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=819k pcq-src-address6-mask=64
add kind=pcq name=download_512k_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=409k pcq-src-address6-mask=64
add kind=pcq name=download_256k_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=204k pcq-src-address6-mask=64
add kind=pcq name=download_128k_rate pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=102k pcq-src-address6-mask=64

teejo · May 5, 2016, 3:28pm

Connection and Packet Marking

For the Queue Tree to work, you will need to use the Firewall Mangle
facility in order to mark the designated connections.

Before I get to that, though, we should cover unknown connections.
You can’t always shape every single device on your network, so how do you
manage bandwidth for those devices? Turns out it’s pretty easy actually.
What we need to do is to create an address list for the devices we are shaping
and create rules to mark connections for devices that are not in the list.

Cunningly, I call the list ‘shaped’. So for every device you shape, add the
IP for the device to the ‘shaped’ address list.

/ip firewall address-list
add address=162.168.0.10 comment=machine1 list=shaped
add address=192.168.0.11 comment=machine2 list=shaped

First we need to mark the primary incoming and outgoing connections.
The rules below cover all pppoe interfaces, so you can copy and paste as-is.

/ip firewall mangle
# Main Incoming
add action=mark-connection chain=forward comment="Incoming Conn"\
    in-interface=all-ppp new-connection-mark=ppp-all_in_conn
add action=mark-packet chain=forward comment="Incoming Pkt"\
    connection-mark=ppp-all_in_conn new-packet-mark=\
    ppp-all_in_pkt
# Main Outgoing
add action=mark-connection chain=forward comment="Outgoing Conn"\
    new-connection-mark=ppp-all_out_conn out-interface=all-ppp
add action=mark-packet chain=forward comment="Outgoing Pkt"\
    connection-mark=ppp-all_out_conn new-packet-mark=\
    ppp-all_out_pkt

Next, we mark unknown connections. This is where the shaped list comes in,
because we will mark all traffic that is not for IPs in the shaped list.

/ip firewall mangle
# Unknown Connections
add action=mark-connection chain=forward comment=\
    "Unknown In Conn" dst-address-list=!shaped in-interface=\
    all-ppp new-connection-mark=ppp-all_unknown_in_conn
add action=mark-packet chain=forward comment="Unknown In Pkt" \
    connection-mark=ppp-all_unknown_in_conn new-packet-mark=\
    ppp-all_unknown_in_pkt
add action=mark-connection chain=forward comment=\
    "Unknown Out Conn" new-connection-mark=\
    ppp-all_unknown_out_conn out-interface=all-ppp \
    src-address-list=!shaped
add action=mark-packet chain=forward comment=\
    "Unknown Out Pkt" connection-mark=\
    ppp-all_unknown_out_conn new-packet-mark=\
    ppp-all_unknown_out_pkt

The next two entries are simply examples, and will need to be modified for
your network configuration. Change the IP addresses to match each device, and
also change the connection and packet mark names.

There are 4 marking rules per device:

1 Incoming Connection
2 Incoming Packets in the connection
3 Outgoing Connection
4 Outgoing Packets in the connection

/ip firewall mangle
# Machine1
add action=mark-connection chain=forward comment=\
    "Machine1 In Conn" dst-address=192.168.0.10 in-interface=\
    all-ppp new-connection-mark=ppp-all_machine1_in_conn
add action=mark-packet chain=forward comment="Machine1 In Pkt"\
    connection-mark=ppp-all_machine1_in_conn new-packet-mark=\
    ppp-all_machine1_in_pkt
add action=mark-connection chain=forward comment=\
    "Machine1 Out Conn" new-connection-mark=\
    ppp-all_machine1_out_conn out-interface=all-ppp src-address=\
    192.168.0.10
add action=mark-packet chain=forward comment="Machine1 Out Pkt" \
    connection-mark=ppp-all_machine1_out_conn new-packet-mark=\
    ppp-all_machine1_out_pkt

/ip firewall mangle
# Machine2
add action=mark-connection chain=forward comment=\
    "Machine2 In Conn" dst-address=192.168.0.11 in-interface=\
    all-ppp new-connection-mark=ppp-all_machine2_in_conn
add action=mark-packet chain=forward comment="Machine2 In Pkt"\
    connection-mark=ppp-all_machine2_in_conn new-packet-mark=\
    ppp-all_machine2_in_pkt
add action=mark-connection chain=forward comment=\
    "Machine2 Out Conn" new-connection-mark=\
    ppp-all_machine2_out_conn out-interface=all-ppp src-address=\
    192.168.0.11
add action=mark-packet chain=forward comment="Machine2 Out Pkt" \
    connection-mark=ppp-all_machine2_out_conn new-packet-mark=\
    ppp-all_machine2_out_pkt

teejo · May 5, 2016, 3:28pm

Queue Tree Configuration

Now to create the actual Queue Tree. First we create the two parents.
One incoming and one outgoing. These parents will be assigned the max_ queue
type that you created earlier. Make sure the assigned queue matches what you
have set.

/queue tree
# Parents
add name=ppp_download packet-mark=ppp-all_in_pkt parent=global queue=\
    max_download_6m
add name=ppp_upload packet-mark=ppp-all_out_pkt parent=global queue=\
    max_upload_1m

Next we handle the unknowns. I tend to limit the bandwidth for unknowns, so
I use one of the rate limited queues.

/queue tree
# Unknowns
add name=ppp_d_unknown packet-mark=ppp-all_unknown_in_pkt parent=\
    ppp_download queue=download_512k_rate
add name=ppp_u_unknown packet-mark=ppp-all_unknown_out_pkt parent=\
    ppp_upload queue=pcq_upload_64k

Next are the entries for each device you want to shape. The next two entries
are purely for example. You will need to modify the packet-mark names to match
what you have created earlier, and assign the desired queues.

/queue tree
# Machine1
add name=ppp_d_machine1 packet-mark=ppp-all_machine1_in_pkt parent=\
    ppp_download priority=5 queue=download_4m_rate
add name=ppp_u_machine1 packet-mark=ppp-all_machine1_out_pkt parent=ppp_upload \
    priority=5 queue=upload_512k

/queue tree
# Machine2
add name=ppp_d_machine2 packet-mark=ppp-all_machine2_in_pkt parent=\
    ppp_download priority=5 queue=download_1m
add name=ppp_u_machine2 packet-mark=ppp-all_machine2_out_pkt parent=ppp_upload \
    priority=5 queue=upload_64k

You will probably need to test and adjust the assigned queues in the queue
tree, but that’s pretty much it.

Enjoy!

InsanityFlea · June 6, 2016, 8:11am

I need to create one for Arma, since some people have shoddy lines.

InsanityFlea · September 27, 2016, 11:09am

Correct, you need the firewall to be configured. It’s posted above. Please see the sections /ip firewall

InsanityFlea · September 27, 2016, 12:48pm

Yes, you need to configure your rules to suit your needs. A basic firewall setup can be found here:

http://wiki.mikrotik.com/wiki/Basic_universal_firewall_script

or

http://wiki.mikrotik.com/wiki/Securing_your_router

teejo · September 27, 2016, 2:53pm

This might help you a bit. This is my current firewall filter list, with my specific protocol forwards removed.
I’ve also included the address lists referenced.

# sep/27/2016 16:48:52 by RouterOS 6.36.2
# software id = SBME-7QMS
#
/ip firewall filter
add action=accept chain=input comment="allow established connections" connection-state=established
add action=accept chain=input comment="allow related connections" connection-state=related
add action=accept chain=input comment="allow icmp internal only" in-interface=!all-ppp protocol=icmp
add action=accept chain=input comment="Accept to Router from inside" in-interface=!all-ppp src-address=192.168.1.0/24
add action=accept chain=forward comment="allow from local network" src-address=192.168.1.0/24
add action=drop chain=forward comment="Drop Src Bogons" src-address-list=Bogons
add action=drop chain=forward comment="Drop Dst Bogons" dst-address-list=Bogons
# Add accept forward rules here
add action=accept chain=forward comment="allow already established connections" connection-state=established
add action=accept chain=forward comment="allow related connections" connection-state=related
add action=drop chain=forward comment="drop invalid connections" connection-state=invalid log-prefix=DROP-INVALID-FWD protocol=tcp
add action=drop chain=input comment="drop invalid connections" connection-state=invalid log-prefix=DROP-INVALID-INP
add action=drop chain=input comment="input final drop" log-prefix=INP-DROP
add action=drop chain=forward comment="forward final drop" in-interface=all-ppp log-prefix=FWD-DROP

# sep/27/2016 16:50:43 by RouterOS 6.36.2
# software id = SBME-7QMS
#
/ip firewall address-list
add address=0.0.0.0/8 list=Bogons
add address=127.0.0.0/8 list=Bogons
add address=224.0.0.0/3 list=Bogons

InsanityFlea · August 17, 2017, 2:04pm

Thanks to this managed to setup PCQ for 183 ip’s and finally everyone is getting decent internet and no more hogs.

teejo · August 17, 2017, 9:37pm

I still want to write a script that will take a list of IPs and generate the routeros commands for you. Will make life so much easier.

InsanityFlea · August 23, 2017, 4:59am

Some nice examples:

 for x from=10 to=100 do={
/ip firewall mangle add action=mark-connection chain=forward comment="Machine $x In" dst-address=192.168.0.$x in-interface=all-ppp new-connection-mark=ppp-all_machine$x_in_conn
}

That will spit out everything from 192.168.0.10 to 192.168.0.100 for instance.

If you need two variables you can do this:

    for x from=10 to=100 do={
    for y from=101 to=200 do { 
    blah=$x blah=$y
    }}

teejo · August 17, 2018, 10:38am

This is a long time coming, but I’ve created a batch file that will make adding machines to the queue MUCH easier. It just generates a text file where you can copy and paste the contents in to a Winbox terminal.

First you must create a file called input.csv which has the following format:
IP Address,Short Name,“Long Name”

There can be multiple lines. A text file for each machine will be created.

The short name must not have any spaces and the long name must be in quotes if it has any spaces in it.

input.csv:

192.168.1.5,mediaplayer,"Media Player"

I’ve updated it so that you can specify queue names and priority.

quegen.bat:

@Echo Off
REM ipaddress,short machine name,"long machine name"
set DOWNQUEUE=download_4m
set UPQUEUE=upload_512k
set PRIORITY=5
set OFILE=commands.txt
set INFILE=input.csv


if not exist %INFILE% GOTO:NOTEXIST

For /f "tokens=1,2,3* delims=," %%A in (%INFILE%) do CALL:GenerateOutput %%A %%B %%C
GOTO:EOF

:GenerateOutput
set OUTFILE=%2-%OFILE%
echo /ip firewall address-list add address=%1 comment="%~3" list=shaped >%OUTFILE%

echo /ip firewall mangle>>%OUTFILE%
echo add action=mark-connection chain=forward comment=\>>%OUTFILE%
echo     "%~3 In Conn" dst-address=%1 in-interface=\>>%OUTFILE%
echo     all-ppp new-connection-mark=ppp-all_%2_in_conn>>%OUTFILE%
echo add action=mark-packet chain=forward comment="%~3 In Pkt"\>>%OUTFILE%
echo     connection-mark=ppp-all_%2_in_conn new-packet-mark=\>>%OUTFILE%
echo     ppp-all_%2_in_pkt>>%OUTFILE%
echo add action=mark-connection chain=forward comment=\>>%OUTFILE%
echo     "%~3 Out Conn" new-connection-mark=\>>%OUTFILE%
echo     ppp-all_%2_out_conn out-interface=all-ppp src-address=\>>%OUTFILE%
echo     %1>>%OUTFILE%
echo add action=mark-packet chain=forward comment="%~3 Out Pkt" \>>%OUTFILE%
echo     connection-mark=ppp-all_%2_out_conn new-packet-mark=\>>%OUTFILE%
echo     ppp-all_%2_out_pkt>>%OUTFILE%

echo /queue tree>>%OUTFILE%
echo add name=ppp_d_%2 packet-mark=ppp-all_%2_in_pkt parent=\>>%OUTFILE%
echo     ppp_download priority=%PRIORITY% queue=%DOWNQUEUE% comment="%~3 Download">>%OUTFILE%
echo add name=ppp_u_%2 packet-mark=ppp-all_%2_out_pkt parent=ppp_upload \>>%OUTFILE%
echo     priority=%PRIORITY% queue=%UPQUEUE% comment="%~3 Upload">>%OUTFILE%
GOTO:EOF

:NOTEXIST
echo Input file %INFILE% does not exist.
echo pause

This is only useful if you’ve already set everything up. If you’ve changed any of the queue names, be sure to change the section that generates the queue tree entries to reflect correctly.