Sunday, April 24, 2011

Low Latency Queuing (LLQ)

For delay sensitive traffic, LLQ is the best choice to ensure good performance. It works pretty much like CBWFQ  but adds the capability for some queues to be configured as low-latency. These are treated as strict priority queues, meaning LLQ always services packets in them first.

Confusion over terms can arise because even a single policy map with one low latency queue configured is known as LLQ. Also, the older Priority Queuing (PQ, no longer in use) is simliar to LLQ and sometimes a single LLQ may be called a ""priority queue." This is due to the IOS command used to configure a low latency queue-  priority.

LLQ queue adds a low latency queue, but prevents queue starvation which was a problem under the older PQ. LLQ will in fact police the low latency queue, making the configured bandwidth both a guaranteed minimum and a policed maximum. So while low latency is guaranteed, some packets might still be dropped to prevent starvation of other queues.

The priority  command is the only addition to queuing configuration commands in IOS for LLQ. It's used instead of the bandwidth command during class configuration. The full syntax is-

priority { bandwidth-kbps | percent  percentage } [ burst ]

An example LLQ configuration follows, taken from the text. It assumes all traffic is already marked as EF, etc.

policy map queue-on-dscp
     class dscp-ef
         priority 58
     class  dscp-af41
        bandwidth 22
     class dscp-af21
        bandwidth 20
        random-detect dscp-based
     class dscp-af23
         bandwidth 8
          random-detect dscp-based
    class class-default
          fair-queue
          random-detect dscp-based

interface Serial0/0
       bandwidth 128
       max-reserved-bandwidth 85
      service-policy output queue-on-dscp

More on Defining and Limiting Bandwidth for LLQ

The priority command for LLQ provides you two choices- define bandwidth as a set number or as a percentage of the total interface bandwidth (the latter is because there's no remaining percent option). Unlike the bandwidth command, you can use both choices within a single policy map. Madness, isn't it?

Of course, you've still got an upper limit set by IOS on total bandwidth usage: max reserved bandwidth times interface bandwidth. Both LLQ and non-LLQ classes can't exceed this magic number. Because both options for the priority command can be used in a single policy map, calculating this number can be confusing unless you pay careful attention. As the book states, you could have one LLQ using the priority option for bandwidth #, another LLQ with bandwidth %, and non-LLQ queues using one of the three bandwidth options.

Let's look at an example using a 256 kbps interface and 3 classes. The unreservable bandwidth is 25% of that, or 64 kbps. Let's say class 1 is configured with priority 32, class 2 with priority percent 25, and class 3 has bandwidth remaining percent 75. This means 32 kbps is used for class 1, 64 kbps for class 2, and 72 kbps for class 3.

As I mentioned before, the calculation for the command bandwidth remaining percent changes when priority queuing is in place. Normally, the command calculates bandwidth remaining as interface bandwidth minus reserved bandwidth (default value 25%). With LLQ in place, this calculation becomes interface bandwidth minus reserved bandwidth minus bandwidth allocated thru priority commands.

Also worth mentioning at this point is why you'd configure multiple queues with the priority command. The answer is policing of traffic. Although you may configure multiple LLQ queues, in reality all these packets will be dumped into one prioritized queue, which will then be serviced FIFO. However, the policing of traffic for each LLQ queue will still take place. Typically, this is used to separately police voice and videoconferencing traffic. Sometimes you'll want to rate limit separately like this.

Miscellaneous Notes
You'll want to configure a bandwidth command under the class class-default. Otherwise, IOS will divide any unallocated bandwidth equally among all classes; this can result in the class-default class having a very small amount of bandwidth.

Similarly, if a queue is empty, IOS will dynamically attempt to redistribute that bandwidth to other queues in proportion to their configured bandwidth commands. Finally, IOS will only use queuing when congestion occurs. Since applying a queuing policy causes IOS to shorten the hardware queue automatically (generally to 1 or 2 packets), "congestion" will occur frequently, ensuring that your QoS policy is enforced on almost all traffic!

Also of note is flow based classification. While only briefly referenced in the text (and in a table at that), flow based classification is for achieving finer granularity in applying QoS. Specifically, it's intended for User Based Rate Limiting.

Thursday, April 21, 2011

Ch 13 QoS Congestion Management and Avoidance

Queuing, or congestion management, is used to manage frames before they exit a device. In routers this is known as output queuing because IP forwarding decisions are made prior to the queuing. Congestion avoidance is a term for managing traffic to decide if/when packets will be dropped during congestion periods. The most common congestion avoidance tool is Weighted Random Early Detection (WRED).

The concept of fancy queuing is defined as using congestion avoidance and management techniques. This is as opposed to simple queuing where you use FIFO. Hardware queuing always uses simple queuing (FIFO) and has the following characteristics:
  • No CPU interrupt is required between packet transmissions from the queue
  • It cannot be affected by IOS queuing tools (since this is the last step before transmission on the wire)
  • The only controllable function from IOS is queue length
  • IOS automatically shrinks hardware queues when other queuing tools are in use
  • These shorter hardware queues mean traffic is more likely to end up in configurable software queues
 One example given in the text is using the tx-ring limit command to configure the number of packets allowed in the hardware transmit queue before buffering into software queues will begin. This is, of course, an interface level command. Interestingly enough, Cisco documents state that this command can be applied to each subinterface PVC, rather than being tied to the interface as a whole. This command is also generally only seen in use on older IOS, with slower speed links (<768K), where manual tuning of parameters is required.

IOS queuing tools maintain software-based queues that feed into the hardware queue, on a per-interface level. Queuing tools can also be used along side traffic shaping, which basically delays packet(s) to ensure that the traffic rate for a class doesn't exceed the defined rate. If you use shaping, it queues packets (by default into a FIFO queue). Also, depending on the shaping tool(s) used, you might be able to manipulate the delayed packets using a queuing tool. However, this won't be discussed until Chapter 14.

The book lists 5 major queuing tools in use with IOS: classification, drop policy, scheduling, maximum number of queues, and maximum queue length. Brief notes on each include:

Classification- checking packet headers to select a queue
Drop Policy- rules for dropping packets during congestion
Scheduling- logic for which packet to dequeue and send next
Max # Classes- how many classes of packets a queuing tool can utilize or is configured to use
Max Queue Length- upper limit of packets contained in a single queue


Queuing Tools: CBWFQ and LLQ
Class Based Weighted Fair Queuing (CBWFQ) reserves bandwidth for each individual queue and provides a way to implement WFQ for the class-default queue. Low Latency Queuing (LLQ) implements a Priority Queue (PQ) in addition to CBWFQ, but unlike the original PQ there can be no starvation of other queues. Both are configured via MQC with its wealth of classification options.

The largest implementation difference is whether you use the bandwidth command (CBWFQ) or the priority command (LLQ). Both support up to 64 queues or classes. Both implement a special queue known as the default (class-default) queue. If a packet fails to match any class-map specifications it is handled by the default queue.

CBWFQ Features and Configuration
CBWFQ implements a scheduler to guarantee a configured percentage of bandwidth to each queue. However, if some queues are empty, bandwidth will be allocated across other classes temporarily. CBWFQ implements that standard classification features. Its drop policy choices include either WRED or tail drop, configurable per queue.

As said, there can be up to 64 queues. Max queue length support varies according to the model of router used. Within each queue, CBWFQ uses FIFO scheduling, except for the class-default, which can use either FIFO or WFQ.


Command Reference
bandwidth { kbps | percent percentage }
             Class level subcommand; sets BW %
bandwidth { remaining percent percent }
              Class level subcommand; sets aside a % of remaining BW for the class
queue-limit queue-limit
             Class level subcommand; sets the maximum length for a queue in packets
fair-queue [ queue-limit queue-value ]
             Class level subcommand; enables QWFQ in the class (class-default only)
max-reserved-bandwidth percent
              Interface level subcommand; sets % of link BW for all classes except class-default. Defaults to 75%

An example config
This is an example config which would place VOIP traffic into one queue and everything else into class-default.

class-map match-all voip-rtp
   match ip rtp 16384 16383
policy-map queue-voip
   class voip-rtp
       bandwidth 64
   class class-default
       fair-queue
interface Serial0/0
   encapsulation frame-relay
   load-interval 30
   bandwidth 128
   service-policy output queue-voip

Define and Limit CBWFQ Bandwidth Usage 
When you apply a service policy to an interface, IOS checks the configured bandwidth (using the bandwidth command) against the bandwidth defined for the policy. The policy defines bandwidth needs using two subcommands- bandwidth and max-reserved-bandwidth. Nonreservable bandwidth is meant for use by overhead traffic.

When defining the bandwidth for CBWFQ, the policy-map uses the product of the bandwidth and max-reserved-bandwidth commands. So if the interface bandwidth is set to 512 kbps and the max reserved value is 75 percent, the policy-map has 384 kbps of bandwidth which can be allocated among the various queues (or bandwidth commands). If the policy-map allocates more total bandwidth than would be available using this calculation, IOS will reject the policy (you can't apply it). Go do some math and try again!

If you're thinking of just changing the max-reserved-bandwidth setting instead to a higher number, be aware that Cisco doesn't recommend you monkey with the defaults unless you know what you're doing. Another way to allocate bandwidth without specifying traffic size in kbps is to use bandwidth percent or bandwidth remaining percent. This makes it unlikely you will oversubscribe. The trade off is you still need to be certain how much bandwidth in kbps is allocated to priority queues such as voice and video.

There's a bit of subtlety to using the latter two commands though. Bandwidth percent sets reserved bandwidth for a class as a percentage of the total interface bandwidth. Again, IOS will check the total value set using these commands within a policy map to ensure it doesn't violate the value set using max-reserved-bandwidth (default 75%).

Bandwidth remaining percent defines bandwidth as a percentage of remaining bandwidth. However, the calculation changes when LLQ is used. Remaining bandwidth is another name for the product of bandwidth and max-reserved-bandwidth. Taking my earlier example of a 512 kbps link, remaining bandwidth would be 384 kbps. If the command bandwidth remaining percent 50 were issued on a router, 192 kbps would be set aside for that particular class.

All total, there are three different bandwidth related commands in place here: explicit, percentage, and remaining percent. IOS will only allow you to use one of the three methods in a single policy map. The following table is taken direct from the book:

Method                   BW Amt Rsvd               Sum BW Values Must Be <=
Explicit BW            As per command           Interface BW x Max-Rsv-BW
Percent                  % of interface BW       Max Reserved BW setting
Remaining %          % of reservable BW    100 %

Thursday, April 7, 2011

Add'l Reading: QoS classification and marking

Some notes that perhaps didn't get clearly stated in my first posts about marking traffic with ACLs versus using the match command. If you use an extended ACL, you get all the options that format presents for matching traffic. This includes IP traffic (Source/Destination IP address, ToS, IP precedence), ICMP, IGMP, TCP, UDP, and of course DSCP.

Using the match command allows you to directly specify the IP Precedence and DSCP fields without using an extended ACL. However, other syntax options give you the flexibility to identify traffic based on source/destination MAC address, MPLS experimental bit, CoS, input interface, RTP UDP port numbers, QoS group, NBAR protocol types, NBAR Citrix applications, and hostname/URL strings.

IP Precedence and DSCP Values
I looked back at my notes and realized I have not yet transcribed these. This table is excerpted from the Cisco QoS Exam Certification Guide. The key to remembering these is of course the binary value arrangement. If you try and use the decimal, you're missing the structure of the design!

Note that under the original IP Precedence design, larger decimal numbers got better QoS treatment. This isn't the case with DSCP values (notably EF) due to the binary value assignments. Also, below you'll see CS values- the meaning of this is "Class Selector"which is the DiffServ name for the backwards-compatible values that match up to IP Precedence.

Field & Value (Decimal)                  Binary         Name                                    
Precedence 0                                   000            Routine
Precedence 1                                   001            Priority
Precedence 2                                   010            Immediate
Precedence 3                                   011            Flash
Precedence 4                                   100            Flash Override
Precedence 5                                   101            Critical
Precedence 6                                   110            Internetwork Control
Precedence 7                                   111            Network Control

DSCP 0                                           000000      Best Effort/default
DSCP 8                                           001000      CS1          
DSCP 16                                         010000      CS2
DSCP  24                                        011000      CS3
DSCP 32                                         100000      CS4
DSCP 40                                         101000      CS5
DSCP 48                                         110000      CS6
DSCP 56                                         111000      CS7
DSCP 10                                         001010      AF11
DSCP 12                                         001100      AF12
DSCP 14                                         001110      AF13
DSCP 18                                         010010     AF21
DSCP 20                                         010100     AF22
DSCP 22                                         010110     AF23
DSCP 26                                         011010     AF31
DSCP 28                                         011100     AF32
DSCP 30                                         011110     AF33
DSCP 34                                         100010     AF41
DSCP 36                                         100100     AF42
DSCP 38                                         100110     AF43
DSCP 46                                         101110     EF

Notes from the Cisco Enterprise QoS SRND
Cisco's text excellently defines QoS as being a combination of 3 items: delay (latency), jitter (variation in the amount of latency), and packet loss. One excellent way to get a real life insight into jitter is to grab a packet capture of RTP and examine it in Wireshark (www.pcapr.net has several). Under Telephony -> RTP -> Show All Streams you will get a window with various statistics on max and mean jitter.

Tuesday, April 5, 2011

Summing Up Classification and Marking, with Add'l References

The end of chapter summary just revisits commands, recommends further reading, and provides definitions. It also suggests you "fill in key tables from memory." After reviewing the original posting, I have concluded that while this chapter of the book provides a decent review, the book as a whole fails to provide a good overview of how marking (Ch 12), congestion avoidance (Ch 13), and shaping (Ch 14) all work together. So, I highly recommend all the additional reading discussed below.

I am omitting the recap of commands because I've already covered them in prior posts. However, I find their table on DiffServ RFCs useful so here it is-

RFC       Title                                                               Comments
2474      Definition of the DiffServ Field                         Contains details of the DSCP field in IP headers
2475      An Architecture for Differentiated Services      Core DiffServ concepts document
2597      Assured Forwarding PHB Group                     Defines 12 DSCP values and usage convention
3246      An Expedited Forwarding PHB                       Defines a DSCP value for use as a low-latency class
3260     New Terminology and Clarifications for DiffServ   Clarifies but doesn't supercede existing RFCs

The book also recommends you be able to write down definitions for the following terms:
IP Precedence, ToS byte, Differentiated Services, DS Field, Per Hop Behavior, Assured Forwarding, Expedited Forwarding, Class Selector, Class of Service, Differentiated Services Code Point, User Priority, Discard Eligible, Cell Loss Priority, MPLS Experimental Bit, class map, policy map, service policy, Modular QoS CLI, Class Based Marking, Network Based Application Recognition, QoS preclassification, AutoQoS.

Recommended further reading includes-

Cisco QoS Exam Certification Guide (2nd Edition), by Wendell Odom
End to End QoS Network Design, by Tim Szigeti
For real life QoS deployments, read the Enterprise QoS SRND Guide, posted at Cisco's website

My next post will summarize a few pertinent points from the Cisco QoS exam certification guide, as well as the QoS SRND Guide.

Monday, April 4, 2011

More on AutoQoS

AutoQoS VOIP on Routers
Meant for use on LAN ports. The command to enable AutoQoS on interfaces is auto qos voip [trust] at the interface or frame relay DLCI level. You'll also need to have the bandwidth command in place before enabling AutoQoS; changing it later won't affect the QoS setup so you'd need to rerun AutoQoS in that instance.

When you enable AutoQoS on a frame relay interface slower than 768kbps, it also enables compression and fragmentation. Regardless of the speed, a router will still perform the standard config for traffic shaping and apply the policy automatically.

On serial interfaces that aren't frame relay which are slower than 768kbps, AutoQoS changes the encapsulation to PPP and enables the features link fragmentation and interleaving (LFI). Interfaces that are faster will retain their original encapsulation.

When the trust keyword is used, the router groups traffic (via class maps) based on DSCP values. Those DSCP values are already present in packets and are assumed to be set by a trusted device(s). Without the trust keyword, the router creates class maps for voice and video traffic as well as call control ports. Those types are mapped into classes and any traffic not matching them is marked with DSCP value 0.

Verifying AutoQoS for VOIP 

show auto qos   displays QoS commands for an interface
show mls qos     shows queuing and mappings for CoS-DSCP and vice versa
show policy-map interface  displays how the policy maps are assigned to interfaces and details of each

AutoQoS for the Enterprise
Supported on Cisco routers; the main difference between this and AutoQoS VOIP is that Enterprise supports other applications and is meant to be used for WAN links. May be configured on Frame and ATM sub-interfaces if they are point to point links only. This has two steps- discovering traffic and a resulting QoS config.

Traffic discovery takes place on interfaces, so the command is run at the PVC, DLCI, or interface config level. CEF must be on, QoS should NOT be already in place on the interface, and you should have the bandwidth command specified. The command used to enable discovery is auto discovery qos [trust] and the trust keyword is used (as stated before) to accept existing QoS markings on traffic.

Activating traffic discovery enables NBAR (for that interface). Ten classes are created and traffic is classified into one of the ten. Default classification with the trust keyword is as follows (sample traffic is listed for each)-

Class                     DSCP/PHB Value             Traffic Examples
Routing                       CS 6                              EIGRP, OSPF
VOIP                          EF (DSCP 46)              RTP Voice Media
Interactive Video         AF 41                            RTP Video Media
Streaming Video          CS4                              RealAudio, Netshow
Control Signaling         CS3                              H323, SIP
Transactional              AF21                             SAP, Citrix, Telnet, SSH
Bulk                           AF11                             FTP, POP3, SMTP, Exchange
Scavenger                  CS1                               Peer to peer applications
Management               CS2                              SNMP, Syslog, DNS
Best Effort                  All others                       All other traffic

Disabling AutoQoS for an interface disables NBAR on there as well. The related command is no auto discovery qos. To validate that QoS is enabled, use the same commands as for AutoQoS VOIP. The only new command is show auto discovery qos, which displays NBAR statistics.