The MPLS WG Archive

[Eric Osborne <eosborne@cisco.com>]

On Sat, Dec 07, 2002 at 09:14:19PM +0800, Jing Shen wrote:
> Thanks for Eric's comment. And, read my comment in line please.
> 
> 
> > >   1) TCP throughput of MPLS is a little smaller than IP when path MTU is
> > > the same.
> > >      and difference increases when path MTU decreases.
> > 
> > MPLS packets are 4 bytes (or more) bigger than IP packets.  If you do
> > not allow for this in your link MTU, then you have smaller effective
> > bandwidth inside TCP.  Allow for that in your link MTU (as any
> > reasonable implementation should) and you don't have this problem.
> 
> Yes. We notice that at the first stage of our implementation and then we
> modify the engine to allow for such situation. Our implementation allows
> for a label stack with depth of 8, and we use only one. But, that do has
> effect on TCP throughput if the same path is taken by IP and MPLS. 
> 

Have you been able to determine what made this impact?  If your TCP
segments can open up to the same size under both MPLS and IP, then all
you add is another 4 bytes of transmission delay, which should be so
close to zero it's inconsequential.  And if everything has this small
additional delay, then that should in no way create jitter in the
network.

> 
> > So this is a test of your forwarding equipment.  Equipment which takes
> > zero or negligible time to perform this encap/decap will not have this
> > problem.
> 
> As I know some of manufacturers announce that their router/switch
> could operate at line speed, and the latency in a switch is
> negeligible. But, what about situation of heavy load?  The ability
> of a linux based forwarding engine is surely not at the same level
> of any hardware based product, but to my personal opinion any
> comparation should be taken at the same situation ( hardware,
> software etc).

A switch that operates at line rate can still do so under heavy load.
There are well-known mechanisms (DiffServ) for controlling performance
under heavy load.  Theoretically, there should be no difference in the
performance of DiffServ whether applied to IP, MPLS, or for that
matter, Appletalk, IPX, or any other switched packet technology.  If
you have found a platform-independent issue inherent to MPLS that
causes jitter, I am very interested in seeing it.

>  Another reason is that we try to find out how MPLS compares with IP
> when realtime multimedia content is loaded. The bandwidth
> requirement of realtime distributed virtual reality system is at the
> level of Gbps, and that value may increase when number of people
> participating increases.  So, we want to measure the situation that
> accumulated dec/enc time is not negligible.

OK, that's a fair thing to want to measure.  But again, you're testing
a specific implementation, and it's not reasonable to extend that
comparison to any broader scope.

> 
> Well, if R12000 is as cheap as 2511 that's not a question at all :-)
> 

OK, if you want to talk specific router models, let me speak to the
ones I know best. :) A 12k and a 2500[*] are different.  They perform
differently, are priced differently, and have different interfaces and
capabilities.  I would never do forwarding performance tests on a 2500
and claim the 12k behaves the same way, nor would I do tests on 12k
forwarding and claim the 2500 performed the same way.  By the same
token, you can't test a Linux forwarding implementation (a lot like a
really fast 2500) and claim that its performance is the same as any
other router maker.

[*] - sidebar to the viewing audience - a 12k (GSR) is a Really Big
router, and a 2500 is a Really Small router.

> > >  Although we have not experimented with Cisco's express forwarding
> > > and the like, and we do not experiment with large scale system with
> > > real internet traffic pattern, we concern that the result could
> > > applied to large scale network.
> > 
> > I'm concerned about that too, because it means that you're building a
> > large-scale network out of Linux-based software-forwarding devices.
> > But even if you do have a performance hit at encap/decap, you only do
> > label push/pop at the edges of the network, so you'd only take this
> > hypothetical hit at the edges, making scale a nonissue for your
> > particular imaginary problem.
> >
> 
> I'm sorry I means label stack operation by encap/decap. That is done at 
> each hop. As I mentioned above, if the load is light there is no
> difference
> but when the load is heavy the difference comes up.

Then what you're testing is the per-hop performance hit taken by label
swapping.  This is, again, very implementation-dependent.  In MPLS
forwarding you have to do a label lookup and label swap, in IP you
have to do a FIB lookup.  Whether these two are the same speed,
insignificantly different, or significantly different is a
platform-specific thing.  


> > >  2) we focus on forwarding difference between IP and MPLS, and we
> > > think technique like CEF and per-port cache could be applied to both
> > > technologies.
> > 
> > The fact that you're considering different lookup methodologies for IP
> > and MPLS suggests even further that what you're profiling is your
> > implementation, and not either
> >                 a) a purely hypothetical situation (a la OPnet or
> >                 whatever)
> >                 b) a particular router implementation
> 
> I agree with you that there is difference between lookup method
> between IP routing and MPLS switching, this may introduce some
> performance difference.

Right, but like I said above, the switching mechanism is not a
function inherent to the protocol.  


> > 
> > I have no problem with you testing your own implementation; I have a
> > major issue if you try to generalize those results to claim that your
> > specific behavior is in fact an artifact of the protocol itself.
> > 
> 
> I agree that implementation introduces performance difference. So,
> we take the experiments with the same hardware and software. The
> conclusion could not be extended to the word like " any
> implementation of MPLS has smaller jitter than any implementation of
> IP", and the switching fabric in commerical product is nearly
> impossible to be congested. But, what about the accumulated delay
> along a heavy loaded path?

If the per-hop delay due to implementation differences in MPLS
processing vs. IP processing is 0, then the accumulated delay is also
0.  If the per-hop delay due to different protocols is non-zero but
insignificant (let's say 1us per hop), then the total accumulated
delay is going to be so small that it's not noticeable by any
application using the network, and it will pale in comparison to other
sources of delay that must exist in this network.  If the per-hop
delay is significant, then yes, the total end-to-end delay will also
be significant.


> > 
> > I don't understand point #4.  What is 'level of cost'?
> 
> I mean that the forwarding engine show has the same building cost.
>

Cost level has little to do with it.  Nothing about either IP or MPLS
forwarding inherently takes more money to do.  The waaaaay-back,
long-ago argument that MPLS is faster at switching because it's a
20-bit lookup rather than 32-bit is obsolete.  There's also nothing
inherent to the cost of building a forwarding device that induces any
jitter, nor is there anything inherent that introduces different
jitter per protocol.
 
> > 
> > Any chance of getting a version in English?
> 
> I'm sorry we have not done a english version.
> If you like I'd send you the experimental result. 

Sure, I'd be interested to see whether there's more to this than I
think.



eric 

> Thanks again. 
> 
> > 
> > eric
> > 
> > >
> > > regards
> > >
> > >
> 
> -- 
> Jing Shen
> 
> State Key Lab of CAD&CG
> ZheJiang University(YuQuan)
> HangZhou, Z.J. 310027
> P.R.China
> 
> Tel: +86-571-87932423
> Mobile: (0)13516813753
> Email: jshen@cad.zju.edu.cn
> 
> **********************************************************************
> * The SunShine of life is made up of very little beams which is      *
> *  bright all the time                                               *
> **********************************************************************