Further enhancing the RapidIO Interconnect Architecture, Serial RapidIO 2.0 offers system designers improved abilities to control, optimize, and accelerate traffic using Virtual Channels (VCs). By implementing techniques explored in this article, future switches can enable VC features even in Serial RapidIO 1.3 systems and facilitate the evolution to Serial RapidIO 2.0 systems.

Serial RapidIO is a high-performance packet-based technology that can be found in an increasing array of applications, including the wireless infrastructure, storage, medical imaging, and military markets. Serial RapidIO’s strength lies in its efficient application-driven features and the breadth of its adoption. The switched fabric continues to offer embedded system designers more compelling features with the Serial RapidIO 2.0 specification.

With the Serial RapidIO 2.0 standard, system designers can select link rates from 1.25-6.25 Gbps and port widths from 1-16x, providing high granularity to select a port’s data rate best suited to individual applications. Beyond the physical layer enhancements, Serial RapidIO 2.0 offers several higher-level features designed to provide greater control over the switch fabric traffic flow.

VCs grant the ability to control how different types of traffic flow through the system. VCs allow system designers to control packet flow by dividing the link into separate channels and assigning packets to a particular channel. The first VC, VC0, is a backward-compatible VC provided in Serial RapidIO 2.0 that operates like a Serial RapidIO 1.3 compliant link. Beyond this, Serial RapidIO 2.0 supports up to eight more VCs (VC1-VC8).

Controlling traffic

VCs offer a variety of enhancements to control data flow in the fabric. Each VC can be guaranteed a portion of the link bandwidth. System designers have control over how multiple traffic types interact and can, in effect, insulate them from one another through bandwidth allocation. Latency-sensitive traffic, such as streaming video, can be allocated a guaranteed portion of bandwidth throughout the switch fabric. This enables system designers to ensure a high-quality experience for consumers because a minimum level of performance can be guaranteed regardless of other traffic present in the switch fabric.

When a VC demands less than its guaranteed bandwidth, Serial RapidIO 2.0 allows other VCs to use that available bandwidth, thus maximizing link utilization. In essence, bandwidth allocation is intelligent, simultaneously ensuring that a greedy VC demanding more than its allocation cannot rob bandwidth from another VC and that bandwidth does not go unused when packets need to be sent.

Figure 1 shows the benefits of VC bandwidth reservation. Three packet streams share a link and are allocated bandwidth such that 10 percent is dedicated to VC_A, 60 percent to VC_B, and 30 percent to VC_C.

 

 

In the first part of the simulation, only VC_A and VC_C need packets transmitted, so their bandwidths increase beyond their allocations to take advantage of the unused portion allocated to VC_B. As shown, VC_C occupies 75 percent of the available link bandwidth while VC_A uses the remaining 25 percent. Serial RapidIO 2.0 allows VCs with packets present to access the unused part of the link in proportion to their bandwidth allocations.

Later in the simulation, VC_B needs traffic sent via the shared link. Serial RapidIO 2.0 allows the switch to rapidly respond to the change in traffic and alter bandwidth usage as needed to match the programmed allocations. In this case, the switch quickly gives an intermittent but latency-sensitive stream such as VC_B its 60 percent allocation. Once the packets from VC_B are transmitted, the switch reapportions link usage to the remaining VCs with packets awaiting transmission.

Changing channels

Serial RapidIO 2.0 provides an additional level of control over link partitioning by offering a function unique to VC0. VC0 can be configured to obey bandwidth reservations. Alternatively, it can be configured to automatically obtain any bandwidth it requires while any remaining bandwidth is divided among all other VCs with packets awaiting transmission. This allows control plane traffic transmitted via VC0 to operate completely independent of data plane traffic and only be constrained by Serial RapidIO 1.3 compliant priority rules.

VCs offer two modes of packet transmission – Continuous Transmission (CT) and Reliable Transmission (RT). RT operates like earlier versions of Serial RapidIO in that retransmitting a packet when it cannot be received makes packet transmission lossless. CT is optimized to achieve low latency for traffic flows that can accommodate packet loss by not performing retransmissions. VC0 supports all defined priorities and operates exclusively in RT mode.

Higher VCs (1-8) can operate in CT or RT mode, allowing customers to optimize the transportation method for different types of data. For example, whereas control plane traffic may require the responses and guaranteed delivery that RT mode provides, data plane traffic (such as from an audio stream) may benefit from CT mode’s reduced latency and possibly suffer if retransmissions are performed.

Switches take on the challenge

Integrating a specification’s new features into present and next-generation systems presents a challenge. Backward compatibility is a cornerstone of the Serial RapidIO specification development process, but promoting progression toward the new standard is equally important. System developers will begin to see more Serial RapidIO 2.0 compliant products in the near future. However, the ecosystem cannot completely transition at once.

Switch vendors will likely be among the first to adopt Serial RapidIO 2.0. Switches serve as the foundation of embedded fabric ecosystems and, in essence, validate new specifications. Processing end points, such as DSPs and FPGAs, may lag in producing new devices compliant to a specification’s latest revision. Switch vendors are thus presented with a unique challenge to produce next-generation solutions and, at least initially, market them into systems limited to current-generation technology based on available end points.

Designers will migrate to end points that support Serial RapidIO 2.0 as existing systems evolve or new ones are initiated. Inevitably, Serial RapidIO 1.3 and Serial RapidIO 2.0 availability will overlap for an extended period of time. Systems that require Serial RapidIO 1.3 subsystems to communicate with Serial RapidIO 2.0 compliant subsystems will likely exist for an even longer period of time.

However, switch vendors can enable many of Serial RapidIO 2.0’s benefits independent of available end points because these benefits are focused on the switching fabric. The advantages of using VCs are so compelling that system designers will desire support for adopting the technology. Thus the challenge for switch vendors is to implement transitional support for VC functionality within the Serial RapidIO fabric in systems that may be dominated by Serial RapidIO 1.3 compliant end points. Switch vendors also must provide a simple transition mechanism so that as systems evolve to contain Serial RapidIO 2.0 devices exclusively, the fabric and its use of VCs will evolve as well.

Taking advantage of VCs in a Serial RapidIO 1.3 system

Switches that support VCs must provide internal circuitry dedicated to routing VC packets and supporting Serial RapidIO 2.0’s bandwidth allocation requirements. Serial RapidIO 1.3 compliant packets can leverage these paths when the switch routes an incoming Serial RapidIO 1.3 packet as if it was a Serial RapidIO 2.0 packet with a higher VC value.

Switch vendors can allow customers to program a mapping protocol to treat incoming Serial RapidIO 1.3 packets as Serial RapidIO 2.0 packets in terms of buffer utilization, switching algorithm decisions, load balancing, and bandwidth reservations. This function gives system designers more control over the fabric defined in Serial RapidIO 2.0, even if they cannot generate higher VC packets, and allows them to dictate that different data flows be treated as though the entire system is using Serial RapidIO 2.0 devices. Figure 2 illustrates this concept.

 

 

Although the devices only generate Serial RapidIO 1.3 compliant traffic, the switch can be configured to treat certain inbound traffic as if it has a higher VC, essentially mapping a Serial RapidIO 1.3 packet to a Serial RapidIO 2.0 packet with a programmed VC value.

One method of telling the switch how to treat a particular packet is using a unique destination identifier. Such an enhancement enables the switch to route the packet to the correct end point while restricting the packet’s outbound flow based on the bandwidth allocation assigned to the VC for which it is mapped. Using this feature, the switch can treat three different end points as VC_A, VC_B, and VC_C generators, and the intended traffic flow to the destination can mimic the flexibility and responsiveness as shown in Figure 1, all without using Serial RapidIO 2.0 end points. Although potentially limiting transmitter-based flow control, this VC mapping capability allows system designers to more closely approximate the intended traffic flow in the system rather than restrict the user to obey only the Serial RapidIO 1.3 packet ordering rules.

Another key element of VCs in Serial RapidIO 2.0 is the ability to operate in CT mode, allowing packets to be dropped in favor of reduced latency. Any migration effort to facilitate VC adoption must address this crucial feature. Unlike Serial RapidIO 2.0, Serial RapidIO 1.3 devices will not accept a packet if buffer space is not available.

Switch vendors can approximate Serial RapidIO 2.0 CT mode operation with Serial RapidIO 1.3 end points by offering a pseudo-CT mode wherein a new packet replaces a packet that needs to be retransmitted. This mode allows customers in a Serial RapidIO 1.3 system to leverage the latency benefit that results from VCs operating in CT mode.

Communicating between Serial RapidIO 1.3 and Serial RapidIO 2.0

As new Serial RapidIO 2.0 end points enter the market, system designers will need a technology that can bridge the two specifications so that subsystems designed for Serial RapidIO 2.0 can leverage their advantages while still communicating with legacy Serial RapidIO 1.3 subsystems. Serial RapidIO switch providers can help system designers in this regard by allowing them to migrate their systems as the ecosystem evolves.

Switch providers can offer another powerful feature – a switch that operates as a translator between Serial RapidIO 1.3 legacy traffic and Serial RapidIO 2.0 VC traffic. Serial RapidIO provides a built-in backward-compatible translation of VC values since the portion of the header that specifies the VC value reuses the priority and critical request flow fields. Although this allows Serial RapidIO 1.3 devices to pass Serial RapidIO 2.0 packets that specify a VC value, it is a static translation that may not meet the needs of system designers who must support both versions of Serial RapidIO in the same system.

However, a switch can operate as an intelligent interconnect between two subsystems that use different versions of Serial RapidIO by offering a programmable translation capability. Consider a system that includes a legacy Serial RapidIO 1.3 system board, a Serial RapidIO 2.0 system board, and a switching board with translation capability, as shown in Figure 3.

 

 

The switch can have a programmable translation function to optionally convert packets between the two systems, enabling system designers to grow into the new specification as subsystems are revised, new subsystems are introduced, and the overall Serial RapidIO 2.0 ecosystem expands to include all necessary components. Rather than simply mapping packets to a VC and changing how the switch handles each packet, this system changes the inbound packet’s header, recalculates the cyclic redundancy check, and essentially generates a new packet.

Accelerating Serial RapidIO 2.0’s success

Serial RapidIO 2.0 promises to be a powerful embedded fabric with robust features supporting all manner of data traffic. VCs are major components that will change how system designers define and control data flow through their fabric. The challenge for switch vendors is to provide customers with powerful Serial RapidIO 2.0 systems that leverage the compelling features of VCs while offering an easy path to evolve into a complete Serial RapidIO 2.0 system as the ecosystem continues to grow.

David MacAdam is an architectural engineer with Integrated Device Technology (IDT), based in San Jose, California, where he focuses on next-generation product concept modeling, analysis, and specification. Prior to this role, David spent two years as an RTL team lead and four years in various aspects of full custom digital design. David received his BAS in Electrical Engineering from the University of Waterloo in 1999. David holds four patents in multiple areas of semiconductor design, with 11 patents pending.

IDT
678-775-2961
[email protected]
www.idt.com

Robert Bishop is a director of technical staff with IDT. He leads a cross-functional team responsible for architecture and algorithm research to support next-generation IDT product definition. During the past 13 years at IDT, Robert has led a broad scope of projects, including global teams developing network search engines and local teams developing dual-port and FIFO memories and logic devices. Prior to IDT, Robert worked at IBM as a custom circuit designer for six years. Robert received his BSEE from the University of Pittsburgh in 1988 and his MSEE from the University of Vermont in 1992. Robert has been awarded five patents, with five additional patents pending.

678-775-2928
[email protected]