Data center evolution is driving demand for ever-higher data transmission rates, necessitating innovation in data center optical interconnect component design. 

By Robert Bruce and Peter Hook, Iridian Spectral Technologies

Demand for data center storage and data transfer capacity has increased dramatically over the past decade¹, exponentially increasing the load placed on backplane-to-backplane data server interconnects within each center.

Further, demand for data center interconnect bandwidth will continue to grow, driven by several factors. These factors include improved technologies (e.g., flash memory and solid-state drive) that enhance the attractiveness of cloud storage, as well as requirements for dynamic allocation of server, storage, and network resources.

Both commercial and consumer users expect continuous data availability, facilitated by distributing virtual computing and storage resources across numerous physical devices, and they expect ever-increasing speed attached to that access. Since a single request can trigger multiple data exchanges between servers in one or more data centers, cloud providers’ warehouse-scale data center servers have had to keep up, progressing from 10 GbE to 100 GbE network adapters in common use, with 800 GbE (and even to 1.6 TbE) forecasted to become standards within the next few years.²

In a large data center utilizing typical Clos topology (Fig. 1), hundreds of thousands of interconnects are required to maintain efficient communication across servers not only in that facility, but in data centers in other locations. Clos provides a more direct interconnection between top-of-rack (TOR) switches and other servers, but every leaf switch in the leaf spine architecture connects to every switch in the network fabric. The spine switches have the same level of connection to the leaf switches as the leaf switches to the TOR switches.

Figure 1 — Clos Topology

In response to ever-increasing bandwidth demand and evolving data center architectures, the copper connections traditionally used to connect backplane to TOR switches and TOR to leaf switches have migrated to optical interconnects to accommodate the higher data rate required.³ Consequently, data center designers and operators constantly seek more capable optoelectronic transceiver components that are cost effective, compact, and energy efficient. While traditional transceiver components used at these data rates in long-haul systems were once considered an option, they are ill-suited to data center use since they are configured for longer-distance interconnects (leading to higher cost),often consume more power, and generally have a larger form factor.

Optical interconnect bandwidth enhancement using single-mode fiber (SMF), multi-mode fiber (MMF), digital coherent optical transmission, dense wavelength division multiplexing (DWDM), multiple spatial modes, and other techniques all have been leveraged to boost optical interconnect data transfer rates to levels approaching those of the long-haul fiber connections. This is because all data passing through long-haul fiber has to be distributed around data centers at the same speed or better.

The optical transceivers used at each end of the fiber interconnect often use optical filters to manage the different wavelength channels used in WDM, CWDM, DWDM, and other multiple-wavelength configurations. Filters used in these shorter interconnect applications (≤ 1 km) utilize the same base filter coating technology as filters used in long-haul and metro applications. However, the optical design and filter size and thickness are adjusted to meet more compact data center products’ unique requirements.

In addition to small, precise form factors, designers integrating these parts require high transmittance and minimal signal loss — as well as high reflectance, if operating in reflection — ensuring they do not have to increase amplification unnecessarily. Related, energy consumption is critical in this application, and using an optimal optical interconnect maximizes energy efficiency while minimizing the associated heat dissipation necessary.

Custom filter solutions — including the typical single-wavelength bandpass and edge pass units used in WDM systems, as well as DWDM edge filters — have emerged as a cost-effective and reliable solution to meet these needs. Etalons or other optical filters also may be used in the integrated laser assembly (ITLA) sources used in these systems.

Limited Space ≠ Limited Options

A server backplane features many connectors, meaning every component containing a filter must be as small as possible to minimize the volume it occupies. In large server farms, hundreds or even thousands of servers occupy a space whose maintenance cost per square foot is expensive, since the vast space must be kept temperature-controlled and physically secure. Consequently, multiplying space savings (e.g., trimming 30% or 40% of each component’s physical volume) across hundreds of thousands or even millions of components has a significant impact. Distributed data centers located in urban centers — small locations interconnected to form, essentially, one big data center — share the same space-saving requirement due to the high cost per square foot of real estate in these more populous locations.

The cost to manufacture such products at the necessary volumes has driven a significant amount of component and filter manufacturing to Asia. However, Iridian Spectral Technologies leverages 20+ years of filter design experience, proprietary design tools, and automation throughout part manufacture, cutting, and testing to maximize efficiency and part quality while continuing to manufacture in North America. This investment in automation enables us to produce etalons and filters at high volume and consistently high quality while minimizing per-unit cost. We regularly sell our product to low-cost manufacturers in Asia and elsewhere as a result of this combination of design expertise and manufacturing excellence.

Iridian Spectral Technologies’ design process typically begins based on the client’s stated wavelength preferences: which wavelengths need to be multiplexed and how they will be spaced (e.g., do they require 5 nm, 10 nm, or 20 nm spacing between each laser wavelength?). Additionally, the client must indicate the desired tolerance on those sources, which helps us determine how much bandwidth is needed. From there, we examine conditions of use — particularly the angle of incidence and the beam’s cone angle. Next, we start examining form factor and part size based on where the customer needs the component to fit, taking into consideration both the available space for the component and the typical operating temperature (though, given that data centers typically are air-conditioned spaces, temperature generally constitutes a minor concern).

In design scenarios like the one described above, we collaborate directly with the client’s chief technical officer or technical team to provide a filter solution before progressing to initial production. At this point, the client may evaluate the part and provide feedback that is applied to another sample round or move directly to high-volume production.

Even exercising this level of meticulousness, Iridian Spectral can ship filters in as little as four weeks (from receipt of customer specs to initial delivery) and etalons in about eight weeks. We strive to turn around a design and quote within two days, as well, and ramp up unit delivery into the tens of thousands per week (i.e., volume production) within eight to ten weeks. Consider the following examples:

• Design Changes — A recent high-volume customer sought to transition from a free-standing component — where free space exists on both sides of the filter product — to a much thinner part featuring an epoxy mount on the backside (i.e., epoxy applied directly to either send or receive optics). This feature allowed the client’s module to be smaller than a free-standing, edge-mounted orientation. The design presented challenges related to the cone angle (i.e., how the light is coupled into and out of the filter) that we addressed through an iterative process with the client before ramping up to high volume.
• Volume Demands — A recent Iridian Spectral client required shipping of 10,000 parts a week: etalons measuring 7mm x 1.1mm.
• Precise Form Factors — A recent Iridian client requested a part with lateral dimensions 0.5mm and smaller, for which we had to upgrade our equipment. The limiting factor in that case was handling after dicing; we modified the vacuum pickup to accommodate this smaller size. Handling and packaging are the principal challenges with miniaturized components, as is (in some cases) optical characterization. When creating smaller form factors, optical characterization typically is performed at the wafer level, after which the “good areas” are mapped out, allowing us to utilize those good areas and pack them for clients.

We also serve as an additional or replacement source for clients whose product already is manufactured in high volumes — organizations looking for a supplementary partner to produce an identical part meeting all the same specs as the original, often at lower cost or with higher package yield. We’ve matched, as much as possible, our in-house language skills to our clients, so we’re well-equipped to deal with customers around the world, having developed habits that allow us to work efficiently with them remotely, even before the pandemic.

Final Thoughts

The evolution of data centers for cloud storage and distributed computing has driven the need for ever-higher data transmission rates within data centers. This, in turn, has necessitated innovation in optical interconnects — both their designs and the manner in which they are produced to meet high-volume demand.

Iridian Spectral Technologies draws on more than two decades of experience to provide specialized optical filters and optoelectronics used in these centers. Our automated manufacturing, cutting, and testing systems are shared across all of our products and product areas, ensuring maximum efficiency not only for this market area, but across our business. We can load and produce a part for one client one day and manufacture another client’s product the next day, using the same equipment with little or no reconfiguration — and without sacrificing quality or volume.

Further, we utilize several different coating platforms, allowing us to apply whichever is most appropriate to the part being produced: larger or smaller optics, higher- or lower-precision specifications. We are adaptable to the most efficient platform for the application. Because we work in numerous markets, we have a range of tools from which we can choose to arrive at the most cost-effective solution, meeting the needs of the customer based on the requirements of the product.

To learn more, visit us at www.iridian.ca and follow us on Twitter (@IridianSpectral).

About The Authors

Robert Bruce is VP of Business operations at Iridian Spectral Technologies. He attended McMaster University in Hamilton, Ontario (Canada), earning a M.Eng. Metallurgical Engineering in 1985.

Peter Hook has been a Senior Applications Engineer with Iridian Spectral Technologies since 2006. He earned a MSc from Queens University’s Department of Physics in 2004.

About Iridian Spectral Technologies

Iridian Spectral Technologies is a world leader in the design and manufacture of custom optical filter solutions. Using advanced, proprietary thin-film design deposition and manufacturing technology, Iridian delivers durable, high-performance optical filters for use in applications including telecommunications and data centers, Raman and fluorescence spectroscopy, mid IR gas sensing, satellite-based imaging and communications, and many more.

References

1. Zhou, X., Optical Fiber Technology (2017), http://dx.doi.org/10.1016/j.yofte.2017.10.002
2. Ethernet Alliance, 2020 Ethernet Roadmap, https://ethernetalliance.org/technology/2020-roadmap/
3. Qixiang Cheng, Meisam Bahadori, Madeleine Glick, Sebastien Rumley, and Keren Bergmen, Optica, Vol. 5, No. 11 / November 2018 / Optica, p 1354-1370 https://www.osapublishing.org/optica/fulltext.cfm?uri=optica-5-11-1354&id=399361

Additional Resources

• Chongjin Xie 978-1-5090-5016-1/17- 2017 IEEE Optical Interconnects Conference (OI) Pages: 37 – 38
• Joseph M. Kahn and David A. B. Miller, NATURE PHOTONICS | VOL 11 | JANUARY 2017 | www.nature.com/naturephotonics.