The technologies and approaches underpinning the future of the data centre

The future of the data centre and the technologies underpinning it has become a central topic of discussion in microelectronics – and from co-packaged optics, to AI-driven infrastructure, two panel sessions at Microelectronics UK in September explored where – and how fast – things are going, with representatives across hyperscalers, telco, and industry bodies alike.

Caroline O’Brien, CEO at Semiconductor Catapult – which had rebranded from CSA Catapult at the event – made a stark prediction. “I think in 10 years’ time, 15 years’ time, we’ll have a completely different ecosystem and architecture of data centres,” she told attendees, “and I think when you get those discontinuities in the market, that’s the real opportunity to not only decrease power consumption but also improve things like latency.”

Part of O’Brien’s reasoning is personal – the Catapult is based in Newport and, in her words, ‘dwarfed’ by the hyperscalers at nearby Imperial Park – but the sustainability question and the need for greater workload cooling is a practical one.

Matthew Povey, senior technologist at AI and semiconductor hardware company SambaNova, offered an important point. It’s one thing to make sure the data centre itself is using resources like power and water effectively, Povey noted, but the chips need to be matched to the workloads. “It just comes back to heterogeneity,” said Povey. “I think if you’re in the semiconductor industry, this is an awesome time to be here, because for the first time in 20 years, we can go and target different workloads.”

Looking at the measurement of power as the industry grapples with the growing demands of AI, and whether PUE (power usage effectiveness) still works, the panellists noted the importance of efficiency. “The real question – and there is no question that this is hard to define, is what intelligence are we getting out per megawatt of energy that we put in?” said Povey. “One way or another, that is the measure we should be going after.”

Both Povey and Pushparaju Thangavel, senior solutions architect at Amazon Web Services (AWS), explored unused capacity. Povey described it as the ‘marginal data centre’; the ‘20% left over that can’t really be used for a training cluster but can… for smaller inference clusters’ for instance. Thangavel noted there was a good research work waiting to be conducted on this topic.

“One of our pictures that we have in our minds is based around the new towns that are being discussed and how they could have a sustainable data centre, an edge data centre nearby, that could be one of the utilities servicing that data centre, but also with the opportunity to use more localised power – even potentially put power back to the grid when there’s a surplus,” said O’Brien.

Liam Round, chair of the advisory council at The Data Centre Alliance, moderating the panel, moved the conversation to scalability. What is the primary limiting factor to the next generation facility? Power availability, grid infrastructure, capital costs, supply chain, or something else?

“Everything else like cooling, or the chips, or supply chain – all these are engineering problems,” said Thangavel. “You can solve it, but specifically on the utilities, the grid infrastructure management, and bringing the actual power, is very challenging when it comes to any data centre build-out.”

Ultimately however, Microelectronics UK is all about identifying key sector challenges and connecting the technology providers who can help. As O’Brien noted, Semiconductor Catapult aims to do just that, and when asked what she would do to redesign the data centre from a blank sheet of paper, O’Brien replied that it was to accelerate companies to get their technologies into the data centres in whatever architecture needed to solve these fundamental problems, from compound semiconductors to silicon semiconductors.

O’Brien had also mentioned adoption of photonics and co-packaged optics – and at a panel session on the Photonics UK stage around harnessing the potential of optical computing, this exact area was explored in detail. Optical compute and optical networking are transforming data centre architectures, scaling infrastructure for AI and cloud workloads – and participants from AWS and Microsoft were able to outline the hyperscaler perspective.

Andrew Lord, senior manager, optics and quantum research at BT, who moderated the session, asked Malav Mehta, network deploy build lead at AWS, whether there was a particular level of demand for optics seeing and why. “Compared to last year, the scaling has been doubled down with AWS about every kind of the rack,” said Mehta. “We are doubling down the scaling, so we will be needing the optical components over there.”

Mehta noted that it was not only about GPUs. “We want GPUs to communicate with each other, especially the clusters of GPUs as well – so the more the GPUs, the more the optics,” he said. “On top of that, as we are moving towards 800G to 1.6T, I believe that the industry is yet to be expanded further.

“The core reason why we need more and more optics is because we don’t want to optimise networking. We want networking to not be a bottleneck for AI deployment.”

From the Microsoft side, Kirill Kalinin, senior researcher at Microsoft Research, noted the company was ‘trying to think from the whole stack… from the communication between the data centres and, for example, the hollow core fibre (HCF) that can help to make it more efficient, faster, in the data centres.’

Microsoft has invested heavily in HCF, most notably from a UK perspective acquiring Lumenisity, originally spun out from the University of Southampton’s Optoelectronics Research Centre. Kalinin noted the ‘big investment’ and ‘really exciting demonstrations’ from Microsoft before painting the picture. Imagine one power station which powered 50,000 GPUs that you put in one data centre, with frontier model training taking a couple of months. HCF would enable another data centre to be put in 10-20 kilometres further away, in a neighbouring power grid with another power station – and the result would be double the levels of compute.

Lord then put it simply: are we looking at optical computing yet, or just ‘very high-speed plumbing’? Kalinin defined it thus: “There is so much effort in academia, in the startups, and in Microsoft about optical compute, and we believe that it maybe will not replace the digital chips, but can be very much complementary for certain workloads,” said Kalinin.

The road ahead, however, may be one which involves a combination of optical and electronic processing. Lord asked Kalinin whether the likes of Nvidia could, in the near- or mid-term, integrate this kind of photonic technology into their GPUs.

“I think for any independent optical company developing the chips, it’s very difficult to think of completely replacing the GPUs,” said Kalinin. “The near-term strategy likely will be where you will combine some of the processing happening in the digital compute and the optical compute.

“The combination of optics for compute in low-bit precision, digital stuff for high-bit precision computations, but maybe a smaller chunk of these computations and some non-linear stuff… that kind of synergy is probably what we are looking after in the near-term.”