Safeguarding Tech Investments in the Age of AI: Insights from RED ICT

In 2024, RED ICT hosted a series of industry Q&A panels, thought leadership workshops, and one-on-one interviews to explore the rapidly evolving landscape of technology infrastructure.

These sessions revealed a growing sense of uncertainty among decision-makers tasked with planning for the future of data centres and tech investments. Among the attendees were data centre owners, developers, occupiers, operators, investors, and service providers—each voicing their unique concerns and priorities in an era where technology is advancing at an unprecedented pace.

What became clear is that the decisions being made today are heavily influenced by the transformative power of artificial intelligence (AI), alongside the remarkable evolution of both CPU and GPU performance. This article delves into the feedback from industry leaders, the challenges they face, and the opportunities presented by these cutting-edge technologies.

The CPU Revolution: Moore’s Law and the Data Explosion

Over the last decade, advancements in CPU technology have been primarily driven by Moore’s Law, which predicts that the number of transistors on a microchip doubles approximately every two years. This exponential growth has unlocked remarkable gains in processing power, storage, and efficiency. Key developments include:

• Increased Transistor Density: Modern integrated circuits now incorporate far more transistors, enabling chips to process and store vast amounts of data in smaller footprints.
• Advancements in NAND Flash Memory: Technologies like NAND flash memory and MRAM have drastically improved storage capabilities without expanding the physical size of chips.
• Cost Efficiency: The cost per gigabyte of storage has plummeted, making advanced technologies accessible to both enterprises and consumers.

These advancements have fuelled an explosion in global data creation, which is projected to reach a staggering 394 zettabytes by 2028. As data volumes continue to grow, the need for robust infrastructure to support this expansion becomes ever more critical. RED ICT is uniquely positioned to help stakeholders navigate this challenge, ensuring their infrastructure investments are future-proofed and scalable.

The GPU Renaissance: Powering AI in the Last Five Years

While CPUs have driven the broader technology landscape, GPUs have taken centre stage in the AI revolution. Over the past five years, GPU technology has undergone exponential growth, revolutionising the way AI systems are trained and deployed. Key trends include:

• Exponential Growth in Compute Power: The compute capabilities used to train AI systems have doubled approximately every six months, far outpacing Moore’s Law.
• Technological Advancements: Innovations in GPU designs—such as Tensor Processing Units (TPUs) and Neural Processing Units (NPUs)—have propelled GPU performance to unprecedented levels. For context, GPU performance has increased by an astonishing 7,000 times since 2003.
• Market Expansion: The AI chips market, valued at $53.7 billion in 2023, is expected to grow by over 30% in 2024, reaching more than $71 billion.

These advancements are not just about speed—they are reshaping entire industries. From real-time ray tracing for photorealistic graphics to specialised AI cores capable of executing trillions of operations per second, GPUs are setting the stage for a new era of technological capability.

Looking Ahead: The Future of GPUs by 2030

As we move toward 2030, the evolution of GPU technology promises even more transformative breakthroughs. Here are some trends that will shape the future of GPU performance and its role in AI and beyond:

1. Exponential Growth in Performance: GPUs could be 5-10x faster than today’s most powerful chips, thanks to advancements in transistor densities, chiplet-based designs, and AI-specific processing cores from leaders like Nvidia, AMD, and Intel.
2. AI and Machine Learning Optimisation: Future GPUs will feature dedicated AI cores capable of executing operations at unprecedented speeds, revolutionising AI inference and training.
3. Heterogeneous Computing: GPUs will increasingly integrate specialised cores for tasks like deep learning, ray tracing, and quantum-assisted computations, enabling them to handle highly specific workloads with unmatched efficiency.
4. Memory and Bandwidth Increases: Next-generation memory technologies, such as HBM4 and GDDR8, will deliver bandwidths exceeding 10 TB/s, enabling instant data transfers and ultra-high-resolution processing.
5. Energy Efficiency: Advanced cooling solutions, optical interconnects, and 3D stacking technologies will ensure GPUs remain efficient even as their power demands increase.
6. Photorealistic Graphics: Real-time ray tracing and AI upscaling will make 8K+ gaming at high refresh rates a mainstream reality.
7. Quantum and Neuromorphic GPU Enhancements: Emerging technologies like neuromorphic computing and quantum acceleration could enable GPUs to process AI workloads millions of times faster than today’s systems.

These advancements will empower organisations to tackle challenges in AI, big data, and high-performance computing with greater efficiency and innovation than ever before.

Cooling Innovations: The Key to High-Density AI Infrastructure

As power demands rise, cooling systems must evolve to ensure optimal performance and efficiency. High-density cooling solutions, capable of supporting racks with power densities exceeding 25 kW, are now essential for deploying AI-accelerated hardware. Traditional air-cooling systems, while still in use, are increasingly inefficient at these levels, driving the adoption of advanced liquid cooling technologies. Key innovations include:

• Rear-Door Heat Exchangers: These systems dissipate heat directly from the rack, improving efficiency.
• Direct-to-Chip (DTC) Cooling: Coolant is applied directly to the hardware, ensuring precise thermal management.
• Immersion Cooling: Entire servers are submerged in non-conductive liquids, offering maximum cooling efficiency.

These solutions not only address the thermal challenges posed by AI workloads but also support the growing density of AI clusters. For example, the NVIDIA SuperPOD can house up to 32 server units in a single row, each equipped with 8 GPUs. Such systems require 384 fibre links and 64 copper links per row, underscoring the need for high-speed, high-bandwidth communication infrastructure.

Beyond Power and Cooling: Weight, Latency, and Market Trends

The rise of AI has ripple effects across all aspects of data centre infrastructure. As servers become more powerful, additional considerations come into play:

1. Weight: Racks incorporating liquid cooling systems can weigh up to 2,500 kg per rack, compared to the current OCP standard of 1,600 kg. This increased load requires reinforced designs and careful planning.
2. Latency: AI algorithms are highly sensitive to latency, requiring 3-5x more connectivity than standard cloud applications. Fibre connection architectures must meet stringent specifications to ensure seamless communication.
3. Market Growth: Generative AI investments are expected to rise from 19% in 2023 to 34% in 2024. Nearly half of Asia-Pacific organisations plan to increase their technology investments by over 20% in the next 12 months, further accelerating the demand for AI-ready infrastructure.

Navigating the Uncertainty: RED ICT’s Role as a Trusted Partner

While technological advancements are exciting, they also come with challenges. Data centre stakeholders must grapple with questions about scalability, cost, and long-term sustainability in an environment where the pace of change can be overwhelming. This is where RED ICT shines.

At RED ICT, we bring together industry expertise, cutting-edge insights, and a collaborative approach to help our clients safeguard their technology investments. By staying ahead of industry trends and leveraging the latest advancements in CPU and GPU technology, we empower our clients to make informed decisions that align with their business goals. Whether you’re a developer, operator, or investor, RED ICT ensures your infrastructure is not only capable of supporting today’s demands but also adaptable to the needs of tomorrow.

Conclusion: Preparing for the Future of AI and Infrastructure

The era of AI has ushered in transformative advancements in CPU and GPU technology, reshaping industries and redefining what’s possible. However, with these opportunities come challenges, particularly in planning and building infrastructure that can keep up with the pace of innovation. At RED ICT, we are committed to helping our clients navigate these dynamic market trends. With our involvement in 50 data centre projects across 17 countries, we ensure that organisations are well-equipped to meet the growing demands of technology investments. Our expertise positions us as a trusted partner in driving innovation and supporting the future growth of AI infrastructure.