Cloud Gaming — The Infrastructure for the Metaverse

Free-to-play gaming services such as Fortnight, Roblox and Warzone have shown us glimpses of what's possible with the current gaming infrastructure. However, the experiences are simplistic and are limited by the capabilities of end user hardware. As a result, there still isn't mass appeal outside of younger demographics.

Cloud gaming services (e.g. GFN, Stadia)  have attempted to decouple end-user hardware from the experience, but the experience tends to be poor when playing with & against players with native hardware (due to input lag). It's also inefficient, where players are essentially renting an individual server in the cloud, often with comparable performance to a console or decent gaming PC.

As a result, uptake of cloud gaming services is poor unless they're bundled with existing services such as PS Plus or Xbox Live. The killer app for cloud gaming are to create cloud native games, with the complexity and fidelity that would be impossible with a home setup.

There are a number of barriers to achieving this, but we are seeing the emergence of startups building the infrastructure for cloud gaming and unlocking a true metaverse.

Distributed Operating Systems

Most applications running in the cloud use what we'd know as a traditional OS, nodes and processes are constrained to a single "machine" and any distributed workload has to be coded explicitly e.g. the classic example of MapReduce to process huge datasets across multiple machines.

In a distributed OS, the distributed is run at the top level and processes become independent of the machine they run on. It's a huge challenge to get this to work.

European startups such as Hadean and Improbable are building in this space, but  have had issues with monetisation. Building this technology is complex and requires significant R&D investment before the tech can be monetised.

Even once built, there are few programmers outside of the OS team that understand how to create applications on the distributed OS. The lack of developer support means that the OS start-up usually needs to develop first-party game engines or custom-built applications to entice game developers to the platform.

Some have even elected to build their own first-party games, a potentially existential risk considering the resources required to launch a successful AAA title.

Improbable and Coherance have attempted to partially solve the problem by moving to a simpler architecture where parts of compute can be offloaded to separate servers. This enables a game developer to keep using engines they know and love such as Unity or Unreal, but can offload computationally intensive tasks (e.g. enemy AI) and coalesce this into a single experience.

It remains to be seen whether this approach will improve the gaming experience or if bottlenecks in the netcode stitching the separate servers together will cancel the benefits out. The approach is more of a stepping stone to a true cloud gaming engine, but could create an on-ramp for developers to explore more ambitious distributed solutions.

Distributed Game Engines

In current cloud gaming setups, an instance of the full game is deployed to a virtual machine in the cloud. As a result the compute available to the game instance is limited by the capabilities of the VM, which is rarely more capable than a top gaming PC or even games console. Networked engines take it a step further where whole components are deployed in a VM.

These approaches work, but they have the downside of rapid degradation of performance if there are any issues with the VMs in use. Even connecting to a cloud gaming server an extra 200 miles away will noticeably impact FPS and lag.

A fully distributed game engine would allow all subsystems of the engine to be dynamically allocated to compute as required. This requires a thorough rethink of how these engines are built, with calls to subsystems narrowly-defined to ensure the best performance.

A distributed gaming engine decouples the subsystems and would require generic calls to the various subsystems that produce the end result. For example, instead of  calling a graphics renderer directly, the game's orchestration server requests the asset (e.g. an object in the gaming environment) and the result is returned by the cloud using whichever available resources provide the best result.

This approach is still at the academic stage and looks to be a while off.

Games

In my previous post, we discussed some of the pitfalls of offloading processing to the cloud. Twitch games such as multiplayer FPSs are some of the hardest to get right due to the impact of latency and input lag on player competitiveness.

Crazily enough, Improbable decided to build a battle royale — Scavengers. Currently in Steam Early Access, reviews haven't been stellar so far with many reviewers complaining of poor quality netcode (typically associated with some form of lag) and, perhaps more importantly, lacking a fun gameplay loop.

Scavengers is primarily PvE with some PvP elements, presumably to make the most of SpatialOS's cloud AI and has been in development since 2018. The difficulty with this approach is that better AI tends to worsen the player experience, as the performance of the AI needs to be imperfect to prevent the feeling of being cheated by perfect aim and evasion.

Improbable were probably forced into creating Scavengers after few developers wanted to use their tech and are trying to find their Unreal Tournament as a showcase. Building both at the same time is a Herculean task, especially in such a competitive market for battle royale games.

Microsoft have taken an interesting approach with Microsoft Flight Simulator, where the rendering of highly complex environments is done in the cloud. This approach allows the game to process 2.5 petabytes of data and also have live weather — something that would be impossible on a consumer device.

This is a smart choice. The relative velocity of the aircraft relative to the skybox is low, meaning that the player doesn't need millisecond updates of the environment to have a good experience. The game has also been designed that if the player doesn't have sufficient bandwidth, then the game can still be played with lower-res environments and without live weather and events.

For now, the Microsoft Flight Simulator approach appears to be where the cloud will be best utilised until we see further improvements in the tech infrastructure to orchestrate truly distributed computing.

Once these hurdles are overcome, we have the chance to unlock a step change in immersive gaming experiences.