Heads Down with Kyle and Rob

Anthony “Minus” Guidi, Kyle Olsen, Robert Crane

On Pantheon’s path to alpha, two key milestones were the implementation of a new High Definition Render Pipeline and the new network stack. Both of these promise to take Pantheon to the next level. Today we talk to the two key pieces of these projects in Kyle Olsen and Rob Crane, to get an update on the progress and delve into why they are so impactful.

High Definition Render Pipeline

On the September Development stream, Chris Perkins stated that the terrain and asset streaming project was done. Can you clarify what this means? Furthermore, can you explain what other key pieces need to be checked off to call the HDRP project complete?

Rob: Code wise, the project has been converted and is running on the latest beta version of Unity with HDRP shaders and settings. But there are a lot of assets from the old project that still need to be converted to HDRP and copied over. We have scripts to help this process but it still needs an artist to verify that the assets look correct. On top of this, Odin, our main zone editing tool, needs to be converted to HDRP. All of these tasks are in process right now.

At one point, there was discussion about which of the two unity pipelines to implement, whether it was URP or HDRP. Why was the decision to push forward with HDRP made? What advantages does this bring with it?

Rob: When we started discussing the pros and cons for both, HDRP wasn’t as mature as it is now and we weren’t sure the game would run smoothly in HDRP. But we knew that if we could get HDRP working, it would yield far greater visuals. Also, we had just made the decision to use Gaia for the procedural baking of our zones and they were focusing on HDRP which gave us confidence to go with HDRP as well.

With HDRP implemented, even at its current state, how has this already affected overall development? How does it help you both in your roles?

Rob: We haven’t had a lot of issues so far. The HDRP conversion was a catalyst for quite a few changes on the way we develop our zones. We split the old project into three distinct sub projects- terrain, game art, and code, which allowed us to optimize development time by focusing on the needs of each department. This allowed for much faster compile times when developing new code.

Was there a specific moment in implementing this system that really made you smile, or you viewed it as a major breakthrough?

Rob: For sure. When I implemented the terrain generation tools to transform our old graybox scenes into nice looking mountains, and I saw the reaction from Joppa when he started using it, that was something I will never forget. I’ve been writing tools for games companies for 30 years now. I know how important it is to get the right tools in front of the team. I will always seek new ways to speed up the workflow for the artists and designers.

What can you tell us about Gaia which was discussed on this month’s live stream?

Rob: Procedural Worlds, the makers of Gaia, have been in the business of making terrain tools for many years and they have a wonderful product. Every time I talk to Adam (the owner of Procedural Worlds), I realise we are still only scratching the surface of what Gaia and GeNa can do for Pantheon. It has been a great collaboration so far. It will allow us to speed up terrain development on the project much faster than we thought possible before.

In your opinion, how does this pipeline raise the bar for the graphical standards of Pantheon, in regards to what we see in the genre currently?

Kyle: Honestly it’s not taking us into some cutting edge technology that isn’t seen elsewhere in the industry. Previously Pantheon was “stuck” in terms of graphic fidelity and asset densities we could achieve. This upgrade allows us to move those limitations back more.

Anything else you would like to add in regards to the HDRP project?

Rob: So far, no complaints. There are the occasional strange things that happen in Unity but you will get that using the very latest cutting edge beta versions of software. I think the results will speak for themselves once we have a fully fleshed out world ready to show.

Implementing the Network Stack

Moving on to the networking stack, Kyle you have been working on this eventual switch for some time. It started with cleaning up the code and implementing hooks that could be used in the future. Can you give us a brief overview of the origins of this project, and why this was something you focused on early in joining the Pantheon team?

Kyle: I am still somewhat new to the team when compared to the whole lifetime of the Pantheon project so I don’t think I’m the right person to ask for the origins. I can speak to the improvements and hooks you mention though. When I joined Visionary Realms “Project Faerthale” was in full swing. I was immediately tasked with implementing some of the Sleepless boss abilities while I learned the code base better. It was immediately obvious that there was a lot of technical debt built up over the years due to focusing heavily on delivering milestones instead of keeping a “clean” code base that was lean and easy to work with. There was also a lack of designer focused tools in some areas, or the tools we had simply didn’t function properly. I took it upon myself to deal with what I thought was the most obvious problem, and that was the current character controller we had. I gave a small demo one week of the updates and that got the ball rolling on our large “refactoring” process that we were somewhat public about. That refactor ended up being a pretty significant success and allowed us to explore climbing in a much more “real” way. From there we continued identifying the biggest areas of technical debt, or areas that designers had limited or no control over for refactoring.

It has been stated publicly that this networking stack being created has attracted quite a bit of interest from parties outside of Pantheon, in general terms, what is it about this project that is garnering this attention?

Kyle: There simply aren’t many options when it comes to networking in Unity. One of the main reasons our library is a bit different is the adherence to Unity’s Job System and Burst Compiler. Basically just performance and scale! 🙂

We have heard that the network stack implementation had to be completed before the Ranger and Summoner class could be worked on further, why is this so important for these classes?

Kyle: We have several NPC pathing and performance updates slated for post-networking upgrades. The summoner being a pet based class meant that it would be much easier to implement once we’ve done that on the new stack. The ranger has several things implemented for it already, but the complete reveal won’t happen until a bit later.

What impact will this new network stack have on concurrent player player count and NPC density, and overall stability of the game?

Kyle: By far our main resource drain on zone servers is sending (serializing) and receiving (deserializing) packets through our current network stack. This can fall apart when reaching low 10,000 packets/sec as it stands now. With our newer tech we’re able to process literally millions of packets in a single frame. What that ends up meaning for our NPC and player counts is unknown right now. We are still limited by things like bandwidth, but we are able to greatly reduce the CPU overhead of running a large scale network simulation inside a Unity server.

I’m no pro, but I believe one of the most challenging aspects of networking for an MMORPG is the speed at which commands can be sent and received from a client to a server. What strategies will Pantheon implement to make this feel seamless to players?

Kyle: Honestly this is a much larger problem in something like a FPS game. MMOS are not highly dependent on extremely low latency interaction with a server. We can do things like queue ability casts up invisibility when you are mashing on your keys to make it feel lower latency than it really is. Because we’re not doing something like a roll back simulation (CS:GO, or Valorant for example), it’s completely viable to play with 0ms-150ms latency without much of an impact on your experience. That all being said, our library is completely capable of “next-frame” responses. In local examples the RTT for a ping can be as low as 0ms-16ms (one “frame” of 60 fps gameplay is 16.6ms).

You’ve spoken in the past about creating a network structure that would ensure there wouldn’t be exploitation from the client side. Can you talk briefly on this topic and how Pantheon will combat exploitation in its network design?

Kyle: We currently force *all* actions to be performed server side only, with the exception of client movement. This means that the client can only ask the server for permission to do some action. Client movement is validated heuristically on the server side and accounts are flagged for possible speed/teleport hacks.

Without specifically naming any games, have you experienced anything in your previous MMORPG experiences that were a result of poor networking decisions? Have any of these things helped you in designing the network stack for Pantheon?

Kyle: The most extreme recent example of poor networking decisions actually comes from a first person combat game I played. The initial release of that game’s networking was implemented via Peer-to-Peer. Meaning that players were actually hosting servers on their own computers. This meant constant disconnects when people quit, poor connection quality and worst of all – cheating. Whenever a client gets control of a server, or the server isn’t properly validating actions, everything comes crashing down very quickly.

After so much time head down working on these important projects, finally seeing the light at the end of the tunnel, what would you say to the community about the health and future of Pantheon?

Rob: It’s been a very long road so far. Nobody can question that. Yes, we have made some mistakes along the way as all game companies do. But I believe we have turned a corner now. We are on the home stretch. You are going to see large jumps in progress as we build out the zones. Have faith in the process and we will deliver for you.

Producers’ Letter	Monthly Recap	Heads Down with Kyle and Rob	The Space Between the Lines