In this article, you can expect to learn more about developing WebGL games using Unity, what challenges our team had and how we tackled them in order to release a new version of Top Eleven to the Web.
Waiting for Technology to Mature
In January of 2015, we released a new version of Top Eleven developed in Unity. The big update brought a new and modern UI and a greatly improved user experience on mobile. Unfortunately, it was released on our mobile platforms (Android and iOS) only. This meant that users who play in browsers with the old Flash version were omitted.
At that time, the whole Web technical ecosystem was in a period of transition with Flash slowly becoming obsolete and browsers (more specifically Chrome) no longer supporting NPAPI plugins, which included the Unity Web Player. The Web has started turning towards HTML5 and underlying technologies. At that time, Unity already had an option to export the application for WebGL, but the feature was still not officially supported by them and not quite ready for releasing larger-scale projects. Nevertheless, we spent a few days trying it out, but there were too many blockers and risks, so we decided to postpone the effort and leave the Flash version on the Web.
During the following months, the Top Eleven development team focused mostly on new gameplay features, but we also kept our eyes on technology improvements. HTML5 and WebGL were becoming more and more supported by Web browsers. Unity had also continued developing and improving their pipeline for building WebGL apps.
So in late 2015, it started to look like technology was coming to its feet, creating some hope that an application the size of Top Eleven could be published. Since the Top Elevendevelopment team was at the time working hard on creating the new Training feature, a few enthusiasts and Unity know-hows from a couple of different Nordeus teams got together and started digging. So what have we learned along the way?
What is WebGL and Where Does Unity Fit In?
Web browser developers have been working hard on implementing new technologies that support running interactive 2D and 3D content, such as using GPU power to render content directly into the web page canvas. WebGL is the name of the JavaScript API that makes all this possible. So with Unity and its WebGL build target, making games for web sounds like a piece of cake. Well, not completely.
Unity applications are written in C# and we need the JavaScript code to leverage the browser’s JavaScript API, right? So to run Unity applications in browsers, Unity created the whole build pipeline.
In the process of building a Unity project for WebGL, the C# code is first compiled into IL (Intermediate Language), which is then compiled to C++ by the IL2CPP compiler. Then, the C++ code is compiled into JavaScript by Emscripten. At the end of the build process, you get one text file with your distributable JavaScript code of your whole game and everything it uses — Mono runtime, Unity engine and third-party plugins. For release builds, this JavaScript code has been minified and the file is packed in gzip. Besides the code, all embedded game resources are stored in another file.
So, when the user opens a web page, all those files are downloaded and the code is parsed by the browser. Code parsing causes the first (and probably the largest) performance hit, which in the case of Top Eleven lasts for at least a few seconds (depending on the user’s hardware). This prevents any other web page code from being executed and generating a big memory spike in some browsers. If the browser “survives” that, your game will at last be shown.
Making the Build Work
After learning the basics of this technology, it took us a couple of weeks to make the Top Eleven WebGL build work by fixing and patching things in one way or another (mostly another). The first big challenge was that TCP sockets and threading are not available for WebGL and these were the two things our communication stack was built on. So we had to create support for the WebSocket protocol on both the front-end and the back-end. On top of this, all communication — including services used by Top Eleven — had to be over HTTPS, since Facebook uses HTTP Secure.
On the client side, we used the Simple Web Sockets for Unity WebGL library to implement a brand new comm stack. On the server side, we implemented a proxy server using the Netty library that communicates with WebGL clients using the WebSocket protocol and just passes messages to our game servers using TCP sockets. In that way, we didn’t change our server infrastructure much, nor changed game server functionalities. We just added one layer between clients and servers. As a result, our mobile clients could keep communicating with game servers by TCP sockets.
But in order to create a proof of concept that would show us Top Eleven could be published on WebGL, we had to experiment and build the project numerous times (each build lasts for 15-20 minutes). Fortunately, we already had our Nordeus automated Unity build server, which we adjusted to handle WebGL builds so that we were now able to continue working in the Unity Editor while long-lasting builds were in progress.
Soon, we had the proof of concept which showed that Top Eleven could be published on WebGL. So the devs formed the new Nordeus Labs team and their first project officially kicked off!
Improving the Performance
We focused our efforts mostly on improving the performance of the game, or more precisely, the game initialization. Around that time, the 5.3 version of Unity was released, in which WebGL was officially supported as a build target. The new version of Unity, as well as the latest Firefox and Chrome versions, have brought a lot of performance improvements during both app initialization and runtime, which showed us that things are still getting better for browser games. There were 3 main areas that we tackled.
Heap Size
A WebGL application needs heap memory to work with, which is actually a continuous block of RAM that an application requests from the browser during the initialization process and it cannot be enlarged afterwards. So we just had to determine the size of the block. If we aimed too high, we risked that the browser wouldn’t give us that amount of memory (especially if the user were to have many tabs open in the browser). If we aimed too low, we risked the chance of using up all that memory during runtime. In both cases, if an error were to occur, the user wouldn’t be able to play the game any more and would have to reload the page or even the whole browser.
So we started with 256MB of heap but soon realized that there were some use-cases when we needed more — the app was crashing because of a lack of memory. We then increased it to 384MB and it turned out to be the sweet spot that worked. At some point in production, we experimented with 256MB, but the number of memory allocation errors during application initialization hadn’t decreased, so we turned it back to 384MB.
Application Size
Initially, there were too many resources embedded in the application, making the built data file initially around 70MB. That was unacceptable, since the game would need more than 9 minutes to be downloaded via a 1Mbps low-end Internet speed; users should be able to play the game much sooner than that! Also, our tests showed that web browsers cache files smaller than 30MB, which is a limit we had to go below.
Our initial estimation was that we could decrease the data file down to 15-20MB by extracting assets to Unity asset bundles. That wasn’t a trivial job because we didn’t have mechanisms that allows the user to play the game while certain types of resources are being downloaded (such as sounds, some 3D models for Ground screen, fonts). Beside resources, we made our serialization feature work with WebGL, which gave us both a runtime performance boost (mostly in decrease of scene loading times) and a smaller build size. At the end, we ended up with data file below 9MB.
Size of the Code
Size of the code does affect the total size of the game, but since the packed size of the JavaScript file was around 7-8MB, that wasn’t the biggest issue. The biggest issue was the memory spike during the parsing of the code while the app initializes. The size of the spike was proportional to the size of the code itself. And the bigger the spike is, the bigger the risk of running out of memory and the more time it takes until the code parsing is finished.
We did a few things to address this issue. Firstly, we used a custom C# attribute stripper (forked from Unity3D.UselessAttributeStripper), which is called alongside Unity’s one during the build process. The stripper removes C# attributes from the code that are not used in runtime at all, such as System.Obsolete, System.Diagnostics.*, and others.
Secondly, with the help of Unity developers, we used Unity’s new build report tool to check which exact classes/methods entered the build, as well as their sizes. It turned out that the Unity Physix module was entering the build and we found other solutions for the few places where we actually used its functionalities.
So with these and some other smaller improvements, we reduced our unpacked JavaScript code size from 48MB to 42MB, which decreased the memory spike by about 15%.
NRE Problem
A problem that we ran into at one point concerned NREs (Null Reference Exceptions); it looked quite serious. Since our code in C# was being converted into C++ by the IL2CPP compiler and C++ doesn’t have built-in NREs, without an additional code being generated during the build process the WebGL build couldn’t detect them.
Unity has an “Exception Support” setting for the build and if “Full” is selected, then all null checks and the throwing of NREs are generated in the C++ code. This makes exceptions work like in the original C# code. The other option is the “Explicit” exception support, which doesn’t generate the above mentioned code. In this mode, only exceptions thrown with the “throw” keyword in the original C# code are replicated and the NRE behaviour is unpredictable. In practice we found that this often meant that when a NRE happens, the whole game stops working. So “Full” would be an easy choice, if that wasn’t for the fact that besides the exception support it also adds full support for replicating C# stack traces in C++ — a feature that almost doubles the code size and reduces performance by an order of magnitude!
Needles to say, we needed a setting in between, so we started hacking Unity to try to get it. Shortly after, it turned out to be a matter of passing different flags to the IL2CPP compiler and voilà — we had it setup to output the NRE exception code and omit the slow stack traces code. The code size increased by 2MB, which was acceptable bearing in mind the risks we would otherwise have to deal with.
Other WebGL Specific Tweaks
There were numerous other tweaks in the code that had to be done in order to make Top Eleven WebGL completely functional and browser-friendly, such as improving PlayerPrefs logic, handling some third-party libraries used for our mobile apps, handling in-app help, etc. But one of the biggest technical challenges we faced was handling the fonts for WebGL.
We are using dynamic fonts in our GUI and since the access to system fonts is not possible from browsers in JavaScript, we had the potential problem of showing user-generated content (such as club and player names) to every user. We refactored our label and font system to support more fonts, so depending on its text, a label can in runtime determine which font it should use to render its characters. That way, players that are using around 30 different languages in Top Eleven have an unspoiled experience.
Preparation for the Release
Even before releasing the WebGL application, we started gathering data from our current Flash game about how many users had WebGL supported in their browsers. There were a few reasons for not supporting WebGL content. Some browsers don’t support WebGL at all and the ones that do support it only starting from some specific versions. Also, the users’ computers need to have GPUs and the drivers that are compatible with WebGL, which is the case with all newer GPUs but there are problems on some older user computers.
Our analytics showed that we had around 20% of users that could not play the new WebGL version of the game because of one of these reasons. With that in mind, we prepared a strategy that would allow them to play the old Flash version when we were to release the new one. Also, to be able to monitor what’s happening in the production, we implemented tracking of exceptions and assertion failures, as well as issues users might experience during the application loading and initialization.
Release
We had a few issues on the day we released WebGL, but they were minor bugs and tweaks that we fixed immediately (fortunately the Web is a platform that we can do that without our users even noticing it).
For the first few weeks after the initial release, the biggest issues were that we didn’t find a good way to decipher the minified stack traces we got in our exception reports. Fortunately, we found the solution later by passing the –emit-symbol-map parameter to the Emscripten compiler.
Also, for a while we were keeping a close look at data from production, both on exception reports and at the ratio of users playing on WebGL vs. Flash. We noticed that there were more users playing on Flash than we had expected. After a more detailed research, we concluded that this was mostly because:
- there are users who are not ready to make a change for the better, or
- the technical requirements are causing problems to users with older computers that we cannot control.
At the end, the team was really satisfied that after only a couple of months of development, we managed to deploy Top Eleven on a completely new platform, and do so with a technology that not many games had used until that point.
Unite Europe 2016 Talk
Tomislav Rakic and I shared our experience on this topic at Unite Europe 2016 about this topic, so if you are interested I would recommend that you watch the video below.