Reflections

In the last part, we talked about SSR reflections – reflections that work exclusively with the information that is currently on the screen. It is for this reason that such reflections “disappear” if the camera looks down at a strong angle – there is not enough information on the screen to build a full-fledged reflection.

But they are not the only ones used in the game. For example, when we approach a metal post, we want it to reflect what is around us – even what is beyond our visibility (behind our back, for example). To do this, the game must put the camera in place of this column, and then make a “capture” of the image of everything that is around and subsequently it is this “captured” image that is used to draw the reflection. A kind of reflection map.

In order for the game to know where to create such reflection maps, special markers are placed in the editor, called Sphere Reflection Capture and Box Reflection Capture. In most cases, for example, spheres are used, and boxes are used in rooms.

This is what these spheres look like in our MTA client

Their main purpose in these places is reflections in the glass of the shopping center.

In the glass, you can see the reflection of houses that are out of sight. They lookd odd because the reflection quality settings are low and blur is also turned off.

And this is an example with columns. You can see red shades in them – this is the reflection of anearby shopping center.

Apart from the fact that these reflections allow you to show what is off the screen, they are also less demanding on computing power (especially static reflections).

The difficulty that we encountered in their implementation is that such reflections for optimization are built either very rarely, or even only once. But then they do not take into account the sky (time of day, color of the sky, etc.). I had to think carefully to combine all this together and at the same time not lose performance.

Splines

After we decided to use Unreal Engine as the map editor, we also thought about adding to the MTA some of the features that are widely used in UE and make life much easier for modelers / level designers.

One such feature is splines. After all, it is very convenient when you can put several points and between them, for example, a fence, rails, a road or a hanging wire will be automatically built.

This feature is actively used, for example, for roads and wires (screenshots from later stages of development):

Clouds

For weak computers, we just use a high-quality skybox (similar to what we did for RPBOX), but for owners of powerful hardware, we decided to make real dynamic volumetric clouds.

To do this, we considered various options, including ready-made solutions, like trueSky, but in the end we decided to make our own implementation, close in image quality and capabilities to Unreal Engine clouds.

This job took almost a month.

The tall window is a list of cloud settings. All these parameters are controlled by the weather system, which we will discuss in one of the following articles.

And these are the clouds with the correct tonemapping and post-processing:

Like it? We are pleased with the result.

Not without ‘interesting’ bugs

Combined with atmospheric dispersion, which we talked about in the second part of the diaries, this is the beauty:

Optimization

Optimization covers a huge number of aspects. But now we will talk about optimization, which is called Occlusion culling.

This is a technology that allows you not to draw on the screen those objects that are hidden by other objects.

GTA SA also uses this technology, but it is implemented there at a rather primitive level, which is why it cannot be used for too many objects, otherwise it can cause a noticeable performance drop. Not to mention the fact that there all the overlapping planes must be placed manually.

Since our models are much more “heavy” and complex, the use of such optimization becomes very critical. Therefore, we made our version of optimization, where the size of visible objects is automatically calculated, and if they occupy more than a certain area on the screen, then occlusion culling is turned on for them and all objects that are behind them stop being drawn.

A little demonstration:

Advanced implementation of Occlusion Culling.

Here, the red boxes show the boundaries of the objects that were excluded from being drawn on the screen by the occlusion culling algorithm.

That’s all for today, see you in the next part!

Reflections and PBR

From time to time we read messages like “there are no reflections in MTA / GTA SA” under our screenshots. Yes, there are no such reflections as we have (although in GTA SA there are, for example, mirrors and mirrored floors). But we have everything quite seriously redesigned, and we made Screen Space Reflections (by the way, on the first version of the render, such reflections would be very, very problematic, if at all possible).

In these early screenshots, the scene is unconfigured (there are no light sources, the environment is not taken into account), and small artifacts (small squares) are visible on the surfaces – that is because the normal textures are compressed, and support for compression in the render at that time was not yet added. But in general, the reflections are quite visible.

A few technical details. Before that, the normals were compressed by using the DXT5 algorithm, just like regular textures. But this kind of compression does not have enough precision to store the normal. Therefore, we added support for BC5 (in fact, these are two combined DXT3s), which greatly increases the accuracy and the normals look correct.

Here is a later version – added compression support, as a result of which artifacts on surfaces disappeared, and took into account the environment in reflections, thanks to which the scene began to look noticeably better.

If you take a closer look at the second screenshot, you will see graphic artifacts in the form of small light horizontal stripes on the towels:

It’s funny, but these are not graphics engine bugs, as one might think. These are jellyfish, which you probably saw more than once underwater in GTA SA

And this is a test scene where the PBR result is clearly visible. All these abrasions on the metal in the speaker look even cooler!

Remember what this scene looked like in the previous article? Well, we added reflections, placed several light sources, and the scene changed noticeably, didn’t it? And it is worth noting that this is far from the best result that can be achieved.

And this is more or less the final version of the reflections. The scene is not tuned and the reflections are specially twisted “to the maximum” so that they can be better seen.

The graininess of the reflections is a characteristic feature of SSR. By the way, this graininess can be seen in many of our screenshots; it is enough to enlarge the image on some reflection.

To completely get rid of this graininess, you need to use ray tracing. We do not have it yet, but it is proposed for the distant future (alas, the prices for RTX video cards somehow do not contribute to active development in this direction).

Landscape

The usual method that is used to create landscapes for GTA SA is modelling in some kind of editor (3ds Max, Blender, etc.), then slicing it into pieces no larger than 256 × 256 meters and then transferring these meshes to the game. And if you need to make any changes, this entire process must be repeated over and over again. In general, it is difficult, long, terribly inconvenient and can easily lead to all sorts of problems, like gaps between pieces of the landscape, sinkholes, invisible walls, and other problems that can cause a lot of headaches.

We decided to go the other way and create our landscape system, which will not be limited in size (yes, you can do even a thousand by a thousand kilometres), is easy to change, does not load the system, and is convenient for modellers. No sooner said than done!

As a basis for the landscape system, we took the way the landscape is implemented in the Unreal Engine. Our system completely reproduces the result created by the UE. However, this applies not only to the landscape; we have already mentioned that we actively use UE to develop the project (in particular, we set up all materials, lighting, reflections in it, compose the map, etc., but we will talk about this somehow another time). In our project, we are completely trying to reproduce the picture that the Unreal Engine produces, and, as you can see from the screenshots in the group, we do an excellent job with this task.

An example in which a landscape created in UE is reproduced one-to-one in MTA (slight colour difference due to lighting). And our system supports almost every feature that the UE landscape has.

The standard GTA SA system for physics uses collisions stored in col files. It is also used to set various properties of surfaces: appearance (where the grass grows, for example), sounds, shadows, and determines whether an object is visible on the screen.

In order to dynamically create the landscape, we had to modify this system so that it was possible to create collisions “on the fly”, and of arbitrary size, not limited to the maximum for GTA SA 256 × 256 meters (this is not entirely true, as in the standard GTA SA can use much larger collisions, but it can hardly be recommended to do this because a significant part of the optimization is done precisely due to this “slicing”).

Moreover, we attached a modern physics engine to the landscape, which gave us multithreading in handling collisions with the landscape, the ability to dynamically change the landscape (including digging holes, for example) and some other things that we will talk about in the future.

The landscape itself is drawn in small parts with different qualities – those that are closer are drawn with maximum detail, and the farther from the camera, the lower the quality. This technology is called Continuous Geo-MipMap LOD.

In a special debug mode, the polygonal mesh of the landscape is clearly visible. It is best to watch videos in 4K quality, even if you have a much smaller screen – this will force YouTube to play videos with a higher bitrate, otherwise the picture is very blurry and all these thin lines turn into some kind of spots.

It has already added smooth blending of different levels of LODs, which gives an excellent result.

GUI

Due to the method that we used to transfer the game to DirectX11 / 12, we could not use the usual GUI systems for the MTA, as this greatly reduced the performance in our case. We needed a solution integrated directly into the game client, directly working with DirectX11 / 12. We checked many available options and settled on a completely atypical solution for GTA SA – NoesisGUI.

It is a modern and advanced XAML-based GUI that supports vectors, complex animations, themes, and more. And the most interesting thing is that interfaces are not built directly into the game code but in special config files. This means that changing and creating new interfaces is much easier. And you can check their performance even in a browser without starting the game.

It took NoesisGUI to connect to MTA only a few days, while Lua integration took significantly longer. In total, it took about a month.

Just don’t ask us to show our interfaces now We will show you when we are ready

For today, perhaps everything! See you in the next part of our blog!

In order to avoid potential problems with the former employer, we decided to do ReflectRP completely from scratch, without using anything that we did for RPBOX. The same goes for the rendering system. We planned to use the system with a completely different approach, and at the same time get an incomparably better result.

As you know, GTA SA runs on DirectX 9, which was released back in 2002. Naturally, over almost 19 years that have passed since that moment, many technologies – both software and hardware – have emerged that provide many benefits to both developers and ordinary users.

Therefore, the first thing we did was transfer GTA SA to DirectX 11/12. Although it sounds so simple, the work was enormous (which took somewhere in the region of six months of work), since the entire MTA is “sharpened” for DX9. That is, if you switch the game to DX11 / 12, then almost all the MTA functionality related to graphics will stop working and there is no simple effective way to get around this problem. However, we successfully coped with this task, which allowed us to make a bunch of optimizations, use modern development tools and get a high-quality result. And some things without this would be impossible to do at all.

For example, we have a very low memory consumption of the game, because now all the possibilities of video memory are fully used – there is no need to duplicate models and textures in the main memory.

Shadows

After switching to DX11 / 12, we started working on the shadows. At first it’s just black spots

to which transparency is later added

these are the earliest screenshots, but even here you can see that the shadows are of higher quality than in the first version of the render.

If someone is interested in technical details, the shadows already supported up to eight cascades (in the previous version there were a maximum of four), smoothing seams, smoothing PCF with different grain settings, algorithms for adjusting and optimizing cascades for the best ratio of pixels of the shadow map to the rasterized texel, as well as the ability to choose different types of shadow cards, depending on the capabilities of the hardware.

Sky

The next step was to work on the sky.

So far, basic – no clouds, sun, moon and stars. It would seem, what in that case is still there? It’s very simple – the atmosphere remains. Depending on the time of day, the sky color and atmospheric light scattering are correctly adjusted.

Some of you have probably seen a similar sky in Unreal Engine or Unity, or in games made on these engines (there are even those who thought that screenshots from one of these engines, seeing such a sky). And someone will say: “just some kind of fog, what is it?”… But no, this is not a simple fog. Here are a couple of pictures from the internet showing what atmospheric dispersion does:

Beauty, isn’t it? So we will have it when we complete the work *.

* Although we try to talk about the work done in chronological order, at the time of publication of this article, atmospheric scattering is still not completely finished – some improvements and adjustments are required, which we will be doing closer to the opening of the project, and therefore such screenshots of our game map are not to be posted anytime soon.

PBR

The next step is to add PBR, a physically based rendering. This is a modern standard that allows you to get a photorealistic image: metal will look like metal, glass – glass, leather – leather, wood – wood, etc. Thus, this standard allows you to reliably reproduce the optical properties of surfaces.

Street scenes look pretty good even without post-processing, don’t they? If there were self-shadowing (Ambient Occlusion) and anti-aliasing, it would be almost the same as in the screenshots that you saw in the group. But the interiors at this stage look much worse due to the lack of anti-aliasing, tuned light sources and reflections.

Performance

Taking into account the fact that the models on our map are worked out in much more detail than in other projects based on GTA SA, the volume of resources goes to a completely different level. And, of course, the game simply cannot handle such volumes of data.

To make it work, we also had to significantly rework a lot of things in GTA SA and MTA. For example, we use our own model and texture formats.

The existing toolkit for GTA SA / MTA is also unsuitable for working on a project of our level – for this we had to adapt modern tools and develop our own. And this is also a huge amount of work. It is possible that we will also tell you about this someday.

Almost three million triangles at an average of 100 FPS. As already mentioned, at this stage of development, scenes in interiors look inconspicuous due to the lack of anti-aliasing, adjusted light sources and reflections (well, we decided not to waste time arranging and adjusting light sources in test scenes – we did not even think that someone besides us will see these screenshots).

Ambient occlusion and anti-aliasing

The next step was work on self-shadowing (Ambient Occlusion) and anti-aliasing. For computers of different power, several of different algorithms are made – fast, but with a lower quality result; and slow, but with excellent quality.

If you are interested in technical details, at the time of this writing, SMAA 1x, SMAA 1x + TSSAA and SMAA 4x are supported for anti-aliasing, and SSAO, HBAO and Alchemy AO for Ambient Occlusion.

But right at this stage of the development of our rendering system, only the most basic algorithms were implemented.

Simple anti-aliasing result:

And this is the result of the basic self-shadowing algorithm:

Then the resulting image is used when drawing the final image: the white color remains unchanged, and the dark one just adds shading.

We combine self-shadowing and anti-aliasing, which significantly improves the result:

What happened next?

Later, we added better self-shadowing and anti-aliasing algorithms, we did serious work on dynamic lighting, transparent objects, reflections, tessellation and much more, but we will talk about this in the next parts.

The development of our own render began at the end of 2019, when our team was working for RPBOX. The project management did not want to waste time on something that did not bring money, so we then started developing the render on our own initiative, in our spare time – in the evenings and on weekends. And only after seeing the first results, the management approved the full-time work in this direction.

To avoid confusion, this render is not ENB. This is a much more complex. In short, we completely disabled the part of GTA SA that is responsible for displaying the game and instead made our own system.

At first we just wanted to add shadows and qualitative reflections.

Qualitative reflections mean that they reflect exactly what is currently around and even what is outside the screen.

Then we added Ambient Occlusion (self-shadowing of closely spaced surfaces):

After that, we added the ability to adjust the gamma and palette in order to make a different picture depending on the time of day.

This was followed by multiple processes of customizing the visual part.

In 2020, we added all this to the RPBOX, and also began to tune the vehicles for it

and gradually came to something like this:

Later we implemented anti-aliasing, gorgeous sky, PBR, dynamic shadows, planar reflections and some other features.

Unfortunatly, we did not have enough time to release the glass, since RPBOX’s founder (who tried to rip us off for money some times) returned to the project, our whole team decided to leave the project. Just look at this beauty:

The problem of all transparent materials was also solved there: there were no more problems with edges around tree crowns, transparency of glass in vehicles, etc.

At that moment, all the main parts were completed, apart from the water.

This completes the story of the first version of the render history, since we did not take any of our developments for ReflectRP and started working completely from scratch, which we will talk about in the next part.

In conclusion, a couple examples of funny bugs that came out in the process of working on the first version of the render.