Technical question : mm_pool_size internals and memory ?

Post Reply
User avatar
michbret
Posts: 200
Joined: 27 Sep 2013 13:06
Location: France

Technical question : mm_pool_size internals and memory ?

#1 Post by michbret » 06 Jan 2014 19:55

Hello again,

For those who are not aware, mm_pool_size is an option that can be added when launching the game executable by using a shortcut or the steam launch option.
The format is -mm_pool_size=xxx where xxx is a number like 600, 1200, ...

When I started to play this game, one year ago (when it was published on Steam), using this option was almost mandatory if you wanted to play on some mod maps.
At this time, the game was in version 1.2.5.1, I think. After some research, I discovered that the game was using a default value off 200

The effect of this option (or the default value) is shown in the beginning of game.log (game.log.txt) now with the following line (below with -mm_pool_size=1200) :
00:00:00.000 : [mem] Trying to allocate memory pool (1228800K)

Ok, with this big value, the game allocate 1.2 GB which is quite big since the game (been 32 bit) can only use 2.0 GB on a Windows 32 bit system or 4.0 GB on a Window 64 bit system (or Linux, I presume). That also explained that using too big values of mm_pool_size can have the adverse effect of crashing quickly the game.

So be warned of this since many things you can read on the web are erroneous or misleading. Do not use too big values. Another stupid thing that you can find of the web is the reference to the amount of physical memory available. Your paging system (page file, swap file) and working set estimation are taking care of this.


However, the game quickly use more memory than this initial allocation. This is easily verified with some process monitor.
So how the fact of using a bigger value for mm_pool_size (eg 800) could avoid the game crash on some map ???

It stayed mysterious ... except if this is the size of the dynamically allocated memory when the game needed more memory.

Let take an example : you are driving and enter a town where the map designed has put, let's say, 100 of buildings that are totally new (not the prefabs and classical building you find everywhere). To load and draw all these new building, the game need fresh memory so it allocate mm_pool_size of memory (call malloc(mm_pool_size) ).

Three things can happen now:
1) There is enough virtual memory available (the 2 GB or 4GB limit is not reached) and this memory is sufficiently large to fit all that has to be loaded and drawn: Everything is OK. The game goes on.
2) There is enough virtual memory available (the 2 GB or 4GB limit is not reached) and this memory is not large enough to fit all that has to be loaded and drawn : The game crash
3) There is not enough virtual memory available (the 2 GB or 4GB limit is reached) : The game crash


All of this is pure speculation, let's say just a guess that would explain the behaviour that was observed on these former version of the game.

So, my first question is : Is this guess basically correct or should I stop smoking illegal substance quickly :mrgreen:


Let's go on. From version 1.2.5.1, the game has received many updates and has been seriously improved. That's the opportunity to thanks all the SCS developer team for such involvement and commitment. That's properly amazing :o .
So again a big thanks to SCS

I noticed several improvements after version 1.4.8, especially in map and memory management. So let's talk about the current version (1.8.2.5)
The mm_pool_size option is still effective but the default value has changed : It is now 400 instead of 200. This solve most incompatibilities with mod maps.

After some research I also found that Flemming V was thinking that the mm_pool_size was not useful any more


So, my second question is : Does the memory management change a lot (so the mm_pool_size is not useful any more) or is it only the change in the default value that solve the problem for let's say 90% of the mods.



Why am I asking such details.

The memory can quickly become a serious problem since, in the 32 bit world, it is limited to 2 GB or 4 GB. And, if you put all the graphics settings to high,, the memory demand is even higher.

This game has a strong community of enthusiast, creative and talented people that create nice mods. If they are too enthusiastic in their creations, this can go into a dead end. I have the feeling that the new Promods map (v1.52) is dangerously flirting with such limits. If I play some hours and visit several "heavily" defined town, I quickly reach to 3.6 GB of VM usage (monitored with the minicon + gps). This usually finish with a crash (and may be some blue screen - I am currently checking 2 stop 0x19 subcode 0x22).

And to finish, if the memory management is done by chunk of mm_pool_size, this should be carefully used by people and the underlying behaviour should be understood. If not, many false problem may arise, the root of such problem been often found in some misleading web advices.


Thanks in advance and to SCS team, thanks and go on with the nice work.

Best regards and happy new year
User avatar
Max
SCS Software
Posts: 6481
Joined: 26 Nov 2012 10:00
Location: SCS, Prague

Re: Technical question : mm_pool_size internals and memory ?

#2 Post by Max » 06 Jan 2014 20:52

mm_pool is used for some type of data not all allocated memory the game uses. thus 200M was enough for most games and only some mods needed more. the game still used 2G (or 4G) available. therefore is clear why the game crashed if you set too big mm_pool - there was not enough memory for other things.

yes, there was change in memory usage so the mm_pool parameter should not be needed anymore. only if you still have memory problems it may sometimes help (depends of amount and type of additional data in your mods). but the problems are caused mainly by memory fragmentation so its case by case.

of course, as map is ever growing and textures are more detailed we are still working on improvements.
Nobody can tell you anything about it. Those who know about it aren't allowed to talk. And those who talk know nothing.
User avatar
michbret
Posts: 200
Joined: 27 Sep 2013 13:06
Location: France

Re: Technical question : mm_pool_size internals and memory ?

#3 Post by michbret » 06 Jan 2014 22:26

Hello Max,

Thanks for the answer.
Max wrote:mm_pool is used for some type of data not all allocated memory the game uses. thus 200M was enough for most games and only some mods needed more. the game still used 2G (or 4G) available. therefore is clear why the game crashed if you set too big mm_pool - there was not enough memory for other things.
Yeah. I was quite sure that it was more complex that my simplistic approach. I know that many items are statically allocated, thus in a limited number (like drivers, trucks, AIs car, ...). Was it a mistake to suppose that many map items (buildings, ...) fall in the mm_pool_size buffer ?

Max wrote:yes, there was change in memory usage so the mm_pool parameter should not be needed anymore. only if you still have memory problems it may sometimes help (depends of amount and type of additional data in your mods). but the problems are caused mainly by memory fragmentation so its case by case.
Nice information. I'll discard my mm_pool_size option and will provide feedback if needed. I also have seen the changes and they are improving the game/engine. Memory fragmentation is a real pain in the ass for developers and garbage collectors are real-time killers
Max wrote:of course, as map is ever growing and textures are more detailed we are still working on improvements.
I'm confident on this but the 32 bit wall may become a problem. A 64 bit version would give much more margin but that's not an easy move.

Thanks again.

Best regards
User avatar
Cadde
Posts: 6806
Joined: 24 Apr 2013 18:08
Location: Have no fear, i am from the internets!

Re: Technical question : mm_pool_size internals and memory ?

#4 Post by Cadde » 07 Jan 2014 02:16

Just wondering, is there any reason as to why you are running a 32 bit OS? Wouldn't it help to change it over to a 64 bit to alleviate the issue?

I am not sure how much this applies to ETS 2 but some graphics cards with limited amounts of memory can also run out of memory when you have too many textures needing to be loaded into graphics memory.

Another option would be to reduce the size of the largest textures, I.E not running 4k resolution textures on your truck skin etc.
Kerbal Space Program suffers from this, to a greater extent i might add. Their solution is to reduce the size of the textures. (making the game look blurry and not so nice, but at least not crashing)
On extended hiatus.
User avatar
michbret
Posts: 200
Joined: 27 Sep 2013 13:06
Location: France

Re: Technical question : mm_pool_size internals and memory ?

#5 Post by michbret » 07 Jan 2014 06:27

Hello Caddle,
Cadde wrote:Just wondering, is there any reason as to why you are running a 32 bit OS? Wouldn't it help to change it over to a 64 bit to alleviate the issue?
In my personal case, I am using windows 64 bit on all my computers. I have even upgraded my laptop from Win seven 32 bit to Win seven 64 bit but that was not easy since most laptop have proprietary drivers for which you must find a 64 bit version. I talked of windows 32 bit since many users are still using such systems (look at game.log.txt).
Cadde wrote:I am not sure how much this applies to ETS 2 but some graphics cards with limited amounts of memory can also run out of memory when you have too many textures needing to be loaded into graphics memory.
You are right but that's not my case (I am also monitoring graphic card memory usage).
Cadde wrote:Another option would be to reduce the size of the largest textures, I.E not running 4k resolution textures on your truck skin etc.
Kerbal Space Program suffers from this, to a greater extent i might add. Their solution is to reduce the size of the textures. (making the game look blurry and not so nice, but at least not crashing)
Yes but I like graphical quality. The true solution would be to develop a 64 bit game engine! But usually, companies like SCS want to target both end of market which would imply to have 2 versions of the game engine : a 32 bit for the low end which mainly uses Windows 32 bit and a 64 bit version for people lucky enough to have a 64 bit system.

Best regards
User avatar
Max
SCS Software
Posts: 6481
Joined: 26 Nov 2012 10:00
Location: SCS, Prague

Re: Technical question : mm_pool_size internals and memory ?

#6 Post by Max » 07 Jan 2014 08:22

we can make 64bit build but it has its own issues. however, its possible that it will be released some day in future.
right now, for almost everything is good enough for player to have 64bit OS.
Nobody can tell you anything about it. Those who know about it aren't allowed to talk. And those who talk know nothing.
User avatar
michbret
Posts: 200
Joined: 27 Sep 2013 13:06
Location: France

Re: Technical question : mm_pool_size internals and memory ?

#7 Post by michbret » 07 Jan 2014 13:18

Hi Max,

I never doubted you could make a 64 bit version (some games are already doing this).

However, you are right. The current state running with a 64 bit OS is quite enough.
Post Reply

Return to “Help center - player to player”

Who is online

Users browsing this forum: No registered users and 4 guests