Very interesting topic.
midix wrote:The question then is - why does the caching utilize only about 25% of my CPU cores/threads? In comparison, when I run some test benchmark tools with full multicore support, I see all my CPU cores and threads working at 100%.
midix wrote:... So, clearly ETS2 caching is doing something not efficient enough. Maybe it spawns threads from time to time and there are no 4 threads running during all the caching process. Or maybe it suffers from the infamous "false sharing" issue and needs some paddings for its datastructures. I'm just guessing here, multithreading is tricky and sometimes it can go wrong.
Max wrote:... jut note that running four independent tasks with local data only is bit different than run four tasks, that share and access some common data ..
Exactly, how cpu is consumed by threads depends on what threads do and other factors (including OS resource management subsystems).
The fact that a given thread is not consuming cpu all the time doesn't mean that a problem is present in the code, unless one find clear information that proves the problem.
Please Max, if possible, what type of graphs and what variants of Dijkstra's algorithm are you using for trade routes calculations?Max wrote:... the 2 minute task you found is preparation of trade routes based on map topology.
AFAIK, early versions of Dijkstra's algorithm spend a lot of time sorting nodes. Time is exponential when lists are used, but tends to be more linear for other modified versions (for example, when binary heap or Fibonacci heap are used).
Regarding the Thorup's Linear Time Algorithm, do you think that some variant of it could be used as replacement for Dijkstra's-based algorithms in the future? (More info here, including interesting performance comparisons - slides 122 onward).
Thank you very much in advance.
Kind regards.