View Issue Details

IDProjectCategoryView StatusLast Update
0002871OpenFOAMFeaturepublic2018-03-13 08:27
Reportercraigwhite87Assigned Towill 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformGNU/LinuxOSUbuntuOS Version16.04
Product Versiondev 
Fixed in Versiondev 
Summary0002871: Follow up to 0002688: Barycentric Tracking Makes DSMC Simulations Significantly Slower
DescriptionFollowing on from this bug report: https://bugs.openfoam.org/view.php?id=2688#c8979

I now have some data on which functions have the most influence on simulation execution time. These are performed using OpenFOAM-dev pulled from the repository earlier this week.

OF-4.x: 772 sec -> reference
OF-5.x: 1205 sec -> +56.1%
OF-dev: 944 sec -> +22.3%

The lower execution time from OF-dev is from the patch Will introduced previously.

Functions that can be inlined in the particle class can be subdivided into different groups:
I] Patch interactions: e.g., hitPatch, ...
II] Tetrahedra functions: e.g., stationaryTetGeometry, ...
III] Transformations, topology and geometry changes: e.g., changeTet, changeFace, ...

I had a look at what was done in previous versions (OF-4.x). All of I] and a few functions in II] were inlined.

Inlining III] does not result in any noticeable speed-up.

Inlining I] (all functions in the group) results in a 5% speed-up, and inlining II] (all functions in the group) after that results in an additional 5% speed-up.

OF-dev original: 944 sec -> +22.3%
OF-dev I]: 913 sec -> +18.3%
OF-dev I] and II]: 875 sec -> +13.3%

In I] Patch interactions, the speed-up factor for a specific type patch interaction depends on whether it is implemented into the benchmark case and on what its extent is. For example, we don't have processor boundaries but it is reasonable to expect this to become significant when using many cores.

In II] Tetrahedra functions, after iterating, a single function is responsible for almost all of the speed-up observed: stationaryTetGeometry

OF-dev I]: 913 sec -> +18.3%
OF-dev I] and stationaryTetGeometry: 880 sec -> +14%
OF-dev I] and II]: 875 sec -> +13.3%
Steps To ReproduceSee this bug report: https://bugs.openfoam.org/view.php?id=2688#c8979
TagsNo tags attached.

Activities

will

2018-03-08 17:18

manager   ~0009396

Interesting, thank you for the data.

The effect of inlining stationaryTetGeometry is odd, as the function is private, so it should only be used within the particle.C compilation unit. I would have thought that would have given it a good chance of being inlined anyway. The same question arises with the patch interactions. They are templated, so they are available for inlining wherever they are invoked. The accepted wisdom on the internet seems to be that the "inline" keyword itself isn't particularly relevant when it comes to optimisation such as this, and that the compiler should just be trusted, but maybe that doesn't hold true in this case.

Anyway, it's good to know which methods are worth looking at. I will get the case running again and try inlining these key methods as you suggest.

Just to complete my information, what compiler and version are you using? Is it gcc 4.8?

craigwhite87

2018-03-09 13:13

reporter   ~0009397

The results were generated on a machine using gcc 5.4.0. The report should say Ubuntu 16.04, apologies.

will

2018-03-13 08:27

manager   ~0009418

I have inlined the *TetGeometry and *TetTransform methods (I didn't see the speedup without the latter), and the patchData method. Like you, I didn't see significant speedup with anything else. A couple of percent, maybe, but within measurement error. Pushed as commit 81947c80. My timings are as follows:

4.x 1157s (reference)
5.x 1833s +58.4%
dev-before 1424s +23.0%
dev-after 1245s +7.6%

Thank you for investigating. If you spot any other optimisations which don't require inlining too much library code, please let us know.

Issue History

Date Modified Username Field Change
2018-03-08 14:13 craigwhite87 New Issue
2018-03-08 17:18 will Note Added: 0009396
2018-03-09 13:13 craigwhite87 Note Added: 0009397
2018-03-09 20:44 wyldckat OS Version 14.04 => 16.04
2018-03-13 08:27 will Assigned To => will
2018-03-13 08:27 will Status new => resolved
2018-03-13 08:27 will Resolution open => fixed
2018-03-13 08:27 will Fixed in Version => dev
2018-03-13 08:27 will Note Added: 0009418