View Issue Details

IDProjectCategoryView StatusLast Update
0001944OpenFOAM[All Projects] Bugpublic2016-02-02 18:30
Reporteruser1281Assigned Tohenry 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionunable to reproduce 
PlatformGNU/LinuxOSUbuntuOS Version12.04
Product Version 
Fixed in Version 
Summary0001944: foamToEnsight induces core dumped error if case is in a very long directory
Description1) if the case is in a short directory, let's say /home/case, then foamToEnsight can make the data conversion.

1) if the case is in a very long directory, let's say /home/OpenFOAM/OpenFOAM-2.3.0/run/...very long directory.../case, then foamToEnsight always gives:


#0 Foam::error::printStack(Foam::Ostream&) in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#1 Foam::sigFpe::sigHandler(int) in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#2 in "/lib/x86_64-linux-gnu/libc.so.6"
#3
 in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/foamToEnsight"
#4
 at foamToEnsight.C:0
#5
 in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/foamToEnsight"
#6
 in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/foamToEnsight"
#7
 in "/home/yk/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/foamToEnsight"
#8 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
#9
Steps To Reproducei attached my simple case.

1.Unzip it and move it to a very short directory and run foamToEnsight. this will finish with success.

2.move the case to a directory as long as possible and run foamToEnsight. i got the float point exception error.
TagsNo tags attached.

Activities

user1281

2015-12-09 10:55

 

channelToEnsight.zip (34,957 bytes)

wyldckat

2015-12-10 21:08

updater   ~0005731

I've tried to reproduce the error you've reported and I haven't managed to do it yet.
I have a folder named "~/Desktop/mantis/bug1944/mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm" and the conversion worked without problems for an adaptation of the tutorial "incompressible/pimpleFoam/channel395", while using OpenFOAM 2.3.0.


I tried using the case you provided, but I got the following error:

  --> FOAM FATAL IO ERROR:
  Cannot open file.

  file: /home/ofuser/OpenFOAM/ofuser-2.3.0/run/river/river1/rawData/upZeta.txt at line 1.


Please provide the complete details which can be used to reproduce the same exact error, as well as the missing files, if possible. Otherwise it's not possible to reproduce the error.

user1281

2015-12-11 02:24

 

upZeta.txt (1,081 bytes)
51
(
(0	0)
(1800	0.124344942)
(3600	0.240876833)
(5400	0.342273548)
(7200	0.422163958)
(9000	0.475528255)
(10800	0.499013363)
(12600	0.491143628)
(14400	0.452413534)
(16200	0.385256634)
(18000	0.293892643)
(19800	0.184062298)
(21600	0.062666642)
(23400	-0.062666589)
(25200	-0.184062248)
(27000	-0.2938926)
(28800	-0.3852566)
(30600	-0.452413511)
(32400	-0.491143618)
(34200	-0.499013367)
(36000	-0.475528271)
(37800	-0.422163987)
(39600	-0.342273587)
(41400	-0.24087688)
(43200	-0.124344993)
(45000	-5.35898E-08)
(46800	0.12434489)
(48600	0.240876786)
(50400	0.342273509)
(52200	0.422163929)
(54000	0.475528238)
(55800	0.49901336)
(57600	0.491143638)
(59400	0.452413556)
(61200	0.385256668)
(63000	0.293892687)
(64800	0.184062348)
(66600	0.062666695)
(68400	-0.062666536)
(70200	-0.184062199)
(72000	-0.293892557)
(73800	-0.385256565)
(75600	-0.452413488)
(77400	-0.491143608)
(79200	-0.49901337)
(81000	-0.475528288)
(82800	-0.422164016)
(84600	-0.342273626)
(86400	-0.240876927)
(88200	-0.124345045)
(90000	-1.0718E-07)

)
upZeta.txt (1,081 bytes)

user1281

2015-12-11 02:36

  ~0005732

thanks for pointing out my missing, Bruno. now the file is uploaded and you may need to change 0/zeta.boundaryField.inlet dict for the location of the file.

and on my platform i can solve the case without problems but failed to make the data conversion. if there is still any problem about the file, please let me know.

thanks again!

wyldckat

2015-12-13 15:54

updater   ~0005739

When you wrote: "move the case to a directory as long as possible" - you should have been specific about the exact number of characters you used. But I should have also been specific in asking about the exact number of characters... so we were both vague about the details :)

Anyway, I've finally managed to reproduce the problem. A path with 254 characters in total is able to trigger this error, when using OpenFOAM 2.3.0. When using 3.0.x, I got this:

  --> FOAM FATAL IO ERROR:
  keyword uniformValueCoeffs is undefined in dictionary "/home/user/Desktop/mantis/bug1944/aaaaaaaaaaaaa/bbbbbbbbbbbbbbb/cccccccccccccccc/ddddddddddddddd/eeeeeeeeeeee/ffffffffffffffffff/gggggggggggggggggg/hhhhhhhhhhhhhhhhhh/iiiiiiiiiiiiiiiiiii/jjjjjjjjjjjjj/mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm/0/zeta.boundaryField.inlet"

  file: /home/user/Desktop/mantis/bug1944/aaaaaaaaaaaaa/bbbbbbbbbbbbbbb/cccccccccccccccc/ddddddddddddddd/eeeeeeeeeeee/ffffffffffffffffff/gggggggggggggggggg/hhhhhhhhhhhhhhhhhh/iiiiiiiiiiiiiiiiiii/jjjjjjjjjjjjj/mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm/0/zeta.boundaryField.inlet from line 36 to line 41.

      From function dictionary::subDict(const word& keyword) const
      in file db/dictionary/dictionary.C at line 648.

  FOAM exiting


Using a symbolic link for the case folder also doesn't solve the problem, given that the real path is unpacked from the symbolic link.

Using a path with 256 or more characters can lead any version of OpenFOAM to complaining about not being able to get the path, given the hard-coded 256 character limit in "Foam::cwd":

  --> FOAM FATAL ERROR:
  Couldn't get the current working directory

      From function Foam::cwd()
      in file POSIX.C at line 256.

  FOAM exiting


I'm going to look into how to fix this properly, but I have to ask: Why exactly do you need a path so long?
Is it because of the folder name for identifying the case? Or is it the path to the folder that is very long, due to the machine you're using being shared with several people?

wyldckat

2015-12-13 16:59

updater  

POSIX.C (29,928 bytes)

wyldckat

2015-12-13 17:11

updater   ~0005740

@Henry: Attached is the file "POSIX.C" for the folder "src/OSspecific/POSIX" that has the ability to deal with paths with more than 256 characters in the method "Foam::cwd()".
It relies on "List<char>" instead of a hardcoded "char buf[256]" and resizes the list with +256 in length for each iteration that "getcwd" fails with "errno == ERANGE".

In theory, this will only break if the "setSize()" call fails to allocate as much memory as requested.
The attached file should work with both 3.0.x and dev, if I remember correctly that "POSIX.C" hasn't changed recently.

____________

@Karelke (cfdopenfoam): I didn't notice that the previous error I got about "uniformValueCoeffs" is because OpenFOAM 2.4/3.0/dev uses another naming convention. I changed the inlet in "zeta" to this for working with OpenFOAM 3.0/dev:

    inlet
    {
        type uniformFixedValue;
        uniformValue tableFile;
        tableFileCoeffs
        {
            outOfBounds warn;
            fileName "$FOAM_CASE/upZeta.txt";
        }
        uniformValueCoeffs
        {
          $tableFileCoeffs;
        }
    }

This way it works with both 2.3.0 and 2.4/3.0/dev versions.

As for sigFpe error you're getting due to the path with ~254 characters, it no longer occurs with the more recent versions of OpenFOAM.

But please do answer the questions I asked above, namely:

  Why exactly do you need a path so long?
  Is it because of the folder name for identifying the case?
  Or is it the path to the folder that is very long, due to the machine you're using being shared with several people?

henry

2015-12-13 17:18

manager   ~0005741

I think it would be slightly better to use DynamicList rather List given that the list is resized as necessary.

henry

2015-12-13 17:24

manager   ~0005742

I will make the change to DynamicList. Also I think it would be a good idea to include a hard-coded upper limit in case it goes into an allocation loop because errno is set to ERANGE due to some other problem. What should the upper limit be? If it is modest is it worth the effort of resizing, why not set the buffer to the upper limit? Or maybe set the size to 256 initially and the first time it goes over set it to the upper limit and if it goes over the upper-limit issue a fatal error?

wyldckat

2015-12-13 17:37

updater   ~0005743

I also thought about that, namely the issue with the upper limit not being bound.
In Linux, the limit is defined by PATH_MAX as 4096 in "/usr/include/linux/limits.h". Problem is that this isn't the POSIX way of defining this limit.

The other problem is mostly on how much future-proof the code should be.
As for always setting to 4096 as the default size for long paths, seems overkill in most situations... although the construction of "fileName" will automatically trim to the first null character, making it always shorter than 4096.

The file systems on Linux do not have hard-coded limits on the path lengths: https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits - it's only the interfacing code that is restricted, hence the 4096 value can change in future versions.


By the way, I later noticed that the attached POSIX.C won't work with 3.0.x, due to a small change made in the commit "ErrorIn -> ...ErrorInFunction".

henry

2015-12-13 17:44

manager   ~0005744

ErrorInFunction is also supported in OpenFOAM-3.0.x or did I miss some?

wyldckat

2015-12-13 17:49

updater   ~0005745

Last edited: 2015-12-13 17:50

View 2 revisions

I copied POSIX.C from dev to 3.0.x and it built without any error messages nor related warnings. But there are a lot of these updates that weren't done in 3.0.x yet, so I wasn't sure it would have worked.

henry

2015-12-13 18:37

manager   ~0005747

I have added an upper-limit which can be changed in POSIX.H as necessary in the future. See commit 72bc2ac4a901616d60fd5fe99f3965175838f9f8 in OpenFOAM-dev.

wyldckat

2015-12-13 19:04

updater   ~0005749

Tested with a 410 character path with success.
Tested with a 3602 character path and it gave me this error message:

    --> FOAM FATAL IO ERROR:
    string "/home/user/Desktop/mantis/bug1944/aaaaaaaaaaaaa/bbbbbbbbbbbbbbb/cccccccccccccccc..."
        is too long (max. 1024 characters)

    file: IStringStream.sourceFile at line 0.

        From function virtual Foam::Istream& Foam::ISstream::read(Foam::string&)
        in file db/IOstreams/Sstreams/ISstream.C at line 561.

    FOAM exiting


Took a look into the class "Foam::ISstream" and it has several hard-coded limits, some of 128, others of 1024 and others of 8000, depending on what each method is designed to read.

I still have to take a better look into how DynamicList does its job (namely if it resizes automatically upon usage), but perhaps simply replacing in "Foam::ISstream" these entries:

  static char buf[maxLen];

with

  static DynamicList<char> buf(maxLen);

and then adapting the limiter check to handle only when a resize fails, would make this class future-proof. Then again, this feels like performance sensitive class, so it really depends on how DynamicList can do this.

wyldckat

2015-12-13 21:14

updater  

ISstream.C (17,485 bytes)

wyldckat

2015-12-13 21:14

updater  

ISstream.H (5,739 bytes)

wyldckat

2015-12-13 21:14

updater  

ISstreamI.H (2,472 bytes)

wyldckat

2015-12-13 21:25

updater   ~0005755

This is still a preliminary proposition, but I've attached 3 files for the folder "src/OpenFOAM/db/IOstreams/Sstreams" (for OpenFOAM-dev):

  - ISstream.H - Now using a private buffer "DynamicList<char> buf".

  - ISstreamI.H - The "buf" variable is initialized at 128 characters.

  - ISstream.C - Several changes, all related to dropping the "static char buf[maxLen]" local definition for each method that needed it and instead using the private "buf" dynamic list. Most of the "avoid buffer overflow" checks were removed, given that the limit is now triggered when the resizing fails on its own.

This still needs to be checked on:
 - how it impacts performance;
 - what happens when the resizing fails;
 - fully testing all read methods that use "buf" to double check they still parse properly.

I did do a few tests with:
 1 - generating with blockMesh a 2 million cell mesh, in "ascii" output format;
 2 - then running checkMesh;
 3 - then foamFormatConvert;
 4 - and then checkMesh again.

Then compared with using the same in OpenFOAM 3.0.0, i.e. with the original code. The mesh files were identical, apart from the version in the headers, and the output from checkMesh was also identical.

I won't be able to run these tests before the next weekend :(

user1281

2015-12-14 01:58

  ~0005758

thanks for your responsible efforts, Bruno and Henry.

@Bruno: my exact path of the case is /home/yk/OpenFOAM/yk-2.3.0/run/shallowPimpleFoam/rectChannel

i use this for identifying the case and if it is the exact casePath, then the foamToEnsight utility will do breakdown (as in the Description). and i never got the FOAM FATAL ERROR: Couldn't get the current working directory...

however, if the casePath is /home/qq/rectChannel, then i can make the data conversion using foamToEnsight with success.

it's strange that i only changed the case path so that i made foamToEnsight work.

Issue History

Date Modified Username Field Change
2015-12-09 10:55 user1281 New Issue
2015-12-09 10:55 user1281 File Added: channelToEnsight.zip
2015-12-10 21:08 wyldckat Note Added: 0005731
2015-12-11 02:24 user1281 File Added: upZeta.txt
2015-12-11 02:36 user1281 Note Added: 0005732
2015-12-13 15:54 wyldckat Note Added: 0005739
2015-12-13 16:59 wyldckat File Added: POSIX.C
2015-12-13 17:11 wyldckat Note Added: 0005740
2015-12-13 17:11 wyldckat Assigned To => henry
2015-12-13 17:11 wyldckat Status new => assigned
2015-12-13 17:18 henry Note Added: 0005741
2015-12-13 17:24 henry Note Added: 0005742
2015-12-13 17:37 wyldckat Note Added: 0005743
2015-12-13 17:44 henry Note Added: 0005744
2015-12-13 17:49 wyldckat Note Added: 0005745
2015-12-13 17:50 wyldckat Note Edited: 0005745 View Revisions
2015-12-13 18:37 henry Note Added: 0005747
2015-12-13 19:04 wyldckat Note Added: 0005749
2015-12-13 21:14 wyldckat File Added: ISstream.C
2015-12-13 21:14 wyldckat File Added: ISstream.H
2015-12-13 21:14 wyldckat File Added: ISstreamI.H
2015-12-13 21:25 wyldckat Note Added: 0005755
2015-12-14 01:58 user1281 Note Added: 0005758
2016-02-02 18:30 henry Status assigned => closed
2016-02-02 18:30 henry Resolution open => unable to reproduce