Most time in my program is now spent on the ASCII-dump of a lot of 2D arrays every timestep. Essentially I just do:
Open a file
Write a few headerlines…
close the file…
How can I speed this up substantially?
The file-format is fixed by the requirements of the program that has to read my output. This means that binary files e.g. are not an option.
Sorry Arjan, not much you can do here. If it was unformatted, then you might be able to block the output to make it a bit faster, but here you’re limited by the I/O speed of you system then anything else.
Can’t I first convert the FORMATTED data to something intermediate
that can be written as UNFORMATTED, such that the files created
effectively are FORMATTED? If so, how?
Maybe if you first formatted your data into an internal file and then wrote the entire data to external file at once. I have not tried this myself, nor know if it would help, but it’s worth a try.
Of course, if the amount of data was greater then your available system memory, then this would be a bad idea. So you might need to break up the data into chunks depending upon the amount of memory you have.