Hi,

I have huge CSV files that I need to import a database. For this approach, I have to write my own import function. The code works fine, except... it takes time to import files (sometimes an hour!)

Basically, the process is
Code:
1 ) open file
2 ) read file - fgets()
3 ) split each line into tokens
4 ) for each read token
     4.1 ) check if there are any "bad characters" which will cause SQL errors while executing SQL queries
          4.1.1 ) if there is, remove it
     4.2 ) construct SQL query
     4.3 ) update database
I have to perform step 4 since I had errors because of bad characters before. But evaluating each character really takes time.

To improve the overall performance, I was thinking to map the file into memory before fgets() but mmap() is not portable

any recommendations are welcome....