Developer forum

Forum » Integration » Avoid errors when importing CSV files

Avoid errors when importing CSV files

Tomas Gomez
Reply

Hi,

We get the data from our customer in CSV files that we import in DW by integration tasks. There are many files and they contain lot of data, so errors are frequent. 

For instance, we notice that a task error is thrown if there are not valid CSV files at the same foder (even if they are not used by the integration). As well, an error is thrown if there are empty columns inserted within the data (for visual separation of data blocks).

In the Source setting of the task, there is an option to "Skip troublesome rows" but it only avoid some kind of errors.

Is the any othe option to avoid these errors when integrating CSV files? Which are your suggestion?

Regards,
Tomas


Replies

 
Tomas Gomez
Reply

Hi again,

Beside the aformentioned issues, there are other human-mistakes desiderable to check. For instance, avoid non-printable characters (tab, return), custom format of cell contents, etc.

At your exprience, how would you automatically detect these CSV errors? I can think of several alternatives

  • In the Excel file, create a macro to check the format within the CSV file
  • In DW content, create an private page to read the file and display the errors
  • In DW tasks, create a task to do it. Similar as previous, maybe more porwerful but need compilation

Regards,
Tomas

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply
This post has been marked as an answer

Hi Tomas,

I agree with what you are saying. We have experienced similar issues before and we had to manually check every file before we could identify all the troublesome data.

It would be good to have this type of check built-in in the CSV provider (and probably also in the Excel provider too).

Maybe it is worths adding a feature request?

Adrian

Votes for this answer: 1
 
Tomas Gomez
Reply

Thanks Adrian for your suggestion.

I posted a new feture request at Avoid errors when importing Excel/CSV files

Regards,
Tomas

 

 

You must be logged in to post in the forum