Developer forum

Hello Dmitriy,

- I have tried to follow the suggestion, but I still have some issues using the XML Provider to read any number of files from a folder.

I have created an activity like the one below (image #1), but I don't know if the XSLT is applied for every single file found in the folder - or is all files read, before the XSLT is applied?

Currently (if I have 3 files in the folder), I will have 3x mappings and it would produce one XML file with 3x orders.  If I later have 10 files in the folder, only the first 3 files are proccessed.

Image #1:

Image #2:

Br. Michael Knudsen

Hi Michael,
yes, the xslt is applied first so all your source files will be parsed and saved in a new files with a new time stamp and *.xt extension.

It is needed to have one file per one table. So if you have made a job with 3 tables in the mappings then it will not use 10 files/table mappings. Instead you can try to make the new job with "Create auto mappings" = true so it could make a mappings based on your dynamically changed source files.

By the way it is not clear what problem are you trying to solve :)

BR, Dmitrij

Hello Dmitriy,

- I'm not sure if you're kidding me, but I really don't understand, why it's not clear what problem I'm trying to solve!

A very basic import of XML files from a folder is not possible, simply because the XML file contains more than 1 table!  That's the first problem. That can be solved by reading one table at the time, not pretty but doable.

Next problem is that "Perform mapping automatically on run" only seems to work on field mappings not on a dynamically number of source files! When the activity is created with one source file and "Perform mapping automatically on run" is checked, that activity will only read the first XML file in the folder, even if there is added multiple files!

Do you find it clear now?

Br. Michael Knudsen

Hi Michael,
yes, now it is clear. Sorry if I disappointed you but I just wanted to get more details about the problem. To make your task working you need to remove the content inside the Data Integration job xml file in the <mappings></mappings> tag as shown below:

The job xml file is located in the /Files/Files/Integration/Jobs folder. Then when the job will run next times with auto-mapping ON it will always rebuild the mappings based on the dynamically changed source files. Hope it helps
Also it should build the mappings for the orders and order lines even when you have both of them in one xml file, maybe you can check if your transformed xml file looks like the one here.
BR, Dmitrij

Hello Dmitriy,

- and thanks for sharing the 'hack' with removal of the 'mappings' in the import job XML, that did the trick. First tests have been succesfully.

Br. Michael Knudsen

Hello Dmitrity,

- even this works perfect with my EcomOrders and EcomOrderLines, have I faced a new issue when using this with EcomPrices. Difference between eg. EcomOrders and EcomPrices is, that I only want the current prices to be available and all old prices have to be removed, so I have checked the "Remove missing rows..." on the destination settings.

When I check the imported prices, most of the prices are gone :-|

The Job log shows something like image below, but the problem is perhaps because each XML file with EcomPrices are imported separatly and therefore will the 'remove missing rows...' run after each file instead of running once all files have been imported to the temporary table for EcomPrices.

Do you have another 'hack' to overcome this challange? 

Br. Michael Knudsen

Hi Michael,

it depends how your files are populating, if they come from the "ImprtDataCustomRequest AddIns", there is an ExportIsDone feature available in the Xml Provider: https://doc.dynamicweb.com/Default.aspx?ID=8666#6208
So you need to set ExportIsDone=false for all of the files except the last which must have ExportIsDone=true
Then once all files are processed by DW provider the job completes. (the job mapping needs to be 1 to 1 so all data will be populated to the same temporary table before it gets imported to the main table)
But it will not work if you have all same table files in one folder and then auto mapping is on and when the job is started it will make several mappings (and that will be a problem) instead of one mapping, so several mappings then will be considered as different tables and remove missing will not work.
So you should consider to have some custom "implementation" that will combine all separate files into one file and import one file instead.
BR, Dmitrij