XML import error

Posted on 22/07/2010 09:39:05

While I was trying to import a productlist in .XML, there occoured an error that resulted in the table containing all products on the domain that I was working on, was deleted.
To avoid future accidents I need to know the following:

which encodings are allowed?
If there is a hidden error in an .XML file supplied by a customer, how do you avoid crashing the table with products?
Is it possible to create a simulation-modul that checks an .XML file up against a specific database and then checks for import errors?

best regards
Tom

Replies

Jeppe Eriksson Agger

Posted on 26/07/2010 11:15:39

Hi Tom

Which version of Dynamicweb are you using? In Dynamicweb 19.0.0.0 we changed Product Import to do all import activities on temporary tables until all of the import has completed. Then, and only then, are the main tables (EcomProducts, EcomPrices etc.) replaced. If you're using an older version then this is not the case.

Also if your entire table content is lost, then I suspect you're using either a Full or Clean import. These imports both clear the EcomProducts table. If you're unclear what they are then you're probably not using them.
The major difference is that Full doesn't delete relations between products if a certain product is imported again. It only deletes relations to deleted products that are not imported again. Clean deletes all relations. Even if a product is imported. This is used when relations are imported along with the products.

As for encodings, I would recommend using UTF-8 (as I assume you're talking about the encoding of the XML source file).

The need for a simulation module could have some merit, but with the change made to the way tables are used during the import, it shouldn't be necessary--as long as you make sure that your XML (at least when it reaches the Product Destination Activity) conforms to the schema. You can find the schema in the dialog with the settings for the activity. You might want to copy it into an editor in order to read it better :-)

Regards
Jeppe

Posted on 26/07/2010 12:09:16

Hey Jeppe. Thanks for your response. I have a few follow-up questions...
The Dynamic web version used on the particular solution is Dw version: 19.1.0.2.
Despite the security that you mention, we were still able to loose the whole product table due to a parse error, in part to an export "error" from aXapta (example: kr. 15.200.25). From what you describe we may have found a bug in the module, as there seems to be missing a type-consistancy check on the values in the XML file? Alternatively what is the situation with the database when regarding roll-back functions?

What we need is an outprint of the database table containing the tables structure I.E. column types, constraints etc... Is that possible?

Jeppe Eriksson Agger

Posted on 26/07/2010 13:39:22

Import/Export already uses Transaction-based manipulation. So if an error is detected, then the transaction is rolled back as per MSSQL standard rollback.

What type of error are you experiencing? Is it the "Yellow screen of death" (ie. exception being shown on the page) or is it the "Pipeline failed" error? If it's the failed pipeline, then do you have your error log for me to take a look at? Or if not, then maybe the data you were importing (or just a sample) so I can reproduce the error myself. If you don't want it posted here on the forum, then you can mail it directly to me jea@dynamicweb.dk.

I have uploaded a screenshot if the EcomProducts table design. This includes columns, data type and primary keys. If you need anything else; let me know.

- Jeppe

Posted on 26/07/2010 15:01:43

Thanks Jeppe.
I noticed that you mension "full" and "clean" import. I have looked in the import / eksport module as an advanced user, an am unable to find any reference to these terms in the module.. Where do you find a reference (if any) to these terms in DynamicWeb?

Posted on 28/07/2010 08:29:32

I have just attempted an import of an XML file containing products with the encoding utf-8, and recieved the following error:
"XML File navigation error: Invalid character in the given encoding. Line 65, position 26."
The encoding is : "<?xml version="1.0" encoding="utf-8"?>" (without the "")
The character in question is an 'ø'.

Jeppe Eriksson Agger

Posted on 28/07/2010 12:56:50

I have just completed a run-though of the ProductDestination import and there is indeed a bug that clear the Ecom tables if the import fails. I have fixed the bug and it will be out with hotfix 19.1.0.5.

I have, however, not been able to reproduce the issue with special characters. The only thing that comes to mind is, that the xml file might be DOS encoded and not unicode even though it says "UTF-8" in the declaration.

- Jeppe

Posted on 28/07/2010 13:38:48

Ok..
Is there any way to circumvent that encoding? For instance opening it in dreamweaver and save it again as XML.. Would that work?

Nicolai Høeg Pedersen

Posted on 28/07/2010 13:44:50

Maybe it will work. Can you open the xml file in IE without errors?

Problem with ø is that it is not encoded i think.

You can also do ctrl+k+d in Visual Studio - depending on your formatting options this could fix it too.

Posted on 28/07/2010 13:47:29

yes.. I can open the document in IE without errors, after I opened it in Dream Weaver and saved it again..

Posted on 28/07/2010 14:00:22

hmm.. It seems that I can open the .XML - file in IE only without 'æ', 'ø' and 'å', which are shown as a blank square.. I have no idea how to fix this...

Nicolai Høeg Pedersen

Posted on 28/07/2010 14:10:23

Usually the tool or framework that generates the XML encodes the characters.

Take a look at this - it will explain:
http://www.w3schools.com/XML/xml_encoding.asp

Also try opening your xml file with notepad, and choose save as. In the dialog next to the save button you can choose encoding - ansi or utf-8/unicode. Make sure it is not saved as ansi.

Posted on 28/07/2010 14:16:49

thanks np.
that did the trick!

You must be logged in to post in the forum

Developer forum

XML import error

Replies