Developer forum

Forum » Development » Data Integration - Custom + Ecom Provider

Data Integration - Custom + Ecom Provider

Nuno Aguiar
Reply

Hi,

 

We are developing an Integration Provider, but due to "bad" data on the supplier side, we need some help determining the best appoach.

 

Source Data:

 - Manufactureurs  - ID is string

 - Products

    - Tax Group is Number (we need to get our corrresponding VATGroupID)

    - Manufacturer is String (we need to get our corresponding Manufacturer ID)

    - Name is concatened string (we need to parse/split the value according to the ecom language in the Destination parameters)

    - Others string values (we need to apply them to category fields and the ProductNumber holds the ERP primary key (not the ProductID))

 

My questions are:

  - How can we address the top issues?

  - Can we create a chain of jobs by running just 1? - In case we need sync manufactures and Vat groups first?

 

Best Regards,

Nuno


Replies

 
Morten Snedker
Reply

Hi Nuno,

Just to clarify, before answering "how can we address the top issues":

When you write "developing an Integration Provider", does that mean that you will be coding your own provider that inherits from ours? Or do you just mean "building a new set of jobs via UI"?

Is your source file CSV or XML?

As for "creating a chain of jobs": Yes, you can. If you are using Ecom Provider we implicitly know the order of importing. So you may have one job containing import of products, groups, manufactures etc, and they will automatically be imported in proper order.

Though, if you want to split into several jobs, you may do so as well. You may call the jobrunner once, with multiple jobs to be executed. They will be executed in the order that you put them. You may use the URL builder to create the URL for executing multiple jobs.

 

Let me have a comment on the above.

 

Regards /Snedker

 

 
Nuno Aguiar
Reply

Hi Snedker,

 

We are developing our own (source) provider. We get data from a webservice that converts the ws methods into temporary tables.

 

Chain of imports

We are using a single Data Integration Job, so if the Ecom Provider knows what to do, that's great!

 

If you need, I can provide you with a link so you can give it a look.

So how about the "related" data (manufacturers, tax groups) without corresponding ID's, what's the best way to do that?

 

Nuno

 
Morten Snedker
Reply
This post has been marked as an answer

Unless specified otherwise, the actual key column(s) of the table will be the key being matched upon import. However, if you try creating a test job, having Ecom Provider as destination, you will in the job file find that each column has the attribute "isKey" that is either true or false. You may set this attribute to perform a lookup at any column. It may be a single column og multiple columns. On 8.4 this setting is available also from the backend UI, second icon next to each table mapping (triggers a pop-up window).

So, you may perform a lookup on ManufacturerName rather than ManufacturerID by setting "isKey" to true for ManufacturerName, and to false for column ManufacturerID.

I think this answers your questions - otherwise let me know.

 

Regards /Snedker

Votes for this answer: 1
 
Morten Snedker
Reply

PS: If you perform lookup on another column than the "real" key column (using isKey), you should make sure that there's no duplicate content on the given key. If multiple rows apply the value of the key, all of these will be updated.

 
Marco Santos
Reply

Good morning.

Where are the job files located?

Marco

 
Jonas Krarup Dam
Reply

Hi Marco,

 

They are located in the files archive, in the folder files/integration/jobs/

 

Regards, Jonas

 
Nuno Aguiar
Reply

Hi Snedker and Jonas,

 

We will try to adapt the provider. Thanks for the important info. If we run into any other complications, we will bug you again :P

 

Regards,

Nuno

 
Marco Santos
Reply

Hello.

I'm still having trouble with this and am not sure how to fix it, so here are some extra details, see if you can think of anything.

In our ISource I have (among other fields) the equivalent of the ProductNumber and ProductLanguageId in the EcomProducts. There are no keys in the source schema, but I am filtering the source by number and language, so to get only distinct values.

In the job mapping I am only mapping as table keys the ProductNumber, ProductLanguageId and ProductVariantId. When I run the job I get this error:

Job Failed with the following message: Failed to move data from temporary table [dbo.EcomProductsTempTableForBulkImport] to main table [dbo.EcomProducts]. Exception message: Violation of PRIMARY KEY constraint 'EcomProducts$PrimaryKey'. Cannot insert duplicate key in object 'dbo.EcomProducts'. The duplicate key value is (PROD1, LANG1, ).

Any Ideas on how to get around it in order to update/create the data already in the system?

Marco.

 
Nuno Aguiar
Reply

Hi Jonas and Morten,

 

Any ideas on Marco's problem? We are a bit stuck sad

 

Nuno

 
Jonas Krarup Dam
Reply

Hey Marco (and Nuno)

Which provider are you using as your destination?

It sounds like it is probably the Dynamicweb Provider.

when using the dynamicweb provider, the ProductID is NOT generated automatically, if it is not included in the mapping, and you will get an error similar to the one you describe here.

The simplest solution is to map your ProductNumber to the ProductID column as well - as long as the productNumber is unique for each row, it should work just fine.

If that is not your problem, please let me know, and I'll see what else I can think of.

Regards, Jonas

 
Marco Santos
Reply

Morning.

The problem has been solved. It was a mix of bugs in our source and mapping configuration. We are using the Ecom Provider, by the way.

Thanks for the help.

Marco

 

You must be logged in to post in the forum