Developer forum

Forum » Integration » Perform updates with the Data Integration Framework

Perform updates with the Data Integration Framework

Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Hi there,

I am trying to import data from a custom XML file to a custom SQL table using the SQL Provider. The table looks as follows:

CREATE TABLE [dbo].[CustomPrices](
    [ContractId] [NVARCHAR](30) NOT NULL,
    [ProductId] [NVARCHAR](30) NOT NULL,
    [Price] [DECIMAL](18, 0) NOT NULL,
    [MarketingNotes] [NVARCHAR](MAX) NULL,
 CONSTRAINT [PK_CustomPrices] PRIMARY KEY CLUSTERED
(
    [ContractId] ASC,
    [ProductId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

The primary key of the table is a composite of ContractId and ProductId.

In the UI these two columns are correctly inferred as keys (I can see they are selected when I click Set Keys for the table). Yet in the XML they are not:

<name>ContractId</name><type>System.String</type><isNew>False</isNew><isPrimaryKey>False</isPrimaryKey>

When I execute the import job a second time, it fails with the eror:

Import job failed: failed to move data from temporary table [dbo.CustomPricesTempTableForSqlProviderImport] to main table [dbo.CustomPrices]. Exception: Violation of PRIMARY KEY constraint 'PK_CustomPrices'. Cannot insert duplicate key in object 'dbo.CustomPrices'. The duplicate key value is (, 6552). The statement has been terminated. Sql query: update [dbo].[CustomPrices] set [ContractId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ContractId], [ProductId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ProductId], [Price]=[dbo].[CustomPricesTempTableForSqlProviderImport].[Price], [MarketingNotes]=[dbo].[CustomPricesTempTableForSqlProviderImport].[MarketingNotes] from [dbo].[CustomPricesTempTableForSqlProviderImport] where [dbo].[CustomPrices].[ContractId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ContractId] and [dbo].[CustomPrices].[ProductId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ProductId] ; insert into [dbo].[CustomPrices] ([ContractId], [ProductId], [Price], [MarketingNotes]) (select [dbo].[CustomPricesTempTableForSqlProviderImport].[ContractId], [dbo].[CustomPricesTempTableForSqlProviderImport].[ProductId], [dbo].[CustomPricesTempTableForSqlProviderImport].[Price], [dbo].[CustomPricesTempTableForSqlProviderImport].[MarketingNotes] from [dbo].[CustomPricesTempTableForSqlProviderImport] left outer join [dbo].[CustomPrices] on [dbo].[CustomPrices].[ContractId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ContractId] and [dbo].[CustomPrices].[ProductId]=[dbo].[CustomPricesTempTableForSqlProviderImport].[ProductId] where [dbo].[CustomPrices].[ContractId] is null);

How do I tell the DIF to use the composite key and use an UPDATE rather than an INSERT when a duplicate key is found?

Thanks,

Imar


Replies

 
Jonas Krarup Dam
Reply

Hi Imar,

that sounds strange.

I just testedt this by creating a table in my own DB using your script, and importing the following xml:

<?xml version="1.0" encoding="utf-8"?>
<tables>
  <table tableName="CustomPrices">
    <item table="CustomPrices">
      <column columnName="ContractId"><![CDATA[1001]]></column>
      <column columnName="ProductId"><![CDATA[2000]]></column>
      <column columnName="Price"><![CDATA[2000]]></column>
      <column columnName="MarketingNotes"><![CDATA[some price changed]]></column>
    </item>
  </table>
</tables>

And this worked just fine, both for inserting and updating.


Are you working with an older version of Dynamicweb?

If I am to look into this more, I think I'll need a copy of your database, and the XML file you are trying to import.

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Hi Jonas,

Sorry for wasting your time; it turned out to be an issue with the XSLT file. This: "The duplicate key value is (, 6552)" should have given it away to me. The first value (the ContractId) was empty. I had multiple products with an ID of 6552 for different contracts and the missing ContractId caused this exception.

I noticed that the job's XML is over 2MB. It seems that the entire database schema is stored in the job, and not just the target table that I have selected. Is this by design?

Imar

 
Jonas Krarup Dam
Reply

hi,

No worries  - I'm glad you figured it out.

It is true that the job contains the full schema for both source and destination. This is a concious choice, but I think it was mostly to keep the code for saving a job simple.

 

/Jonas

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Right, thanks for clearing that up.

Imar

 

 

You must be logged in to post in the forum