Case sensitivity issue with OData and delta replication

E-mail notifications

Imar Spaanjaars

Posted on 14/08/2024 13:08:09

Hi there,

I am trying to use Delta replication in the OData provider but I think I have hit a case sensitivity issue. My JSON looks as follows;

"value": [
{
"no": "823332",
"description": "FIKTIV-1-2",
"description2": "FIKTIV-1",
...
"lastDateTimeModified": "2024-08-09T11:40:00.423Z",
"dimensions": "16CM",
...

Note the lower case l in lastDateTimeModified. However, in EndpointSourceReader.cs I see this code:

private string GetModeAsParameters()
{
string result = "";
if (_mode.Equals("Delta Replication", StringComparison.OrdinalIgnoreCase))
{
DateTime? lastRunDateTime = _mapping.Job.LastSuccessfulRun;
if (lastRunDateTime != null)
{
DateTime dateTimeInUtc = TimeZoneInfo.ConvertTimeToUtc(lastRunDateTime.Value);
string dateTimeFilterName = "";
bool isEdmDate = false;

foreach (var column in _mapping.SourceTable.Columns)
{
switch (column.Name)
{
case "Last_Date_Modified":
dateTimeFilterName = "Last_Date_Modified";
isEdmDate = true;
break;
case "Order_Date":
dateTimeFilterName = "Order_Date";
isEdmDate = true;
break;
case "LastDateTimeModified":
dateTimeFilterName = "LastDateTimeModified";
break;
case "lastModifiedDateTime":
dateTimeFilterName = "lastModifiedDateTime";
break;
case "modifiedon":
dateTimeFilterName = "modifiedon";
break;
}
}

This code switches on the column name and then tries to match with LastDateTimeModified using an upper case L.

Can / should this code be updated to treat the names ignoring casing?

Imar

Replies

Rasmus Sanggaard

Posted on 14/08/2024 13:18:56

Hi Imar,

Which endpoint is giving you that modifier?

Which version of the OData provider are you using? In the newer versions you can set the Delta Modifer to whatever you like.

BR Rasmus Sanggaard

Imar Spaanjaars

Posted on 14/08/2024 14:05:24

It's a custom end point built by an ERP partner. Not sure where that column is coming from exactly but I have no control over it.

I added the custom modifier which now adds the filter, but always with DateTime.MinValue as the value:

&$filter=(no+eq+%27823332%27)+and+lastDateTimeModified+gt+0001-01-01z'

I do see the last run / results files for the job and they contain a valid date and Completed as expected:

I'm running on 9.17.6 with Dynamicweb.DataIntegration.Providers.ODataProvider.dll version 3.0.7 (just upgraded it but that doesn't seem to make a difference).

Is there anything else I need to upgrade to make this work?

Rasmus Sanggaard

Posted on 14/08/2024 15:54:26

Hi,

Yes DateTime.MinValue is the fallback. We look at the time from lastrun.log. Could you send that to me, then we can maybe see what is happening.

BR Rasmus Sanggaard

Imar Spaanjaars

Posted on 14/08/2024 16:28:26

I think I found it. I attached a debugger and noticed this in GetTheDateTimeInZeroTimeZone:

So the conversion from 08/14/2024 fails and results in DateTime.MinValue.

The date in the log file uses this format:

2024-08-14T16:09:39

so it may be the current culture on the solution that causes things to fail. I have my (local) system set to a US culture but I am also seeing the same issue on dw-STA001 where this site is hosted.

OData+test+PimItems_lastrun.txt

Rasmus Sanggaard

Posted on 15/08/2024 08:35:33

This post has been marked as an answer

Hi,

Thanks for the investigation! We will take a look at why dataintegration fails to convert the format. Bug #20755.

BR Rasmus Sanggaard

Votes for this answer: 1

Imar Spaanjaars

Posted on 22/08/2024 11:15:55

FYI: I just upgraded to Dynamicweb.DataIntegration.Providers.ODataProvider.dll version 3.0.8 and that has resolved the issue; the last run date is now appended correctly.

Thanks!

Imar

You must be logged in to post in the forum

Developer forum

Case sensitivity issue with OData and delta replication

Replies