Developer forum

Forum » Integration » Providers clearing too much cache?

Providers clearing too much cache?

Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Hi there,

In the UserProvider (and possibly others) I see this:

                if (isReadFromSourceFinished)
                {
                    Writer.FinishWriting();
                    sqlTransaction = Connection.BeginTransaction();
                    Writer.MoveDataToMainTables(sqlTransaction);
                    Writer.DeleteExcessFromMainTable(sqlTransaction);
                    sqlTransaction.Commit();
                    Writer.SendUserPasswords();
                    //Clear group cache for refreshing users in the groups they were imported
                    Group.ClearCache();
                    //Frontend.Reset.ResetHash(Frontend.Reset.HashType.Area);
                    Ecommerce.Common.Application.KillAll();

                    UpdateIndex(job);
                }

This clears a lot of cached data every time the job runs. It does that regardless of whether there were any records affected or not. On some of our sites, we run this job very frequently: every 15 minutes or so to pull in new users from an ERP. For some customers, we run this multiple times against different ERPs which means we may run this every 5 minutes.

If I am reading the code right, we're blowing away tons of Ecommerce data away every time this job runs.

That doesn't feel right to me ;-) Should the providers be a little more selective about what they clear, and at the very least try not to kill the cache when no records were imported (i.e. the source file was empty)?

Thanks!

Imar

 


Replies

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

No one? This is turning out to be an issue on a bunch of our sites....

 
Martin Vang
Martin Vang
Reply

Hi Imar,

I'm going to write this with a very positive spin; I know that we currently have a lot of possible areas for increased performance a lot, by utilizing our cache in a better way. :)

It's a work in progress, and I hope we'll be able to eventually Kill the KillAll method completely (along with all the other clear-cache methods in application) - this requires better management of our eCommerce data: When I import a User a, what related information has been changed inside the cache? -> Currently, we just don't know for sure, because our integration works directly with the database.

We have a plan in the making for refactoring all of Integration, and I will make sure to mention this benefit during the development of this work.

Sorry I cannot be of more actual help regarding this.

BR

Martin

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Hi Martin,

Yeah, I can see how it's hard to deterine what to clear exactly. But would you agree that if the import job did nothing (because there are no source records) there's no point in clearing the cache? That would be simple to check and probably solve 90% of our cache clearing issue.

Thanks!

Imar

 

You must be logged in to post in the forum