We’ve been running a multilingual XML feed from Dynamicweb 10 to EasyCatalog for quite some time. It has always worked well, but recently the feed has grown so large that it now takes over 2 hours to fully load. This has made live triggering by EasyCatalog impractical and unstable.
The Problem
The XML feed has become too heavy for live generation. We're now exploring alternative approaches to keep it usable and performant.
Options We’re Considering
-
Pre-generating the XML on a scheduled basis and delivering it as a static file (e.g. via FTP).
-
Using the 65bit pagination tool for EasyCatalog to reduce the data volume per request (still being evaluated).
-
Rebuilding the feed logic as a Dynamicweb Integration Activity, though we're unsure if this is suitable given the Razor complexity and volume of joins.
Questions
-
What’s the recommended approach in Dynamicweb for handling very large XML feeds like this?
-
Does anyone have experience with 65bit pagination module in combination with Dynamicweb feeds?
-
Is using an Integration Framework provider a better long-term solution?
Any input is greatly appreciated!