Hi all,
I’m still quite new to timextender so I appreciate all the help you guys give me.
I have quite alot of different XML files containing alot of data. It’s product catalogs with porentially 40K products that contains different kind of parameters .. each product’s parameters varies from product to product .. som tell you the country of origin, som dont .. on some of the products there is information about delivery dates some and on some there aren’t.
So all-in-all the different products and the different files varies a quite alot.
The content of these files can change each day and some multiple times each day .. I would think that we all in all are talking about 600.000 different products.
How would you approach this ?
Is it possible for timextender to handle this or do I have to make some kind of XML-parser ?
I guess I’m going to make a unique key there is based on the products no. and another parameter, but would you predefine the different parameters as a Field so I have to add a new field when there is a new parameter in the XML files, or are smarter and more dynmaic way to do this ?
Dynamic XML as datasource
Best answer by Thomas Lind
Hi
If you get stuck, do not hesitate to contact us. If it needs to be more secure, you can always send it in a message to support@timextender.com and we will handle it in a Zendesk ticket instead.
+7Hi,
product catalogue implementations tend to follow the EAV-like models which is not very conducive to reporting scenarios. You will need to decide what kind of model to apply and can then use this feature to decompose the data:
I think you will need to do some XML SQL to map all your input into Entity, Attribute, Value tuples.
If your XML structures have little overlap, you will have few dimensional attributes to filter on. Similar to how an online store will not have filters for “number of toggle switches” across all their products.
+5How do you connect to the XML files? Does the changes come through in a new file or will existing ones be updated?
I was thinking that synchronizing a lot of XML files multiple times a day would be really labor intensive.
How do you connect to the XML files? Does the changes come through in a new file or will existing ones be updated?
I was thinking that synchronizing a lot of XML files multiple times a day would be really labor intensive.
that is not determined yet, but I thought of just getting them from a storage facility like a FTP-server that I have access to .. they will probably be updated every day.
+5The XML provider we have from CData will merge all files with a similar structure into one table.
So if you have multiple files with different names but the actual content, meaning tags and so, have the same names it will be merged into one table or multiple depending on the data model setting.
If you can control that part of the files, you can use the Metadata discovery URI field to point at one file to synchronize against to save on time when doing this.
+5Hi
Hi
both yes and no .. my problem is that there is only a certain amount of the data that is always “the same” .. so I’m still not sure how to solve this, but I’m pretty sure that I’m going to code some kind of parser, and then see if I can make a mapping to a uniform datastructure (or ingest into a DB) and then make a connection from TX to this DB ..
but I haven’t not cracked it yet ..
+5Hi
If you get stuck, do not hesitate to contact us. If it needs to be more secure, you can always send it in a message to support@timextender.com and we will handle it in a Zendesk ticket instead.
Reply
Most helpful members this week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
