The system parses XML product-feeds and puts them in a database into a through user friendly and efficient system.
The system should goes through a bunch of product-feeds and then inserts item per item into the database. The structure is different per feed, although there are feeds with very similar structures. This depends on the intermediate network that is supplying the XML feed, they create the structure.
The product-feeds will be inserted into system from non technical user, he retrieves an XML feed link from the intermediate system and a name of the source daily deal site. He inputs the feed link and the name into the system and the system processes it. The system detect the fields and present them. for example lets say that one feed has 15 item blocks. Each item block has different feed fields. With every of these fields, it should propose a fix field from database table to map to and give some other options too. These other options are a field function. The field functions convert the value of the field and act like filters on the input value.
After everything is entered, the user also sets an interval (every X minutes or every X hours). The feed then goes in the database and the system processes it regularly. It handles any errors (processing errors, unexpected/incomplete data, 404 errors, timeout errors) and recovers from these. The user should be able to set the status of a feed: active or paused.
A dashboard table gives an overview of all the feeds, shows the status, the last processed timestamp and the number of items added during the last processing. It also has a column that shows the number of errors during the last parse. If this number is higher than 0, the whole row is marked with a red background.
