Re: Price Comparison Website Questions
You could write a multi-threaded screen scraper which would glean the pricing/availability info from the vendor sites in realtime.
This is pretty easy if there is a common "item number" like an ISBN for books. If there is not, then you will need to set up a process to "crawl" a vendor's site daily to store a mapping between your item # and each vendor's item #.
I did something this about 7 years ago using ASP and MTS. It typically went to a vendor's product page using the Item # and parsed the HTML using a vendor-specific RegEx expression. I did this for multiple vendors concurrently and had a timeout to "give up" on a vendor if that request took too long.
That's all I can remember w/o reviewing the code.
|