Use a stable listing identifier when the directory provides one, and keep the original URL with each record. If no identifier exists, define a cautious matching rule using several fields, such as address and phone number, while leaving uncertain cases for review. Also record when each page was fetched, because a moving sort order can place one listing in different positions during the same run. A proxy handles the connection; it does not make the source data static. Report distinct records separately from total rows collected so duplicate handling is visible instead of hidden inside a final number.
|