From 70aba9458260aa9073949830e079df6d3cabca7a Mon Sep 17 00:00:00 2001 From: Anthony Lacruz Date: Tue, 17 May 2022 21:13:31 -0500 Subject: [PATCH] links/DMOZ: url updated --- solutions/system_design/web_crawler/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/system_design/web_crawler/README.md b/solutions/system_design/web_crawler/README.md index e6e79ad2..411197ee 100644 --- a/solutions/system_design/web_crawler/README.md +++ b/solutions/system_design/web_crawler/README.md @@ -77,7 +77,7 @@ Handy conversion guide: ### Use case: Service crawls a list of urls -We'll assume we have an initial list of `links_to_crawl` ranked initially based on overall site popularity. If this is not a reasonable assumption, we can seed the crawler with popular sites that link to outside content such as [Yahoo](https://www.yahoo.com/), [DMOZ](http://www.dmoz.org/), etc. +We'll assume we have an initial list of `links_to_crawl` ranked initially based on overall site popularity. If this is not a reasonable assumption, we can seed the crawler with popular sites that link to outside content such as [Yahoo](https://www.yahoo.com/), [DMOZ](https://dmoz-odp.org/), etc. We'll use a table `crawled_links` to store processed links and their page signatures.