|
||||||||||
|
Follow @CraftingDir
Number of listings removed from our directory since 1st November 2019 = 55
|
Ensuring our Crafts directory is FRESH!
Total No. of sites temporarily removed from directory due to offline status: 2
No. of sites temporarily removed TODAY from directory due to offline status: 0
No. of URLs updated in total: 86 (8.23% of total listings)
No. of URLs updated TODAY: 0 (0% of total listings)
No. of websites spidered today: 128 (12.25% of total listings) - view list
Curating an online website directory doesn't just mean adding websites to it. Websites go offline, get repurposed, get 301'd to a different domain. There is a LOT of "churn" online, and our directory has to reflect these changes regularly. Most directories fail at these tasks...you will see a LOT of "link rot" in the vast majority of directories online - you browse a category, you click on twenty links, and five or more are "dead" links, or 301'd to different domains owned by different companies.
There are three things we need to be aware of: Let's look at all three, and see live data from our directory of examples where this is happening right now. Listings that are considered "temporarily" offlineThese are listings that our spider has detected have been offline for up-to the last 20 days. Currently we check the site every 1 day(s) to check its status. It has to have "20 strikes" against it (roughly 20 days in total) before it is removed from our directory. If by that time it is still offline, then we consider it "permanently offline" as around 20 days is a LONG time for a website to be offline. Having said that, if a website comes back online within the (roughly) 20 day period, its counter is reset back to zero to be as lenient as possible to websites (as there are many reasons why sites go offline that are temporary in nature).Websites found by our spider to have been recently discovered offline
wallsofthewild.co.uk
10 day(s) offline
Found in Art Last spidered: 29/02/2020 No. of times spidered in a row and found offline:
danell.co.uk
76 day(s) offline
Found in Floral Crafts Last spidered: 28/02/2020 No. of times spidered in a row and found offline: 301 redirectsHow common are these? VERY common! Especially when redirecting from http:// to https://. However, sometimes a 301 redirects to a brand new domain name. Most of the time it's a company rebranding, but sometimes it's a completely different company that has bought the domain name (that's usually just been made available), then they do a 301 redirect to their own website. This is something we have to look out for, because the website we review can end up redirecting to another website that has nothing to do with the website we reviewed! This involves both an automated process to record the 301 redirect, and ultimately a human-review process to ensure the new domain name belongs to the same company as the previous domain name, and that the nature of the website is the same.Even if it's just a regular http:// to https:// redirect, it's good housekeeping to update our directory to reflect that change. 301 redirects discovered in the last 3 days
https://www.sonyawinner.com/ 301'd to >>
https://www.sonyawinner.com/?v=79cba1185463 [internal change] Change discovered on 28 February 2020
https://opsandops.com/ 301'd to >>
View full list of 301 redirects >>
https://opsandops.com/?v=79cba1185463 [internal change] Change discovered on 27 February 2020 Websites that are considered permanently offlineSometimes it happens - a website is offline too long, and it is considered "dead". Therefore, we have to remove it from our directory. We have a page dedicated to websites we have removed from our directory for this reason. Note, that these are NOT websites we have rejected initially (that would be a MUCH bigger list, and it's not one we need to show here!). |
