|
Number of listings removed from our directory since 1st November 2019 = 106
|
Ensuring our Crafts directory is FRESH!
Total No. of sites temporarily removed from directory due to offline status: 3
No. of sites temporarily removed TODAY from directory due to offline status: 2
No. of URLs updated in total: 29 (2.49% of total listings)
No. of URLs updated TODAY: 1 (0.09% of total listings)
No. of websites spidered today: 647 (55.54% of total listings) - view list
Curating an online website directory doesn't just mean adding websites to it. Websites go offline, get repurposed, get 301'd to a different domain. They are also being updated all the time. There is a LOT of "churn" online, and our directory has to reflect these changes regularly. Most directories fail at these tasks...you will see a LOT of "link rot" in the vast majority of directories online - you browse a category, you click on twenty links, and five or more are "dead" links, or 301'd to different domains owned by different companies.
A selection of live updates from our directoriesHow do we keep our directory fresh?There are four things we need to be aware of:
Listings that are considered "temporarily" offlineThese are listings that our spider has detected have been offline for up-to the last 20 days. Currently we check the site every 1 day(s) to check its status. It has to have "20 strikes" against it (roughly 20 days in total) before it is removed from our directory. If by that time it is still offline, then we consider it "permanently offline" as around 20 days is a LONG time for a website to be offline. Having said that, if a website comes back online within the (roughly) 20 day period, its counter is reset back to zero to be as lenient as possible to websites (as there are many reasons why sites go offline that are temporary in nature).Websites found by our spider to have been recently discovered offline
amazinglypersonalised.co.uk
offline today Found in Personalised Gifts Last spidered: 27/11/2020 No. of times spidered in a row and found offline:
londondesignguide.com
offline today Found in General Art and Craft Last spidered: 27/11/2020 No. of times spidered in a row and found offline:
annamarias.co.uk
65 day(s) offline Found in Handmade Jewellery Last spidered: 27/11/2020 No. of times spidered in a row and found offline: 301 redirectsHow common are these? VERY common! Especially when redirecting from http:// to https://. However, sometimes a 301 redirects to a brand new domain name. Most of the time it's a company rebranding, but sometimes it's a completely different company that has bought the domain name (that's usually just been made available), then they do a 301 redirect to their own website. This is something we have to look out for, because the website we review can end up redirecting to another website that has nothing to do with the website we reviewed! This involves both an automated process to record the 301 redirect, and ultimately a human-review process to ensure the new domain name belongs to the same company as the previous domain name, and that the nature of the website is the same.Even if it's just a regular http:// to https:// redirect, it's good housekeeping to update our directory to reflect that change. 301 redirects discovered in the last 14 days
https://www.goldelite.com/password 301'd to >>
https://www.goldelite.com/ [internal change] Change discovered on 25 November 2020
http://goldelite.com/ 301'd to >>
https://www.goldelite.com/password [http to https] Change discovered on 24 November 2020
https://thebaobabtree.co.uk/ 301'd to >>
https://www.thebaobabtree.co.uk/ [internal change] Change discovered on 22 November 2020
https://www.whatnotwoodcraft.co.uk/ 301'd to >>
https://www.whatnotwoodcraft.co.uk/password [internal change] Change discovered on 18 November 2020
http://www.thorody.com/ 301'd to >>
View full list of 301 redirects >>
https://www.thorody.com/ [http to https] Change discovered on 15 November 2020 Websites that are considered permanently offlineSometimes it happens - a website is offline too long, and it is considered "dead". Therefore, we have to remove it from our directory. We have a page dedicated to websites we have removed from our directory for this reason. Note, that these are NOT websites we have rejected initially (that would be a MUCH bigger list, and it's not one we need to show here!).Newly created and updated website contentWe spider the listings in our directory constantly throughout the day, 24/7. Our spider is looking for new content: new articles, videos, social posts, new pages, new products - maybe an updated phone number or address. Maybe there's new accreditations or associations added to the site. If we find them, we update the listing. |
