LogoNAWG

Registered charity 1059047

The National Association of Writers' Groups

HomeProjectsWebsite ♦ Website Issues

Website Issues

Issue Selection Controls

Choose which issues to display.

Types:

States:

Severities:

Here you can view the known issues with the web site. It's not a full issue management system but it can serve as a reminder for what needs addressing.

Please let us know if you find something wrong with the web site.

(There are also website tasks listed.)

Issue 0037
Headline: Access denied for robots.txt and sitemap.xml.
Type: Fault
State: Fixed
Severity: Major
Found release: 2.9.1
Fixed release: 2.9.2
Description:

Access to the site's robots.txt and sitemap.xml resources is forbidden, which means that automated software may be discouraged from crawling and indexing the site.

This may have serious consequences for SEO.

See the Network Working Group article section 3.1 for more information.

This issue may be due to the security hardening that was implemented in release 2.9.1 on 16-Feb-2018.

To reproduce:
  1. Navigate to /robots.txt.
  2. Navigate to /sitemap.xml (or /sitemap.xml.gz).
  3. Observe the responses.
Behaviour:
  • Expected: A display of the contents of the resource in each case.
  • Actual: An HTTP error response such as "403 Forbidden".
  • Impact: Crawlers and indexers (e.g. from Google search engine) cannot access the resources. According to recommended behaviour, they may assume the site is totally private.
Resolution & notes:

[2.9.2] Fixed by adding a "FilesMatch" rule to the web server configuration file.

Article Actions
Article Information
Author: Kevin Machin Date: June 6, 2014 5:01 pm
Categories: Website Tags: None
Comments: 0 – permitted Article: 3002 – public
Submit a New Comment
Alternatively, you can use your account, if you have one – Log in
Cancel

article as:

Tags
Pages
Archives
Get Creative logo
hissac.logo-t.png

Competition & Mentorships

We are proud to be associated with the Highlands & Islands Short Story Association.

CMP

cmpbookprinting.co.uk

  • Book Printing
  • Self Publishing
  • Anthologies
  • Hard and Soft Back Book Production
  • Free guide and sample book

CMP (uk) Ltd

G3 The Fulcrum, Vantage Way, Poole, Dorset, BH12 4HU

Telephone: 01202 739993

E-mail:

paulford@cmp-uk.com