Info Pages Indexing

[heatherbell]

Recently had time to review our Google Search Console which showed that none of our info pages (that is, pages with url /info.php?pages_id=X) are being indexed.

For every one of our info pages, GSC shows either of 2 reasons for not indexing which are:
'Duplicate, submitted URL not selected as canonical'
which, I guess, seems to indicate the info page is a duplicate which it is not.
or
'Crawled - currently not indexed'
which, I guess, seems to indicate google just doesn't 'like it'.

Some of these pages have been published for nearly 2 years.
They have 'good content' in our opinion.
There are at least 2 internal links to these pages in the store.
Burt also recently extended S06E02 for us to include an .xml sitemap for info pages which google has successfully used to discover and crawl the urls but not indexing.

Is this something to do with HT Canonical Links not including info pages?
What are other users' experience?
Is there something we're not doing or doing wrong?
Any suggestions gratefully received.

[ecartz]

Googling the first reason suggests https://www.contentkingapp.com/academy/ ... canonical/

Key takeaway: there is a URL inspection tool that will tell you what page Google thinks it duplicates.

If you need to add a canonical link to info_pages, it might be better to do so in an injectSiteStart hook listener.

I don't have a good answer for the second reason. Although it couldn't hurt to throw it in the URL inspection tool as well.

[heatherbell]

Googling the first reason suggests https://www.contentkingapp.com/academy/ ... canonical/

Key takeaway: there is a URL inspection tool that will tell you what page Google thinks it duplicates.

If you need to add a canonical link to info_pages, it might be better to do so in an injectSiteStart hook listener.

I don't have a good answer for the second reason. Although it couldn't hurt to throw it in the URL inspection tool as well.

Sure, I've already googled everything.
Of course, the info in my post is from URL inspection which does not show what page it thinks is a duplicate, it only gives User-declared canonical - https://mysite.com/info.php
GSC gives no info to explain the second reason but googling suggests it could also be related to canonical.
Is it not a good idea to change HT Canonical Links?

[ecartz]

User-declared canonical - https://mysite.com/info.php

So it turns out that ht_canonical does do info.php ...incorrectly. For now, edit it to add

Code: Select all

case 'info.php':
  if (isset($_GET['pages_id'])) {
    return $GLOBALS['Linker']->build(null, ['pages_id' => (int)$_GET['pages_id']], false);
  }

before the default at line 44.

This might be something for Burt to fix for 1.0.8.17 -- we can discuss that privately. I think that there might be more to it in the general case. It's not something that broke in an update. It has simply never been correct to use canonical and info pages at the same time.

You should be able to see the incorrect canonical URLs in the page source if you look. Then make this change and they should be fixed. Not sure how long it will take for Google to fix them.

[heatherbell]

For now, edit it to add

Code: Select all

case 'info.php':
  if (isset($_GET['pages_id'])) {
    return $GLOBALS['Linker']->build(null, ['pages_id' => (int)$_GET['pages_id']], false);
  }

before the default at line 44.

Yes, that changes

Code: Select all

<link rel="canonical" href="https://mysite.com/info.php" />

to

Code: Select all

<link rel="canonical" href="https://mysite.com/info.php?pages_id=X" />

[burt]

It would have defaulted to "default:" which just builds a link to itself (which would work nicely for [eg] privacy.php or if one made any manual pages, eg if you made a copy of privacy and called it opening_hours.php [or whatever];

default:
return $GLOBALS['Linker']->build(null, [], false);

The correct fix for any page that has many variations (eg info.php) is as provided by Matt just above, ie adding in the extra case for "info.php". Thanks Matt. This can definitely go into the next core.

[heatherbell]

After making the edit suggested in a previous post to ht_canonical.php and using REQUEST INDEXING in URL inspection in GSC for an info page yesterday, google has successfully indexed within 24hrs and green ticks everywhere - yay!
We discovered there is a daily quota for REQUEST INDEXING and only allowed 10 requests before disallowing so that's a daily task for us for the next several days to get all our info pages indexed sooner rather than later.

Note this line in the edit suggested is for >=v.1.0.8.7

Code: Select all

    return $GLOBALS['Linker']->build(null, ['pages_id' => (int)$_GET['pages_id']], false);

For <v.1.0.8.7 use:

Code: Select all

    return tep_href_link('info.php', 'pages_id=' . (int)$_GET['pages_id'], 'SSL', false);

For <v.1.0.7.14 use:

Code: Select all

    $oscTemplate->addBlock('<link rel="canonical" href="' . tep_href_link('info.php', 'pages_id=' . (int)$_GET['pages_id'], 'SSL', false) . '" />' . PHP_EOL, $this->group);

[Omar_one]

Will these changes go into core ?

[ecartz]

Will these changes go into core ?

I don't see any reason it can't go into 1.0.8.17 if we are agreed that the canonical should be the pages_id URL and not the slug URL. I'm guessing that @raiwa's URL rewriter (possibly only the paid version) could be capable of making the correct canonical URL for rewritten URLs. I don't know if that is an existing capability or something that might be added in the future (presuming someone sponsors).

So really it comes down to @burt's intent for the default canonical representation for info pages. I was reasonably sure that this would fix heatherbell's issue, as they have been satisfied with the pages_id URLs. It may be that some other people are using slug URLs instead.

[burt]

intent for the default canonical representation

For interested readers, I know Matt knows this (however have quoted for reference);

The "default: case" is simply in there to catch pages that don't have their own stated case...

This works well for pages such as privacy, shipping and other pages that are standalone [ie they don't have parameters such as ?products_id=x or ?cPath=y etc etc]

However it breaks when things are added either to core [as per the info pages] or as addons where pages have multiple parameters. [eg info.php?pages_id=x] - I can't think of any addon examples off the top of my head