Amazon

Ask the community for help and support.
Xpajun
VIP Member
VIP Member
Posts: 177
Joined: Thu Mar 04, 2021 1:18 pm
Has thanked: 3 times
Been thanked: 6 times

Amazon

Post by Xpajun »

Hi has anyone else had problems with Amazonbot constantly flooding their "whos on line" and ultimatly crashing their sessions database.
All Amazonbot IPs start with 3.*.*.*
If anyone has had this problem did you manage to stop it (and how)
Current Store is now running 1.0.9.0 - php 8.2.18
Now working on taking a short rest :D - php 8.2.18
User avatar
bonbec
VIP Member
VIP Member
Posts: 39
Joined: Mon Oct 26, 2020 12:23 pm
Has thanked: 5 times
Been thanked: 7 times

Re: Amazon

Post by bonbec »

Amazonbot respects the robots.txt directives user-agent and disallow.
Take a look to this : https://developer.amazon.com/fr/amazonbot
Old MS2.2 PHP7.4 site being converted to CE Phoenix v1.0.9.1 PHP 8.2
ecartz
Lead Developer
Lead Developer
Posts: 2726
Joined: Tue Nov 05, 2019 6:02 pm
Has thanked: 4 times
Been thanked: 185 times

Re: Amazon

Post by ecartz »

It would probably make more sense to add it to includes/spider.txt so that it doesn't create a session rather than ban it from sharing your site.
Xpajun
VIP Member
VIP Member
Posts: 177
Joined: Thu Mar 04, 2021 1:18 pm
Has thanked: 3 times
Been thanked: 6 times

Re: Amazon

Post by Xpajun »

ecartz wrote: Sun May 26, 2024 7:58 pm It would probably make more sense to add it to includes/spider.txt so that it doesn't create a session rather than ban it from sharing your site.
I'll have a look at that Matt but it is just continuous not just one session but multi sessions if you delete the sessions they are back again instantly creating another session and totally ignore any bans/blocking
Current Store is now running 1.0.9.0 - php 8.2.18
Now working on taking a short rest :D - php 8.2.18
Xpajun
VIP Member
VIP Member
Posts: 177
Joined: Thu Mar 04, 2021 1:18 pm
Has thanked: 3 times
Been thanked: 6 times

Re: Amazon

Post by Xpajun »

bonbec wrote: Sun May 26, 2024 6:39 pm Amazonbot respects the robots.txt directives user-agent and disallow.
Take a look to this : https://developer.amazon.com/fr/amazonbot
In that case I'm afraid I have a rogue Amazonbot attacking my store because it is ignoring robots.txt directives user-agent and disallow
Current Store is now running 1.0.9.0 - php 8.2.18
Now working on taking a short rest :D - php 8.2.18
14Steve14
VIP Member
VIP Member
Posts: 639
Joined: Fri Oct 25, 2019 7:01 pm
Has thanked: 9 times
Been thanked: 50 times

Re: Amazon

Post by 14Steve14 »

If you really want to stop them from searching your website and they are not taking notice of the robots.txt file then bad then in you htaccess file.

I do believe that there is firewall in the addons section.
Xpajun
VIP Member
VIP Member
Posts: 177
Joined: Thu Mar 04, 2021 1:18 pm
Has thanked: 3 times
Been thanked: 6 times

Re: Amazon

Post by Xpajun »

14Steve14 wrote: Mon May 27, 2024 10:18 am If you really want to stop them from searching your website and they are not taking notice of the robots.txt file then bad then in you htaccess file.

I do believe that there is firewall in the addons section.
Had a problem todsy Steve Amazonbot has overloaded my store causing too many connections to the database - now I can't get in to delete them :evil:
Current Store is now running 1.0.9.0 - php 8.2.18
Now working on taking a short rest :D - php 8.2.18
azpro
VIP Member
VIP Member
Posts: 20
Joined: Fri Nov 06, 2020 8:25 am

Re: Amazon

Post by azpro »

I have the same problem ... Slowing the site and sometimes "Too many requests".

It is (AFIK) not a legitemate Amazon bot/request but Scripted bots using Amazon Web Service computing to srape the web (for AI applications?). I understood blocking via .htaccess seems hard - better way should be to block it via Firewall of server.

Up till now I did not succeed. So any help is appreciated.

If you Google - you can see this is a common problem:
https://www.google.com/search?q=how+to+ ... zonaws.com

You could raed:
https://stackoverflow.com/questions/775 ... contains-x

Best regards!
Xpajun
VIP Member
VIP Member
Posts: 177
Joined: Thu Mar 04, 2021 1:18 pm
Has thanked: 3 times
Been thanked: 6 times

Re: Amazon

Post by Xpajun »

azpro wrote: Mon May 27, 2024 7:34 pm I have the same problem ... Slowing the site and sometimes "Too many requests".

It is (AFIK) not a legitemate Amazon bot/request but Scripted bots using Amazon Web Service computing to srape the web (for AI applications?). I understood blocking via .htaccess seems hard - better way should be to block it via Firewall of server.

Up till now I did not succeed. So any help is appreciated.

If you Google - you can see this is a common problem:
https://www.google.com/search?q=how+to+ ... zonaws.com

You could raed:
https://stackoverflow.com/questions/775 ... contains-x

Best regards!
Thanks, as I posted to Steve my site is completely locked up at the moment waiting for my host to sort it as soon as it's back online I will try the stack overflow fix... never know it might work - I'll post here if it has

As I understand it the name is Amazonbot rather than compute.amazonaws.com - that's what I get doing a reverse ip check anyway

Also I've just updated to Apache 2.4 recently and, apparently, because my server now uses Apache 2.4, which doesn't seem to support anymore the legacy 2.2 syntax. :shrug:
Current Store is now running 1.0.9.0 - php 8.2.18
Now working on taking a short rest :D - php 8.2.18
MyGamesShop
VIP Member
VIP Member
Posts: 74
Joined: Wed Mar 10, 2021 3:02 am
Has thanked: 2 times

Re: Amazon

Post by MyGamesShop »

Use this site to find the IP range by pressing and reading the [whois] button, and add to your firewall.

https://www.abuseipdb.com

like 3.0.0.0/8 or 3.?.0.0/16 or 3.?.?.0/24 or the ip exactly. I found it hops so a band was better 3.?.?.0/24

You will find its not an amazon bot but a shitty Amazon cloud user, trying to break in/copy site or DOS you, or worse someone training AI...

Amazon Data Services Singapore, Data Center/Web Hosting/Transit
NetRange: 3.0.0.0 - 3.1.255.255
CIDR: 3.0.0.0/15
NetName: AMAZON-SIN
NetHandle: NET-3-0-0-0-2
Parent: AT-88-Z (NET-3-0-0-0-1)
NetType: Reallocated
OriginAS: AS38895
Organization: Amazon Data Services Singapore (ADSS-3)
RegDate: 2018-08-01
Updated: 2018-08-01
Ref: https:



You could try this plugin if you don't have access to firewall rules.

app.php/addons/paid_addon/phoenix_firewall/

Also your sessions database will be GIANT!!! and gets read each time someone goes to the site, so that's probably what is making your site crawl. Dumb bots like this one creates a new session each product it goes to, Truncate it using phpmyadmin if you have access to it. (Do Not DROP it!!) or ask your database admin to truncate it.
Post Reply