FriendlyCrawler

基本信息

FriendlyCrawler的基本信息如下表。但部分不是很规范的蜘蛛和爬虫，可能存在信息不明的情况。

蜘蛛/爬虫名称: FriendlyCrawler

类型: 爬虫

开发商: Unknown Author

当前状态: 活动

用户代理

关于FriendlyCrawler蜘蛛或者爬虫的用户代理字符串，IP地址和服务器，所在地等信息如下表格所示：

用户代理字符串: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/605.1.15 (KHTML, like Gecko; compatible; FriendlyCrawler/1.0) Chrome/120.0.6099.216 Safari/605.1.15

首次出现: 2024-01-20 14:47:03

最后出现: 2024-10-28 11:54:13

遵循robots.txt: 未知

来源

IP地址(39)	服务器名称	所属国家
35.90.39.23	ec2-35-90-39-23.us-west-2.compute.amazonaws.com	US
54.186.203.122	ec2-54-186-203-122.us-west-2.compute.amazonaws.com	US
35.88.212.83	ec2-35-88-212-83.us-west-2.compute.amazonaws.com	US
54.200.216.189	ec2-54-200-216-189.us-west-2.compute.amazonaws.com	US
35.167.54.118	ec2-35-167-54-118.us-west-2.compute.amazonaws.com	US
35.84.2.8	ec2-35-84-2-8.us-west-2.compute.amazonaws.com	US
35.94.85.211	ec2-35-94-85-211.us-west-2.compute.amazonaws.com	US
35.91.169.96	ec2-35-91-169-96.us-west-2.compute.amazonaws.com	US
52.12.150.192	ec2-52-12-150-192.us-west-2.compute.amazonaws.com	US
35.93.106.29	ec2-35-93-106-29.us-west-2.compute.amazonaws.com	US
35.93.35.65	ec2-35-93-35-65.us-west-2.compute.amazonaws.com	US
18.246.19.76	ec2-18-246-19-76.us-west-2.compute.amazonaws.com	US
35.90.201.65	ec2-35-90-201-65.us-west-2.compute.amazonaws.com	US
54.188.19.214	ec2-54-188-19-214.us-west-2.compute.amazonaws.com	US
54.244.102.88	ec2-54-244-102-88.us-west-2.compute.amazonaws.com	US
54.185.251.210	ec2-54-185-251-210.us-west-2.compute.amazonaws.com	US
35.94.157.182	ec2-35-94-157-182.us-west-2.compute.amazonaws.com	US
35.94.174.9	ec2-35-94-174-9.us-west-2.compute.amazonaws.com	US
44.214.187.82	ec2-44-214-187-82.compute-1.amazonaws.com	US

用户代理字符串: FriendlyCrawler/Nutch-1.20-SNAPSHOT

首次出现: 2024-03-19 16:15:04

最后出现: 2024-04-04 17:47:37

遵循robots.txt: 未知

来源

IP地址(214)	服务器名称	所属国家
34.214.126.131	ec2-34-214-126-131.us-west-2.compute.amazonaws.com	US
54.184.23.66	ec2-54-184-23-66.us-west-2.compute.amazonaws.com	US
54.213.6.202	ec2-54-213-6-202.us-west-2.compute.amazonaws.com	US
52.25.114.96	ec2-52-25-114-96.us-west-2.compute.amazonaws.com	US
34.218.44.125	ec2-34-218-44-125.us-west-2.compute.amazonaws.com	?
34.220.1.117	ec2-34-220-1-117.us-west-2.compute.amazonaws.com	US
52.12.120.40	ec2-52-12-120-40.us-west-2.compute.amazonaws.com	US
35.93.229.202	ec2-35-93-229-202.us-west-2.compute.amazonaws.com	US
35.93.203.68	ec2-35-93-203-68.us-west-2.compute.amazonaws.com	?
54.201.100.0	ec2-54-201-100-0.us-west-2.compute.amazonaws.com	US
34.218.60.34	ec2-34-218-60-34.us-west-2.compute.amazonaws.com	US
34.214.208.239	ec2-34-214-208-239.us-west-2.compute.amazonaws.com	US
34.219.206.29	ec2-34-219-206-29.us-west-2.compute.amazonaws.com	US
35.87.177.110	ec2-35-87-177-110.us-west-2.compute.amazonaws.com	US
54.188.172.61	ec2-54-188-172-61.us-west-2.compute.amazonaws.com	US
54.202.24.255	ec2-54-202-24-255.us-west-2.compute.amazonaws.com	US
34.222.34.206	ec2-34-222-34-206.us-west-2.compute.amazonaws.com	US
54.191.198.55	ec2-54-191-198-55.us-west-2.compute.amazonaws.com	US
54.68.87.1	ec2-54-68-87-1.us-west-2.compute.amazonaws.com	US
54.200.212.90	ec2-54-200-212-90.us-west-2.compute.amazonaws.com	US
35.91.254.10	ec2-35-91-254-10.us-west-2.compute.amazonaws.com	US
34.221.12.242	ec2-34-221-12-242.us-west-2.compute.amazonaws.com	US
52.38.220.116	ec2-52-38-220-116.us-west-2.compute.amazonaws.com	US
35.91.6.231	ec2-35-91-6-231.us-west-2.compute.amazonaws.com	US
54.188.167.66	ec2-54-188-167-66.us-west-2.compute.amazonaws.com	US
35.161.9.6	ec2-35-161-9-6.us-west-2.compute.amazonaws.com	US
35.166.121.148	ec2-35-166-121-148.us-west-2.compute.amazonaws.com	US
54.149.46.48	ec2-54-149-46-48.us-west-2.compute.amazonaws.com	US
54.190.35.236	ec2-54-190-35-236.us-west-2.compute.amazonaws.com	US
34.221.40.13	ec2-34-221-40-13.us-west-2.compute.amazonaws.com	US
35.165.18.197	ec2-35-165-18-197.us-west-2.compute.amazonaws.com	US
52.10.70.62	ec2-52-10-70-62.us-west-2.compute.amazonaws.com	US
35.89.149.209	ec2-35-89-149-209.us-west-2.compute.amazonaws.com	US
35.90.234.218	ec2-35-90-234-218.us-west-2.compute.amazonaws.com	US
35.92.219.127	ec2-35-92-219-127.us-west-2.compute.amazonaws.com	US
35.88.175.126	ec2-35-88-175-126.us-west-2.compute.amazonaws.com	?
54.200.162.8	ec2-54-200-162-8.us-west-2.compute.amazonaws.com	US
34.213.138.16	ec2-34-213-138-16.us-west-2.compute.amazonaws.com	?
35.88.131.39	ec2-35-88-131-39.us-west-2.compute.amazonaws.com	US
54.200.161.125	ec2-54-200-161-125.us-west-2.compute.amazonaws.com	US
54.184.18.118	ec2-54-184-18-118.us-west-2.compute.amazonaws.com	US
34.221.127.26	ec2-34-221-127-26.us-west-2.compute.amazonaws.com	US
35.92.223.242	ec2-35-92-223-242.us-west-2.compute.amazonaws.com	US
35.89.166.202	ec2-35-89-166-202.us-west-2.compute.amazonaws.com	US
35.162.36.87	ec2-35-162-36-87.us-west-2.compute.amazonaws.com	US
35.160.103.228	ec2-35-160-103-228.us-west-2.compute.amazonaws.com	US
35.163.120.39	ec2-35-163-120-39.us-west-2.compute.amazonaws.com	US
35.90.100.35	ec2-35-90-100-35.us-west-2.compute.amazonaws.com	?
35.93.198.110	ec2-35-93-198-110.us-west-2.compute.amazonaws.com	US
54.244.15.51	ec2-54-244-15-51.us-west-2.compute.amazonaws.com	US
54.200.224.62	ec2-54-200-224-62.us-west-2.compute.amazonaws.com	US
54.212.215.118	ec2-54-212-215-118.us-west-2.compute.amazonaws.com	US
52.26.216.186	ec2-52-26-216-186.us-west-2.compute.amazonaws.com	US
35.163.165.227	ec2-35-163-165-227.us-west-2.compute.amazonaws.com	US
35.87.136.7	ec2-35-87-136-7.us-west-2.compute.amazonaws.com	US
54.149.254.232	ec2-54-149-254-232.us-west-2.compute.amazonaws.com	US
34.214.167.36	ec2-34-214-167-36.us-west-2.compute.amazonaws.com	US
34.210.89.167	ec2-34-210-89-167.us-west-2.compute.amazonaws.com	US
52.11.182.254	ec2-52-11-182-254.us-west-2.compute.amazonaws.com	US
34.220.240.53	ec2-34-220-240-53.us-west-2.compute.amazonaws.com	US
35.92.1.230	ec2-35-92-1-230.us-west-2.compute.amazonaws.com	US
54.186.6.64	ec2-54-186-6-64.us-west-2.compute.amazonaws.com	US

访问控制

了解如何控制FriendlyCrawler访问权限，避免FriendlyCrawler抓取行为不当。

是否拦截FriendlyCrawler？

可以考虑拦截。。爬虫通常会下载公开的互联网内容，这些内容默认情况下可以免费访问。不过，如果你不希望你的内容被用于未经授权的目的，你应该拦截它们。

通过Robots.txt拦截

您可以通过在网站的 robots.txt 中设置用户代理访问规则来屏蔽 FriendlyCrawler 或限制其访问权限。我们建议安装 Spider Analyser 插件，以检查它是否真正遵循这些规则。

	# robots.txt
	# 下列代码一般情况可以拦截该代理
	
	User-agent: FriendlyCrawler
	Disallow: /

您无需手动执行此操作，可通过我们的 Wordpress 插件 Spider Analyser 来拦截不必要的蜘蛛或者爬虫。

如何使用WordPress数据包管理应用程序状态

管理任何 WordPress 应用程序的状态（如何处理和组织数据）都是一项挑战。随着项目的发展，跟踪数据流并确保各组件更新的一致性变得越来越困难。WordPress 数据包可以在这方面提供帮助，因为它为状态管理提供了一个强大的解决方案。本文将介绍 WordPress 数据包，探讨其关键概念、实施策略和最佳实践。

FriendlyCrawler

基本信息

用户代理

访问控制

是否拦截FriendlyCrawler？

通过Robots.txt拦截

站长工具

常见蜘蛛/爬虫

相关文章