如何做搜尋引擎蜘蛛日誌分析
搜尋引擎蜘蛛日誌檔案是一種非常強大但未被站長充分利用的檔案，分析它可以獲取有關每個搜尋引擎如何爬取網站內容的相關資訊點，及檢視搜尋引擎蜘蛛在一段時間內的行為。
| IP地址(2) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 138.201.38.179 | 138-201-38-179.crawler.sistrix.net | DE |
| 94.130.88.51 | 94-130-88-51.crawler.sistrix.net | DE |
| IP地址(1) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 135.181.74.195 | crawl1-079.oi.tb.007ac9.net | FI |
| 135.181.74.196 | crawl1-080.oi.tb.007ac9.net | FI |
| 65.108.103.106 | crawl1-007.oi.tb.007ac9.net | ? |
| 135.181.138.9 | crawl1-062.oi.tb.007ac9.net | FI |
| 65.108.79.171 | crawl1-120.oi.tb.007ac9.net | FI |
| 95.217.203.110 | crawl1-018.oi.tb.007ac9.net | FI |
| 135.181.137.62 | crawl1-047.oi.tb.007ac9.net | FI |
| 65.108.99.49 | crawl1-171.oi.tb.007ac9.net | FI |
| 65.21.91.185 | crawl1-248.oi.tb.007ac9.net | FI |
| 65.21.35.229 | crawl1-221.oi.tb.007ac9.net | FI |
| 135.181.138.46 | crawl1-054.oi.tb.007ac9.net | FI |
| 65.21.35.204 | crawl1-213.oi.tb.007ac9.net | FI |
| 135.181.137.100 | crawl1-025.oi.tb.007ac9.net | FI |
| 135.181.140.21 | crawl1-064.oi.tb.007ac9.net | FI |
| 135.181.137.115 | crawl1-028.oi.tb.007ac9.net | FI |
| 65.21.202.117 | crawl1-184.oi.tb.007ac9.net | FI |
| 65.108.103.50 | crawl1-091.oi.tb.007ac9.net | FI |
| 65.21.35.184 | crawl1-196.oi.tb.007ac9.net | FI |
| 65.21.35.198 | crawl1-207.oi.tb.007ac9.net | FI |
| 135.181.4.69 | crawl1-073.oi.tb.007ac9.net | FI |
| 138.201.38.179 | 138-201-38-179.crawler.sistrix.net | DE |
| 94.130.88.51 | 94-130-88-51.crawler.sistrix.net | DE |
| 65.108.103.107 | crawl1-008.oi.tb.007ac9.net | FI |
| 65.108.103.113 | crawl1-014.oi.tb.007ac9.net | FI |
| 65.108.72.9 | crawl1-113.oi.tb.007ac9.net | FI |
| 65.21.35.235 | crawl1-227.oi.tb.007ac9.net | FI |
| 135.181.78.182 | crawl1-086.oi.tb.007ac9.net | FI |
| 135.181.137.60 | crawl1-045.oi.tb.007ac9.net | FI |
| 65.108.99.120 | crawl1-136.oi.tb.007ac9.net | FI |
| 65.108.8.132 | crawl1-121.oi.tb.007ac9.net | FI |
| 65.108.99.177 | crawl1-148.oi.tb.007ac9.net | FI |
| 65.21.206.24 | crawl1-186.oi.tb.007ac9.net | FI |
| 135.181.140.251 | crawl1-037.oi.tb.007ac9.net | FI |
| 135.181.140.38 | crawl1-069.oi.tb.007ac9.net | ? |
| 65.21.35.247 | crawl1-239.oi.tb.007ac9.net | FI |
| 135.181.61.178 | crawl1-074.oi.tb.007ac9.net | FI |
| 65.21.35.185 | crawl1-197.oi.tb.007ac9.net | FI |
| 65.108.99.56 | crawl1-178.oi.tb.007ac9.net | FI |
| 65.108.41.10 | crawl1-103.oi.tb.007ac9.net | FI |
| 65.108.41.9 | crawl1-104.oi.tb.007ac9.net | FI |
| 65.21.35.239 | crawl1-231.oi.tb.007ac9.net | FI |
| 65.21.35.227 | crawl1-219.oi.tb.007ac9.net | FI |
| 135.181.138.47 | crawl1-055.oi.tb.007ac9.net | FI |
| 135.181.137.52 | crawl1-042.oi.tb.007ac9.net | FI |
| 65.21.35.246 | crawl1-238.oi.tb.007ac9.net | FI |
| 65.108.99.124 | crawl1-140.oi.tb.007ac9.net | FI |
| 65.108.99.185 | crawl1-156.oi.tb.007ac9.net | FI |
| 135.181.74.194 | crawl1-078.oi.tb.007ac9.net | FI |
| 65.21.35.230 | crawl1-222.oi.tb.007ac9.net | FI |
| 65.108.103.103 | crawl1-004.oi.tb.007ac9.net | FI |
| 135.181.114.235 | crawl1-021.oi.tb.007ac9.net | FI |
| 65.108.99.123 | crawl1-139.oi.tb.007ac9.net | FI |
| 135.181.140.185 | crawl1-035.oi.tb.007ac9.net | FI |
| 65.21.35.194 | crawl1-203.oi.tb.007ac9.net | FI |
| 135.181.138.4 | crawl1-052.oi.tb.007ac9.net | ? |
| 135.181.137.98 | crawl1-048.oi.tb.007ac9.net | FI |
| 65.108.99.172 | crawl1-143.oi.tb.007ac9.net | FI |
| 135.181.140.19 | crawl1-063.oi.tb.007ac9.net | FI |
| 65.21.35.249 | crawl1-241.oi.tb.007ac9.net | ? |
| 135.181.140.183 | crawl1-034.oi.tb.007ac9.net | ? |
| 65.108.99.50 | crawl1-172.oi.tb.007ac9.net | FI |
| 135.181.137.102 | crawl1-026.oi.tb.007ac9.net | FI |
| 65.108.99.52 | crawl1-174.oi.tb.007ac9.net | FI |
| 65.21.35.195 | crawl1-204.oi.tb.007ac9.net | FI |
| 65.108.103.51 | crawl1-092.oi.tb.007ac9.net | FI |
| 135.181.114.163 | crawl1-020.oi.tb.007ac9.net | FI |
| 65.108.103.105 | crawl1-006.oi.tb.007ac9.net | FI |
| 65.108.103.111 | crawl1-012.oi.tb.007ac9.net | FI |
| 65.108.99.190 | crawl1-161.oi.tb.007ac9.net | FI |
| 135.181.138.5 | crawl1-057.oi.tb.007ac9.net | FI |
| 135.181.137.58 | crawl1-043.oi.tb.007ac9.net | FI |
| 65.108.103.104 | crawl1-005.oi.tb.007ac9.net | ? |
| 65.108.99.119 | crawl1-135.oi.tb.007ac9.net | FI |
| 65.108.99.55 | crawl1-177.oi.tb.007ac9.net | FI |
| 65.108.99.48 | crawl1-170.oi.tb.007ac9.net | ? |
| 65.108.99.109 | crawl1-125.oi.tb.007ac9.net | FI |
| 65.108.103.102 | crawl1-003.oi.tb.007ac9.net | FI |
| 135.181.137.61 | crawl1-046.oi.tb.007ac9.net | FI |
| 65.21.35.202 | crawl1-211.oi.tb.007ac9.net | FI |
| 65.21.35.225 | crawl1-217.oi.tb.007ac9.net | FI |
| 65.108.6.5 | crawl1-108.oi.tb.007ac9.net | FI |
| 65.108.99.54 | crawl1-176.oi.tb.007ac9.net | FI |
| 65.21.35.250 | crawl1-242.oi.tb.007ac9.net | FI |
| 65.21.35.237 | crawl1-229.oi.tb.007ac9.net | FI |
| 65.21.237.134 | crawl1-189.oi.tb.007ac9.net | FI |
| 65.108.103.108 | crawl1-009.oi.tb.007ac9.net | FI |
| 65.108.99.178 | crawl1-149.oi.tb.007ac9.net | FI |
| 135.181.138.2 | crawl1-050.oi.tb.007ac9.net | FI |
| 65.21.35.231 | crawl1-223.oi.tb.007ac9.net | FI |
| 65.108.13.42 | crawl1-101.oi.tb.007ac9.net | FI |
| 135.181.3.203 | crawl1-070.oi.tb.007ac9.net | FI |
| 65.108.103.112 | crawl1-013.oi.tb.007ac9.net | FI |
| 65.108.44.74 | crawl1-105.oi.tb.007ac9.net | FI |
| 65.108.99.42 | crawl1-164.oi.tb.007ac9.net | FI |
| 135.181.136.240 | crawl1-023.oi.tb.007ac9.net | FI |
| 135.181.113.179 | crawl1-019.oi.tb.007ac9.net | FI |
| 135.181.138.45 | crawl1-053.oi.tb.007ac9.net | FI |
| 65.108.99.183 | crawl1-154.oi.tb.007ac9.net | FI |
| 65.21.35.251 | crawl1-243.oi.tb.007ac9.net | FI |
| 65.21.35.197 | crawl1-206.oi.tb.007ac9.net | FI |
| 65.108.103.96 | crawl1-094.oi.tb.007ac9.net | FI |
| 135.181.73.252 | crawl1-077.oi.tb.007ac9.net | FI |
| 135.181.140.22 | crawl1-065.oi.tb.007ac9.net | FI |
| 65.108.103.99 | crawl1-097.oi.tb.007ac9.net | FI |
| 65.108.103.52 | crawl1-093.oi.tb.007ac9.net | FI |
| 65.108.6.6 | crawl1-109.oi.tb.007ac9.net | FI |
| 65.108.74.44 | crawl1-116.oi.tb.007ac9.net | FI |
| 65.21.35.243 | crawl1-235.oi.tb.007ac9.net | FI |
| 65.21.35.189 | crawl1-201.oi.tb.007ac9.net | FI |
| 135.181.62.173 | crawl1-075.oi.tb.007ac9.net | FI |
| 65.21.35.226 | crawl1-218.oi.tb.007ac9.net | FI |
| 135.181.140.252 | crawl1-038.oi.tb.007ac9.net | FI |
| 65.108.99.174 | crawl1-145.oi.tb.007ac9.net | FI |
| 65.108.99.51 | crawl1-173.oi.tb.007ac9.net | FI |
| 135.181.78.184 | crawl1-087.oi.tb.007ac9.net | FI |
| 65.21.35.252 | crawl1-244.oi.tb.007ac9.net | FI |
| 65.21.35.241 | crawl1-233.oi.tb.007ac9.net | FI |
| 95.217.72.254 | crawl1-250.oi.tb.007ac9.net | FI |
| 65.21.35.234 | crawl1-226.oi.tb.007ac9.net | FI |
| 65.21.35.180 | crawl1-192.oi.tb.007ac9.net | FI |
| 135.181.78.178 | crawl1-082.oi.tb.007ac9.net | FI |
| 65.108.99.41 | crawl1-163.oi.tb.007ac9.net | FI |
| 65.108.99.58 | crawl1-180.oi.tb.007ac9.net | FI |
| 135.181.137.2 | crawl1-041.oi.tb.007ac9.net | FI |
| 65.108.99.184 | crawl1-155.oi.tb.007ac9.net | FI |
| 65.108.99.45 | crawl1-167.oi.tb.007ac9.net | FI |
| 65.108.13.104 | crawl1-100.oi.tb.007ac9.net | FI |
| 65.21.35.190 | crawl1-202.oi.tb.007ac9.net | FI |
| 135.181.140.186 | crawl1-036.oi.tb.007ac9.net | FI |
| 65.108.99.181 | crawl1-152.oi.tb.007ac9.net | FI |
| 65.108.99.122 | crawl1-138.oi.tb.007ac9.net | FI |
| 135.181.79.114 | crawl1-088.oi.tb.007ac9.net | FI |
| 135.181.119.46 | crawl1-040.oi.tb.007ac9.net | FI |
| 135.181.140.36 | crawl1-068.oi.tb.007ac9.net | FI |
| IP地址(1) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 138.201.38.179 | 138-201-38-179.crawler.sistrix.net | DE |
一般不需要攔截，尤其是如果你自己也受益於搜尋引擎優化服務。不過，如果你擔心伺服器資源佔用等問題，且您都不使用這些工具，當然也可以選擇攔截它們。
您可以通過在網站的 robots.txt 中設定使用者代理訪問規則來遮蔽 SISTRIX Optimizer 或限制其訪問許可權。我們建議安裝 Spider Analyser 外掛，以檢查它是否真正遵循這些規則。
# robots.txt # 下列程式碼一般情況可以攔截該代理 User-agent: SISTRIX Optimizer Disallow: /
您無需手動執行此操作，可通過我們的 Wordpress 外掛 Spider Analyser 來攔截不必要的蜘蛛或者爬蟲。
