python3-robot-detection

Detect web crawlers from HTTP UserAgent (Python 3 version)

Description

This Python module detects if a given HTTP User Agent is a web crawler. It uses the list of registered robots from the robotstxt.org.

This package contains the Python 3 version of the library.

Upload more screenshots

Please help extend the collection of screenshots. Just make a screenshot and upload it here. You don't need to register or anything.

Hint: upload an image here from your clipboard with Ctrl-V

Homepage

Install this software package

If the package is available for the distribution you are currently using on your computer then install the software by clicking on…