www/py-scrapy - The NetBSD Packages Collection

High-level Web Crawling and Web Scraping framework

Scrapy is a fast high-level web crawling and web scraping framework, used to
crawl websites and extract structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring and automated testing.

Build dependencies

pkgtools/cwrappers

Runtime dependencies

security/py-OpenSSL devel/py-ZopeInterface security/py-cryptography textproc/py-cssselect textproc/py-lxml www/py-parsel www/py-protego devel/py-pydispatcher devel/py-queuelib security/py-service_identity lang/py-six net/py-twisted www/py-w3lib devel/py-setuptools lang/python38 lang/python38

Available binary packages

aarch64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
aarch64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
alpha:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv6hf:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py39-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
earmv7hf:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py39-scrapy-1.8.0.tgz(NetBSD 8.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
i386:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
mips64eb:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
mips64eb:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
mips64eb:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
mips64eb:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py39-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
powerpc:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
powerpc:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
sparc64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 8.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py27-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py36-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py37-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py38-scrapy-1.8.0.tgz(NetBSD 9.0)
x86_64:py39-scrapy-1.8.0.tgz(NetBSD 9.0)

Binary packages can be installed with pkgin or pkg_add(1). The NetBSD packages collection is also designed to permit easy installation from source.

Available build options

(none)

Known vulnerabilities

The pkg_admin audit command locates any installed package which has been mentioned in security advisories as having vulnerabilities.

Please note the vulnerabilities database might not be fully accurate, and not every bug is exploitable with every configuration.


Problem reports, updates or suggestions for this package should be reported with send-pr.


[ Go up one level | Go to top of packages tree | List all packages ]