Options
All
  • Public
  • Public/Protected
  • All
Menu

Interface ScraperOptions

Hierarchy

  • ScraperOptions

Index

Properties

cache

cache: boolean

Enables HTML payload caching on the disk. Keep in mind that papercut will not clear the cache for you. When enabling this, it's your responsability to deal with cache invalidation.

default

false

concurrency

concurrency: { node: number; page: number; selector: number }

Concurrency settings.

Type declaration

  • node: number

    Amount of concurrent promises for node scraping.

    default

    2

  • page: number

    Amount of concurrent promises for page scraping.

    default

    2

  • selector: number

    Amount of concurrent promises for selector scraping.

    default

    2

log

log: boolean

Enables writing pino logs to the stdout.

default

process.env.DEBUG === "true"

Generated using TypeDoc