Skip to content

kbmackenzie/flump

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

90 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A simple CLI tool for mass-downloading full-size images from Fandom wiki pages' image galleries.

Contents

In Fandom galleries, images are usually scaled down and the URL to the full-size image is often not present in the DOM until a user clicks on the image thumbnail (opening the image lightbox). Because of this, most simple image scrapers end up downloading very scaled-down, lower quality versions of images from galleries. The images in galleries are also often lazy loaded, which makes scraping harder.

This tool gets around that. It uses puppeteer and is designed to be reliablenot fast. For big pages (like this one), it can take around 5 minutes to download everything.

Note: Images that are not part of a Fandom gallery (see this definition) are not downloaded.

Installation

This tool can be installed with NPM:

npm install -g flump

Usage

All you have to do is pass the URL to the wiki page as an argument:

flump "page-url-here"

You can also set the destination folder for the images:

flump "page-url-here" --output="folder-name"

Command-Line Options

Usage: flump [options] <url>

Fandom wiki gallery scraper.

Arguments:
  url                  URL of a Fandom wiki page to scrape

Options:
  -V, --version        output the version number
  -o, --output <path>  directory where images should be dumped
  -q, --quiet          silence log messages
  -h, --help           display help for command

API

flump.scrapeImages(url) ⇒ Promise.<Array.<string>>

Scrape a Fandom wiki page for gallery images and get their URLs.

Kind: static method of flump
Returns: Promise.<Array.<string>> - A list of image URLs as strings.

Param Type Description
url string URL to a Fandom wiki page.

flump.downloadImages(url, options) ⇒ Promise.<void>

Scrape a Fandom wiki page for gallery images and download all of them.

Kind: static method of flump

Param Type Description
url string URL to a Fandom wiki page.
options FlumpOptions Additional options. See FlumpOptions.

flump~FlumpOptions : Object

Additional options passed to downloadImages.

Kind: inner typedef of flump
Properties

Name Type Description
quiet boolean Silence log messages.
output string The output folder for images.

Releases

No releases published

Packages

No packages published