- Python >= 3.8
- Scrapy >= 2.2.0
Download or clone this repo.
Change into the source directory.
cd covid19datascrapper/
Setup a virtual environment, if required. Example:
python3 -m venv env source env/bin/activate
Install the dependencies using
pip
:pip install -r requirements.txt
Run the spider,
kerala_patients
, as below:scrapy crawl kerala_patients -o kerala_patients.json
You can use any desired filename as output file.
The above command will collect data of all patients from the spreadsheet. You can optionally specify the start and end row numbers, if you want rows in a specific range only. Provide optional arguments
pr_start
andpr_end
for this. Example:scrapy crawl kerala_patients -a pr_start=4443 pr_end=4593 -o kerala_patients_latest.json
To export data to
csv
, just change the filename extension from.json
to.csv
.