README
wikiscores
A Command-line tool that extracts structured data about all of the US Supreme Court Cases since 2000. Developed for my brother to use to harvest some data.
Prerequisites
- Node JS
brew install node
orbrew install nvm
thennvm install 14.15.0
(or whatever is the most recent LTS version)
Installation
Install wikiscores
as a global module as follows:
npm i -g wikiscores
Simple Usage
From a Terminal run
wikiscores
This will go to the Lists_of_United_States_Supreme_Court_cases page in wikipedia and get a list of all of the 'Term Opinions'. And for Each 'Term Opinion' it grabs the 'Cases', and for each Case it grabs some statistical data.
Options
- By default it will write it to a file
output.csv
but you can specify the relative file path via the-o
flag. - You can rate limit it with the
-r
option to specify the max number of requests per second. - You can limit the concurrency with the
-c
option to restrict the number of simultaneous requests. - You can filer for a specific case name with the
-f
option. (replace spaces with_
)
wikiscores -o some/other/output.csv -r 10 -c 2 -f Bush_v._Palm_Beach_County
See
wikiscores --help
for more info.
Development
Fork this repo and use the forked-git-flow
process as outlined in the Contributing notes.
Branches
Branch | Tests | Code Coverage | Audit | Comments |
---|---|---|---|---|
develop |
Latest Staging Release | |||
master |
Latest Production Release |
Functional Requirements
Scrape a list of Term Opinions from a list of US Supreme Court Cases, and from each one grab a list of individual cases. For each case grab data in the form:
{
"term": 2002, // the term year
"title": "Some title",
"defaultSort": "Some default sort title",
"length": 100, // the length of the article in bytes
"articleId": 12345, // the numeric article id
"contentModel": "wikitext",
"watchers": 33, // The number of watchers, or the phrase 'Fewer than 30 watchers'
"edits": 22, //number of edits
"recentEdits": 20, // number of recent edits
"monthCount": 55, // number of reads in the last month
"createdAt": '2010-12-28T19:35:00.000Z', // date created in Zulu Time format"
"creator": "The username of the original editor",
}
Test it
npm test
— runs the unit tests (quick, runs offline)npm test:unit:cov
— runs the unit tests with code coverage
Lint it
npm run lint
Contributing
Please see the contributing notes.