@mapbox/wikibush

wikibush -------- Put wikidata dumps into `rbush`. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be access

Usage no npm install needed!

<script type="module">
  import mapboxWikibush from 'https://cdn.skypack.dev/@mapbox/wikibush';
</script>

README

wikibush

Put wikidata dumps into rbush. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be accessible via a single interface.

Usage example

# stream wikidata dump into wikibush storage
mkdir workdir
cat wikidata-20161107-geo.ldjson.bz2 - | bzip2 -d | wikibush-writer --workdir=workdir --tolerance=0.001

# read features around point back out
wikibush-reader --workdir=workdir --lon=-73.9975 --lat=40.7308