README
Download the project
install Node if you don't have it, it is sometimes enough to update and open xCode for osx
https://nodejs.org/en/download/
On OSX you will probably be asked to install/update xCode at some point in this process
Open Terminal then
Navigate to your user home directorym create a new folder for your data and the downloader, move into that directory
cd ~
mkdir tumblr-backup
cd tumblr-backup
Make a directory for the MongoDB database files
mkdir data
Initialize the npm project as a blank project, just hit enter through the prompts
npm init
Download the project that saves your data
npm install tumblr-save-express
Install mongodb
Follow MongoDB instructions to install MongoDB Community Edition
You may need to setup your Path variable.
google (setup path for [OS])
Mondo DB stores all of the post information like text and other data which you can browse (perhaps non usefully)
MongoDB Compass
All of the images are stored on disk
Start the DB and Run the Web Server
Run the database
mongod --dbpath ./data/tumblr
Database files are then in the directory /data/tumblr
Open an additional terminal window and go back to the same directory
cd ~/tumblr-backup
Run the web server
node node_modules/tumblr-data-save
Open the url in a web browser
localhost:3000/posts/:blog/:from?/:count?
i.e.
localhost:3000/posts/one-million-bees/1000/2000
Will save your posts 1000 through 3000 from most recent (1000) to the 3000 post back
localhost:3000/posts/one-million-bees/
Default is 0 to 10000, if this isn't enough then this gets you 10000 to 20000... etc.
localhost:3000/posts/one-million-bees/10000
The images will be stored in /store/blog-name/month-year/***
The data that has all of your post data will be stored in /data/tumblr in mongoDB format
--
Troubleshooting
The included keys may or may not have hit max that are included with the project.
The keys go here
api-keys.ts
File should look like this
export var key = 'the-key';
export var secret = 'the-secret';
It is possible you can ignore this step if you aren't having any issue after starting the download.
You may need to use your own tumblr app keys if my keys run out of use bandwidth, here is how to get them.
Register a Tumblr App
Use Default callback URL
http://127.0.0.1:3000/login/callback
Development
Download project and install mongodb as listed above
THIS SECTION IS STILL WIP
Install the global libraries
npm install -g typescript typings
typings install
Some Major Improvement that could use a fork & PR
Always use a original post source for image storing for the main images in original posts. Scripts to cleanup the images on disk.
Refactor JSON to have a, only additional information and then also store original JSON and only additional information in the child posts.
Database was setup by a db novice (me), could use some proper thought and migration scripts.
Fix the callback task cycle code for readability and stability
I'm covered in bees
Random things
If you get an error that says something is already running on port 3000 you can fix it by running these commands. Change [pid]
to the process id found in the first command. (OSX, google if otherwise)
lsof -i tcp:3000
kill -9 [pid]