# rePocketable [](https://github.com/gomods/athens) [](https://svgshare.com/i/Zhy.svg) [](https://svgshare.com/i/ZjP.svg) [](https://svgshare.com/i/ZhY.svg) [](https://github.com/owulveryck/rePocketable/actions/workflows/go.yml) This tool and its webpage are under construction. Best possible option if you want to see what it will eventually do is to run a cli tool such as to epub: \`\`\`shell go run cmd/toEpub/*.go https://whateverpageyouwanttoread/ \`\`\` This utility takes optional \`-H\` arguments to pass headers to the http downloader. This option can be used several times to be compatible with the curl command. ex: \`\`\`shell toEpub -H 'sec-ch-ua: "Chromium";v="94", "Google Chrome";v="94", ";Not A Brand";v="99"' \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36' \ -H 'sec-ch-ua-platform: "macOS"' https://thewebsite/thepage.html \`\`\` I wrote some explanation of the concept in a [blog post](https://blog.owulveryck.info/2021/10/07/reading-from-the-web-offline-and-distraction-free.html) ## Hacktoberfest This is a toy project, but I am more and more relying on it. I think that hacktoberfest is a good opportunity to turn this project into a product. I will write a contributing guide soon; meanwhile if you want to participate the urgent matters are: - Writing a proper vision: discussing it into an issue and submitting a PR to mention in in the README - Writing autonomous end-to-end tests: grabbing a sample page, running an \`httptest\` server, running a toEpub code and analysing the result - Writing a proper documentation - Adding a contribution guide - sky is the limit, discuss in issues and submit PR once issues are discussed :D ## Features The internal libraries (used by the CLI) are implemeting those features: - Webpage fetching and pre-processing - preprocessing and sanitization of figures to fetch the correct image from responsive and/or javascript tags (Medium and Toward datascience) - experimental feature to turn LaTeX figures into pictures ([github.com/go-latex/latex](https://github.com/go-latex/latex)) - extraction of the content based on the ARC90 readility project ([github.com/cixtor/readability](https://github.com/cixtor/readability)) - Opengraph processing to extract meta informations ([github.com/dyatlov/go-opengraph](https://github.com/dyatlov/go-opengraph)) - Generation of a cover picture with the front image of the website, the title and the author of the artible - Generation of a first chapter with meta data such as the publication date - epub generation ([github.com/bmaupin/go-epub](https://github.com/bmaupin/go-epub)) - experimental getpocket integration - reading the article lists and generating epubs from the list - a daemon mode that will eventually runs on a ereader device to sync the list (heavy WIP) ## Configurations Those configuration may influence various internal libraries. | KEY | TYPE | DEFAULT | REQUIRED | DESCRIPTION | |---------------------------------|-------------|------------|-------------|----------------| | DOWNLOADER_LIVENESS_CHECK | Duration | 5m | true | | | DOWNLOADER_PROBE_TIMEOUT | Duration | 60m | true | | | DOWNLOADER_HTTP_TIMEOUT | Duration | 10s | true | | | DOWNLOADER_TRANSPORT_TIMEOUT | Duration | 5s | true | | Those configuration are used for cli using the pocket integration | KEY | TYPE | DEFAULT | REQUIRED | DESCRIPTION | |----------------------------|-------------|---------------------------------|-------------|--------------------------------------------------------------------| | POCKET_UPDATE_FREQUENCY | Duration | 1h | true | How often to query getPocket | | POCKET_HEALTH_CHECK | Duration | 30s | true | | | POCKET_POCKET_URL | String | https://getpocket.com/v3/get | true | | | POCKET_CONSUMER_KEY | String | | true | See https://getpocket.com/developer/apps/ to get a consumer key | | POCKET_USERNAME | String | | | The pocket username (will try to fetch it if not found) | | POCKET_TOKEN | String | | | The access token, will try to fetch it if not found or invalid |