Request: Web Clipper Extension + Integration

[lifeinchords]

It would be super useful to easily save web content into Joplin via browser extensions. This would mean one place to store, search and retrieve research and notes. Similar to Evernote's clipper, but just the key parts:

• clips a web page with full formatting: HTML, CSS, images.
• ignores JS, bypassing security concerns
• option to add one or more tags
• works asynchronously, allowing person to continue browsing without having to wait for saving to complete

I believe Evernote parses the HTML, and injects any CSS inline into a single HTML file. Some pages don't render correctly in the Evernote app, but it does a good-enough job for most URLs. The important part is having the text and images saved, which can be indexed for search in Joplin.

Storing as HTML makes it portable/easily exportable, in line with project goals.
Since Electron is already being used, assuming it would be easy to render the HTML?

If there's interest, here's a working proof of concept Chrome extension for a similar OSS project.

[mckellyln]

This would be awesome and what I need to migrate fully to Joplin.

[mikerathbun]

I am surprised that this is the first time this has been requested in the almost one year since this project was added to GitHub. I am always looking at new note applications which supports Markdown and was crafted with Evernote as a model or inspiration. Every project, both open source and commercial, seems to be missing a component like the Web Clipper. This has become indispensable to so many Evernote users' workflows that it has essentially locked them in and price hikes or rudimentary note creation options are tolerated because of this feature. I don't think you would even need something as polished and featureful as Evernotes. I know I would be happy with just bookmarking.

This is one of those kind of differentiating features that would get people to smash that Patreon donation button. Many of us already give around $6 a month to Evernote. Cross-platform, iOS, Markdown, good synch. This project is on the way to checking all the jump ship boxes.

[laurent22]

I haven't looked into this yet, but I might give it a try at some point. The HTML to Markdown feature is already quite robust since it can handle the messy HTML from Evernote. Now the component that's missing is a browser extension to export a page (or a part of a page) and send it to the app. If anyone knows about an open source extension or has any suggestion on how to implement this easily, please post here.

[sunjam]

Tagspaces webclipper could work, as mentioned by the dev here in browser-extension issue #1; it is also an open source tool built on Electron that supports Chrome and latest Firefox. Sounds like @uggrock is currently re-designing the entire Tagspaces app as mentioned in issue #659.

[wokawoka]

A webclipper that could replicate a bit evernote's one would be the only thing that I'm missing before switching to joplin...
Any comment about the possibility to adapt Tagspaces webclipper? It would be just great

[laurent22]

I haven't had a look at TagSpace yet but that could indeed be a good option considering it already supports Firefox and Chrome.

[laurent22]

I had a quick look at this feature and it seems almost do-able. On Joplin side, the mini-server mentioned below can be done easily. The missing part really would be a good existing browser extension that we could use to clip content.

For now, I'm putting the spec below along with the extension requirements. If you know of any good existing extension that could be used, please post here and I'll add it to the list. I'm thinking it doesn't need to be a web clipper extension, just anything that can take a screenshot of a page or save HTML content would work.

Joplin mini-server

On Joplin side, the desktop and CLI app will start a mini-server that can receive a note and a file (eg. a screenshot) and create a new note from this. This API can then be used from any extension to create notes from clipped content.

Browser extension requirements

Required features

• Saving whole page
• Screenshot of selected area

Bonus features

• Saving selected text
• Screenshot of visible area

Existing extensions

Extension	Description
Parallels.io Chrome Extension	✘ Not working (projet discontinued)
OneNote WebClipper	✔ Good UI, has all the needed features. ✘ However most of the work is done by closed source Augmentation API :(
TagSpace Browser Extension	✔ Works well. Can clip whole page, text selection, and screenshot of visible area. ✘ Cannot do screenshot of selected area. GPLA license so most likely cannot be used.

Conclusion

For now it seems the only option is to create a browser extension from scratch.

[enekux]

+1

[grotesque]

+1

[sunjam]

Save Page WE extension for Firefox and Chrome

Distributed under the GNU General Public License version 2.

[sbstn87]

zim-clip?

https://addons.mozilla.org/en-US/firefox/addon/zim-clip/

[laurent22]

Thanks for the suggestions, although I don't think we can use Save Page WE or zim-clip as they have different goals and don't have most of the features we need.

[realgeek]

Anyone looked at this before? https://github.com/turtl/browser-extension

I believe Turtl stores markdown + native file formats, but I've never used the browser extension. Last time I tried to build the server for self-hosting it was a bear.

[mm64]

This enhancement is one of the most important ones for me, too. I propose the following alternative approach:

• Support Joplin as target for sharing content in other apps as proposed in Please give the ability to share from another Android app into Joplin.  #110 (not sure if iOS has a share-feature like Android)
• First milestone: When sharing a page with Joplin the URL of the page will simply be saved as content of a note (like a bookmark). I guess this is enough in many cases, in some cases even better than saving the content.
• Second milestone: Joplin can retrieve the contents of the URL and save them somehow inside a note (maybe useful for this: https://facebook.github.io/react-native/docs/webview.html, https://electronjs.org/docs/api/webview-tag).

The approach does maybe not allow all features one could wish for, but it could be easier than writing browser-extensions which could additionally be done later. Being able to receive shared content from other apps that are not browsers is an additional benefit.

[zuphilip]

OneNote Web Clipper is Open Source (MIT license) and actively maintained: https://github.com/OneNoteDev/WebClipper

However, much easier might be to look into some transformation from webpages to markdown, e.g. the Chrome plugin https://github.com/JohnnyFee/CopyAsMarkdown which uses some library for the actual work.

[laurent22]

@zuphilip, converting HTML to Markdown is not too much an issue as Joplin already has this feature (used for ENEX import). The OneNote Web Clipper is open source but really all its work is done via a closed source API so we can't really use it.

[laurent22]

A web clipper is now available in the latest desktop release. To use it, go to Option > Web Clipper Options and follow the instructions. You can:

• Clip a complete page
• Clip a simplified version of the page (based on Firefox's Reader View feature)
• Clip a screenshot of the page

Communication between the Joplin app and the extension is done via a mini HTTP server, so you might need to allow that port on your firewall (you should be prompted automatically on Windows).

This is still a beta release so expect some glitches here and there. Feedback is welcome!

Known issues:

• It should be possible to specify the default notebook where the created notes go (currently it will go in the last created notebook)
• Sometimes the preview content goes over the Submit button in the extension popup
• It works in Firefox too but it's not released yet
• Only tested on Windows so far so not sure if it works on macOS and Linux yet

[miniconfig]

It works great on Linux, once I found out where it was putting the notes :-) Thank you!

[mckellyln]

Awesome! Thx so much, can't wait to try it out.
Where does it put the notes ?

[croulibri]

Youyouyouuu ! This is a great news.
The web clipper works on Debian with Chromium.
Congratulation, this is fantastic.

Some improvements possible :

• web clipper extension for Firefox
• when the note is created, it would be interesting to integrate the address of the webpage (at the end, the beginning or in the title)
• choose the notebook where to save the new note

But once again this web clipper does already very well the job.

[laurent22]

when the note is created, it would be interesting to integrate the address of the webpage (at the end, the beginning or in the title)

The URL is already saved with the note, although at the moment that URL is only visible from the CLI or mobile app.

choose the notebook where to save the new note

Coming next

[heluca]

I am unable to select a notebook with the clipper plugin. Running Windows desktop 1.0.96, Firefox 64bit 60.0.1, plugin reports: Service status: searching. Joplin app is running. Tried disabling Windows Firewall, but no change.

[laurent22]

@heluca, is the clipper service running? You need to enable it from the Web Clipper Options. If it is enabled, could you post a screenshot of that option screen to see what port it is running and other info?

[laurent22]

The clipper is available now and seems stable enough. If there's any problem, please open a new issue.