50 times faster collaboration and ProseMirror

When creating a piece of software there are usually many ways it can be optimized. One aspect we didn’t even think about was the collaboration code. At the heart of our collaboration system is a JavaScript library called diffDOM: It can compare the difference between two websites. We take a snapshot before anything has changed in the editor, and another once the user has changed something. The information about what has changed (a “diff”) can then be shared with collaborators over the net. We chose to maintain this library not just as part of the general Fidus Writer code, but in a separate repository, where it is registered in two package installation systems for Javascript libraries – NPM and Bower.io. And it turns out that quite a lot of other projects could use it for their purposes. NPM gives statistics and just there we can see it is downloaded around 1000 times per month. So many downloads also mean we receive lots of comments on these libraries over Github. And one such recent comment was made by github user unbug, who complained that it was slow on very large documents. And indeed, it took 27 seconds to find the differences between two documents of the size of 178kb. Even though we are not a public service and the private companies that use the software we provide for free really should be able to pay for enhancements themselves, I thought of some things that could make it faster and after a few days of trying things out, I was able to lower the time spent to 0.3-0.5 seconds for the same diff. Adding the changed version to Fidus Writer also made Fidus Writer run significantly faster — and collaboration is a lot more stable!. Nevertheless, we will likely give up all of our collaboration system soon — and use ProseMirror to handle text handling inside Fidus Writer instead. ProseMirror has set out to create an open source editing system that will work well for collaborative editing and which is able to accommodate very specific and configurable document contents. Fidus Writer is proud to have helped fund ProseMirror.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.