Thursday, May 17, 2012

Paper.li Platform Enhancement

Dear Paper.li Publishers and Readers,
A platform upgrade will begin on Friday, May 18.  The Paper.li service will be in read-only mode for a period of 24 hours and during this time there will be no new paper creations or manual editing. After that time, we'll be back to business as usual.
Paper.li has grown to become a real publishing powerhouse: 80 million articles published everyday in our user's papers. Such volume requires both a robust processing pipeline for fetching articles from everywhere on the web and semantically analyzing them in 7 languages as well as an inherently scalable storage engine.

Back in March we implemented the first phase of a two-phase platform upgrade in order to keep up with our growing publisher's needs and scale for the future. For the technically curious and pending a more detailed walk-through of the technical stack that support our presses, we thought we would share a few details about what has been happening in the machine room.
  • We transitioned to a new distributed storage engine called Cassandra and a new distributed message broker called Kafka.
  • We significantly changed the way data is gathered, stored and organised to be able to move to the next level of service to deliver even better content to your editions. 
  • For those of you who are into numbers, we can now process over 10'000 articles per second -- we believe we can see future growth coming with a certain degree of optimism!

New features
Along with a data transfer we'll also be rolling out a couple of nifty new features, namely a completely revised & simplified paper creation mechanism, and the access to full fledged archived editions!
New Paper Creation
  • simplified paper creation
  • improved content search across networks keywords, queries and RSS 
  • auto-populated content for viewing before publishing
Improved archiving
  • complete editions of papers available
  • access to video and images
  • Ability to share older editions via Twitter

These are the just the first of numerous new features and functionality to roll out in the coming months. We look forward to your feedback and wish you good continued good publishing!