At 4:16pm last Wednesday I got a short and to-the-point email from Nilay Patel at The Verge with only a link that started with the host “googlereader.blogspot.com”. The sudden spike in NewsBlur’s visitors immediately confirmed — Google was shutting down Reader.

Late night at the office

I had been preparing for a black swan event like this for the last four years since I began NewsBlur. With the deprecation of their social features a year ago I knew it was only a matter of time before Google stopped supporting Reader entirely. I did not expect it to come this soon.

As the Storify history of the Reader-o-calypse, NewsBlur suffered a number of hurdles with the onslaught of new subscribers.

A few of my challenges and solutions

I was able to handle the 1,500 users who were using the service everyday, but when 50,000 users hit an uncachable and resource intensive backend, unless you’ve done your homework and load tested the living crap out of your entire stack, there’s going to be trouble brewing. Here’s just a few of the immediate challenges I faced over the past four days:

  • My hosting provider, Reliable Hosting Services, was neither reliable, able to host my increasing demands, or a service I could count on. I switched to Digital Ocean and immediately got to writing new Fabric scripts so I could deploy a new app/task server by issuing a single command and having it serve requests automatically within 10 minutes of bootstrapping.
  • It didn’t take long to max out my Amazon Simple Email Service (SES) account’s quota of 10,000 emails a day. So a few hours into the melee I switched to Mailgun, which unfortunately resulted in emailing myself 250,000 error reports. If you tried to email me and couldn’t get through, it’s because 50,000 emails about lost database connections made their way ahead of you in line.
  • Eventually, I was just plain blacklisted on SES for sending too many emails.
  • Fortunately, when the PayPal fraud department called because of an unprecedented spike in payments, I was prepared.
  • HAProxy would serve errors (site is down, maintenance, timeouts, etc) with a 200 OK status code instead of the proper 500 Exception status code because of a ridiculous undocumented requirement to include HTTP Headers at the top of the error template. When your webapp uses status codes to determine errors, you get extremely strange behavior when it loads utter crap into your DOM.
  • The inevitable file descriptor limits on Linux means that for every database connection you make, you use up one of the 1,024 file descriptors that are allocated to your process by default. Changing these limits is not only non-trivial, but they don’t tend to stick. This is responsible for bringing down Mongo, PostgreSQL, and the real-time Node servers, all at different times of the night.
  • The support queue is enormous and I’ve had to spend big chunks of my 16 hour days reassuring paying customers that eventually Stripe will forgive me and my unresponsive servers and will send the payment notification that is responsible for automatically upgrading their accounts to premium.

The sad extent of my St. Patrick’s Day

As a one-man-shop it has been humbling to receive the benefit of the doubt from many who have withheld their judgment despite the admittedly slow loadtimes and downtime NewsBlur experienced. Having the support of the amazing NewsBlur community is more than a guy could ask for. The tweets of encouragement, voting NewsBlur up on replacereader.com (If you haven’t yet, please tweet a vote for “#newsblur to #replacereader”), and the many positive comments and blog posts from people who have tried NewsBlur is great.

It has also been a dream come true to receive accolades from the many who are trying NewsBlur for the first time and loving it. Since the announcement, NewsBlur has welcomed 5,000 new premium subscribers and 60,000 new users (from 50,000 users originally).

NewsBlur users are intelligent, kind, and good looking!

The next three months

Over the next three months I’ll be working on:

For those of you who are still trying to decide where to go now that you’re a Reader refugee let me tell you a few of the unique things NewsBlur has to offer:

  1. Radical transparency. NewsBlur is totally open source and will remain that way.
  2. It still feels like RSS, just with a few more bells and whistles. NewsBlur provides actual list of posts, as opposed to the more curated magazine format of some of the other popular replacements. This clean interface makes it easy to see the stories you want. One innovation however is the four different view options you have. NewsBlur can show you the original site, feed, text or story view.
  3. It has training. NewsBlur hides stories you don’t want to read based on tags, keywords, authors, etc. It also highlights stories you want to read, based on the same criteria. This allows you to find the stories you care about, not just the stories that the hive cares about. And best of all, NewsBlur will show you why stories are either highlighted or hidden by showing the criteria in green or red.
  4. NewsBlur has rebuilt the social community that Google had stripped out of Reader. Users can share stories through their Blurblog and discover new content by following friends’ Blurblogs. The People Have Spoken is the blurblog of popular stories.
  5. Because NewsBlur is entirely open-source, if you don’t want to pay you can host your own server. Instructions are on GitHub, where you can also find the source code for the NewsBlur iPhone + iPad app and Android app.
  6. Most importantly, NewsBlur is not entirely a free app. The immediate benefits of revenue have been very clear over the past few days. Not only are NewsBlur’s interests aligned with its users, but as more users join NewsBlur, it makes more revenue that can be used to directly support the new users. Not convinced that paid is better than free? Read Pinboard’s Maciej Ceglowski’s essay Don’t Be a Free User.

Shiloh during better times. Your premium subscription goes to both server costs and feeding her

With NewsBlur’s native iOS app and Android app, you can read your news and share it with your friends anywhere. And with the coming improvements over the next three months, you bet NewsBlur will be the #1 choice for Google Reader refugees.

Join NewsBlur for $24/year and discover what RSS should have been.

Here at NewsBlur HQ, we love greeting each new day by seeing what everyone posts on their blurblogs, but we understand that not everyone might want to have their reading preferences broadcast to the public (or have the public broadcast its opinions on said preferences). So we’re introducing a special new service for premium account holders that allows you to protect your posts from prying eyes.

image

Just click the little sprocket in the bottom left of your dashboard and select “Profile & Blurblog,” where you’ll be given one of three options:

  • Public (default): The good old-fashioned oversharing you’ve come to know and love.
  • Protected: Everyone can see your stories, but only approved NewsBlur followers can reply or comment to your shares. Continue to drop knowledge for a grateful public, minus the peanut gallery.
  • Private: Only your approved followers can see your shares and comment. The outside world will never know about your love of bunny photos and animated GIFs. If you choose one of the latter two options, you’ll receive an e-mail every time someone requests to follow you, allowing you to carefully curate your inner circle. Want to remove any of your existing followers? Just visit their profile to boot them from blurblog access.

Go forth and privatize! It’ll be our little secret, at least until we discover that one of you is having an affair with your biographer.

There are lots of reasons not to post a cool article you’ve seen to your blurblog. Maybe you already follow too many blogs, and don’t have room to add any more to your feed (in which case, may we humbly recommend a Premium account?) Maybe you don’t want everyone to know just how crazy you’ve gotten about jai-alai or aerotrekking or My Little Pony: Friendship Is Magic. Or maybe you found a cool article on Facebook or Twitter or through an e-mail from a friend, and don’t want to go through the hassle of adding the site’s whole feed to your NewsBlur dashboard just to post one piece.

That’s what the NewsBlur bookmarklet tool is for. Whether it’s a great article from a site that you don’t like enough to follow regularly, or a first glimpse at an intriguing new site, the bookmarklet makes it easy to post anything you find around the Web to your blurblog.

image

Setting up the bookmarklet tool is easy: Click on the little sprocket in the bottom left of your dashboard, and choose “Goodies & Extras” from the menu. You’ll see “Add Site & Share Story Bookmarklet,” and next to that, a button that says “Share on NewsBlur.” Drag the button to the Bookmarks Bar in your browser, and you’re in business.

image

The next time you see an article that strikes your fancy, just click the bookmarklet and the NewsBlur window will open. (If you click it by accident or decide you don’t want to share, just click anywhere outside of the box, and you’ll be back where you started.) The bookmarklet automatically selects all text on a page by default, but if you only want to blog a certain chunk of a story, just highlight it beforehand, and it’ll appear solo in the box.

You can also type within the box to edit the headline and text or delete any extraneous junk that may have wandered in. Add a comment of your own, click “Share This Story,” and you’re on your way. It’s frankly almost too easy, so be sure you’re in the right browser tab when you launch it— Allie once accidentally posted the contents of her Gmail inbox to her blurblog when she wasn’t paying attention. If any of you happened to be on there in those two minutes, she hopes you enjoyed the sneak peek into her darkest secrets, like all the books she has on hold at the library.

Using the bookmarklet is awesome and makes NewsBlur’s content (and by extension, the Popular blog) far more interesting and diverse, so get out there and drag back the best of the Web to share with your fellow users. As Richard Marx would put it: Wherever you go, whatever you do (on the Internet, at least), the NewsBlur bookmarklet will be right here waiting for you.

That’s right, t-shirts, stickers, buttons, and magnets. I’ve got a whole lot of good stuff to send out, so give me some critical info and I’ll get you hooked up with the latest in startup love.

The t-shirt entry form has been taken down as of Thursday, November 29th. Hopefully you’ve ordered your t-shirt by now.

Now that NewsBlur has joined the wonderful world of Android, we’re turning our attention back to the iOS app, with a full-scale feature parity push for the new iPhone 5 and iOS 6. It’s perfect for catching up on your reading when you realize that Apple Maps has sent you to the wrong address. Again.

imageimage

We’ve tried to incorporate as many of your suggestions as possible, and here are some of the new features you’ll find:

  • Collapsible folders: Check out folders one at a time (and save your scrolling fingers some work)
  • Saved stories: Access all your saved stories from the Web or other devices, and save other stories for later
  • Mark stories as unread: For that fresh new-story feeling
  • Send stories to Instapaper for reading later, or e-mail them to your friends/enemies
  • Share stories on Twitter and Facebook, and crush social-media naysayers who think you’re just telling people what you ate for breakfast
  • New app-wide menus: An interface almost as stylish as your new device, and far less vulnerable to breakage and theft
  • Bug fixes, improvements, speed-ups, fine-tuning, and other things that aren’t particularly noticeable (but done because we love you)

Check out the update, and be sure to let us know if anything isn’t working the way it should. And Android folks, don’t feel left out; an update with bug fixes is in the pipeline, so stay tuned (and thanks for the feedback!).

You’ve been bugging us for two years about it, and now it’s finally here: NewsBlur’s expansion to mobile is complete, with our first-ever official Android app ready and waiting for your device. Thanks to the gifts of money and time from Y Combinator and the programming prowess of Papermill creator Ryan Bateman (otherwise known as @secretsquirrel), you can now join your iOS brethren on the couch with your daily dose of RSS goodness. Level of accompanying smugness is up to you.

imageimageimageimage

The Android app has all the features of the iOS app, including:

  • All your sites, folders, and friends’ blurblogs
  • Speedy downloading of sites and general alacrity of use
  • The ability to share and comment on stories on your blurblog
  • Direct imports from Google Reader: download the app without ever having to hit the Web to set up an account
  • Follow new people, read their shares, and generally expand your knowledge, worldview, and personal magnetism

Plus, it has that delicious Android flavor, with half the calories.

Download the official NewsBlur Android app. Tell your friends! Tip your waitstaff. Try the veal. Floss twice a day. Call your mom, she misses you.

For a long time, we’ve maintained The People Have Spoken, the blog of what’s popular on NewsBlur, with a simple algorithm that measured how often something was shared. While that’s a great way to see the stuff our users really like (Randall Munroe would probably win the NewsBlur equivalent of the Oscar), it makes it harder for everyone to find new stuff that they might not have seen or heard of before. So we’ve decided to throw a human into the mix. I’ll let Allie introduce herself in her own words:

image

Hi, I’m Allie Pape. I’m a journalist here in San Francisco, and I met Sam when he e-mailed me about an article I wrote about making friends in the city. I love to read and spend most of my day paging through articles on the Internet, so I was excited to use NewsBlur as a way to manage and share my discoveries.

Sam’s given me the keys to the Popular account, so I’m going to be following as many active NewsBlur users as possible, and re-posting 3-4 articles per day that I think are particularly notable. I’ll also sprinkle in some things that I find outside of NewsBlur, if I think they’re interesting enough.

This is an experiment, and since I’m just one person, you will probably find some biases in the kind of stuff I like to post. I prefer long pieces to short, and I definitely have a few areas that are of major interest to me (food, pop culture, feminism and women’s issues, and socioeconomic trends) and a few that aren’t (sports, tech). I’ll try to read outside of my comfort zone when I can, but I can’t promise that I can be all things to all people. The hope is, though, that I can get more people reading NewsBlur, which means Sam can afford to hire other people with different interests to do additional curation for the community. And even if you think my taste is terrible, it’s probably more interesting than seeing the same three or four blogs’ posts over and over again.

If you want to know more about me or have comments or feedback, you can write to me at allie@newsblur.com.

I hope you’ll subscribe to the Popular blurblog, and that I’m able to introduce you to something new and cool you haven’t seen before.

There’s no wrong way to hold an iPad loaded with the brand new NewsBlur iPad app, provided it’s facing you and turned on.

image

This brand new iPad app is not just an accessory to the web. It’s a full blown reading experience. Here are some of the many features now available in the palm of your hands:

  • All your sites, folders, and friend’s blurblogs
  • Super-fast everything, from near-instant downloading of stories to the mouthfeel of the app itself
  • Commenting on stories and sharing them on your blurblog
  • New users can easily get started by importing directly from Google Reader, so you can tell friends about the app without having them to go the website first

The biggest new feature is all about sharing stories with people. Blurblogs, those shared story feeds, are now part of your feed list. And you can expand your network by following more people directly from the app. Following more people means being exposed to stories from sites you may not have even heard of. And because the NewsBlur iPad app is so fast, you can traverse between stories quickly, settling on the ones that look interesting. And better yet, see contribute to a discussion between friends, all on the app.

But wait, there’s more! In fact, there’s a whole lot more, because this is a universal iOS app, which means there’s an iPhone app to boot. Everything you can do on the iPad app is available on the iPhone app.

image

The iPad app and latest version of the iPhone app were built by my good friend Roy. Now that the Y Combinator summer has ended, we wish him well as he’s moving on to pursue his non-news dreams. So long buddy, and thanks for all the hard work this summer. Roy wrote about his impression of Y Combinator and the past summer.

Go download the NewsBlur iOS app and forget about sitting at your desk.

What a difference a few months make. NewsBlur was a side-project of mine for two years. In March of this year, I committed myself full-time and went from developing NewsBlur almost entirely on the NYC subway to writing code every waking minute of the day.

And now there are three big announcements to make.

1. NewsBlur is now a social news reader

The big news of the day is that you can now share stories on NewsBlur. When you share a story, your comments and the original story are posted to your blurblog. Your blurblog is a simple and customizable website. People can comment and reply directly on your blurblog, and you can follow your friends to read the news stories and blog posts that they care about.

Since you’re good at picking your friends, and your friends are good at picking their friends, you will see friends of friends show up, expanding your network with shared stories that you will enjoy. It’s a new way of sharing the news. And because NewsBlur is already an easy to use news reader, it’s simple to find and share stories that your friends will care about.

Every NewsBlur user has their own blurblog. All you have to do is signup for an account on www.newsblur.com and share interesting stories.

2. Y Combinator

For those of you who work with computer science, you may know that a Y-combinator generalizes recursion, abstracting its implementation, and thereby separating it from the actual work of the function in question.1

I’m pleased as punch to announce an investment in NewsBlur by Y Combinator, the investment firm. Over the past two months, we’ve been humbled by the roster of experienced partners giving us candid advice. It’s their tough love that is the catalyst for the next few months of transitioning NewsBlur from side project to world-class news reader. Expect NewsBlur to become simpler and more refined.

3. Introducing Roy Yang

When Y Combinator accepted me as a solo founder, their first piece of advice was to find a co-founder. Looking at every successful startup, a common pattern emerges. Every great startup has multiple people carrying the load when the company takes off.

There is one person on this planet that I would trust as a co-founder. His name is Roy Yang and we have been friends since we met in New York four years ago. We worked together for nearly two years at Daylife, another news startup. I attended his wedding last year in Mexico, and he was the only person I called when I knew I needed somebody talented, focused, and able to complement me on a project that demands enormous time and effort.

Roy is now responsible for both iOS apps and is instrumental in challenging me when I think I’m right and am clearly not. He’s got the patience of a monk and the determination of a true New Yorker. Follow Roy’s blurblog to keep up with him.

A glimpse into the future of NewsBlur

This summer marks the beginning of NewsBlur as a full-time startup. Look forward to new mobile apps, new designs, and new features. Here’s a quick idea of what we’re working on for the next few weeks:

Until then, follow @newsblur on Twitter and start sharing news and blogs on NewsBlur.

— Samuel Clay, @samuelclay


  1. This Y-combinator description is from this StackOverflow answer. Learn about how a Y-combinator works: http://mvanier.livejournal.com/2897.html 

Today, NewsBlur is going real-time. Blogs using the PubSubHubbub protocol (PuSH), which includes all Blogger, Tumblr, and many Wordpress blogs, will instantaneously show new updates to subscribers on NewsBlur. Making this happen, while not for the faint of heart, was straight-forward enough that I’m sharing the recipe I used to get everything hooked up and running smoothly.

Every user, both premium and standard, will now receive instantaneous updates. I’ve been beta-testing this feature for the past few weeks, and I’ve been quite pleased in knowing that I’m now reading on the bleeding-edge.

If you are a developer, you may be interested in how this was done. There are two components in a real-time feed: detecting updates and then informing users of those updates.

Get blog updates in real-time

If you are building a system that consumes an RSS feed and you want it to push to you, you’ll have to subscribe to a special PubSubHubbub hub url that the RSS feed gives you in the original RSS feed.

Take a look at the <feed> section in the NewsBlur Blog’s RSS feed:

>>> # Python
>>> from utils import feedparser
>>> from pprint import pprint
>>> fp = feedparser.parse('http://blog.newsblur.com/rss')
>>> pprint(fp.feed)
{'generator': u'Tumblr (3.0; @newsblur)',
 'generator_detail': {'name': u'Tumblr (3.0; @newsblur)'},
 'link': u'http://blog.newsblur.com/',
 'links': [{'href': u'http://tumblr.superfeedr.com/',
            'rel': u'hub',
            'type': u'text/html'},
           {'href': u'http://blog.newsblur.com/',
            'rel': u'alternate',
            'type': u'text/html'}],
 'subtitle': u'Visual feed reading with intelligence.',
 'subtitle_detail': {'base': u'http://blog.newsblur.com/rss',
                     'language': None,
                     'type': u'text/html',
                     'value': u'Visual feed reading with intelligence.'},
 'title': u'The NewsBlur Blog',
 'title_detail': {'base': u'http://blog.newsblur.com/rss',
                  'language': None,
                  'type': u'text/plain',
                  'value': u'The NewsBlur Blog'}}

If there’s a rel="hub" node under links, then the RSS feed is advertising its PubSubHubbub abilities. If you make a subscription request to that address, then the feed will push out updates to your callback URL.

The code for sending the subscription requests, along with generating the verification token, can be found on GitHub: the PuSH views for handling updates and the initial callback and the PuSH models used to store subscriptions in the DB. Here’s the main request that your server has to send:

# Python
response = self._send_request(hub, {
    'hub.mode'          : 'subscribe',
    'hub.callback'      : callback,
    'hub.topic'         : topic,
    'hub.verify'        : ['async', 'sync'],
    'hub.verify_token'  : subscription.generate_token('subscribe'),
    'hub.lease_seconds' : lease_seconds,
})

The publisher will then ping your server back to confirm the subscription. Once the publisher is configured to send blog updates to your server, you just have to let users know when there’s a new story, and that’s takes some COMET/push technology with the help of WebSockets.

Serving updates to visitors in real-time

When a publisher pushes a new story to your server, apart from dupe detection and storing it in your database, you need to alert users who are currently on the site.

Redis is your new best friend. One of its primary data structures, apart from hashes, sets, sorted sets, and key-value, is a pubsub type that is perfect for this kind of update. Users subscribe to the updates of all of the feeds to which they subscribe. When these sites have a new story, they publish a simple notification to each of the feed’s subscribers.

Here the feed fetcher is publishing to any listening subscribers.

# Python
def publish_to_subscribers(self, feed):
    try:
        r = redis.Redis(connection_pool=settings.REDIS_POOL)
        listeners_count = r.publish(str(feed.pk), 'story:new')
        if listeners_count:
            logging.debug(" ---> [%-30s] Published to %s subscribers" % (
                          feed.title[:30], listeners_count))
    except redis.ConnectionError:
        logging.debug(" ***> [%-30s] Redis is unavailable for real-time." % (
                      feed.title[:30],))

These subscribers have subscribed via Redis. To know that a user is currently connected and wants to be notified of updates, Socket.io is used to connect the browser to a Node.js server that will subscribe to updates via Redis.

The browser opens up a WebSocket and listens for updates for the feeds that they care about:

// JavaScript
setup_socket_realtime_unread_counts: function() {
    if (!this.socket) {
        var server = window.location.protocol + '//' +
                     window.location.hostname + ':8888';
        this.socket = this.socket || io.connect(server);

        this.socket.on('connect', _.bind(function() {
            var active_feeds = this.send_socket_active_feeds();
            console.log(["Connected to real-time pubsub with " + 
                         active_feeds.length + " feeds."]);

            this.socket.on('feed:update', _.bind(function(feed_id, message) {
                console.log(['Real-time feed update', feed_id, message]);
                this.force_feeds_refresh(false, false, parseInt(feed_id, 10));
            }, this));

            this.flags.feed_refreshing_in_realtime = true;
            this.setup_feed_refresh();
        }, this));

        this.socket.on('disconnect', _.bind(function() {
            console.log(["Lost connection to real-time pubsub. Falling back to polling."]);
            this.setup_feed_refresh();
        }, this));
    }
},

The app server is ready to handle thousands of concurrent subscription requests, being Node.js and asynchronous:

# CoffeeScript
fs     = require 'fs'
io     = require('socket.io').listen 8888
redis  = require 'redis'

REDIS_SERVER = if process.env.NODE_ENV == 'dev' then 'localhost' else 'db01'
client = redis.createClient 6379, REDIS_SERVER

io.sockets.on 'connection', (socket) ->
    console.log " ---> New connection brings total to" +
                " #{io.sockets.clients().length} consumers."
    socket.on 'subscribe:feeds', (feeds, username) ->
        socket.subscribe?.end()
        socket.subscribe = redis.createClient 6379, REDIS_SERVER

        console.log " ---> [#{username}] Subscribing to #{feeds.length} feeds"
        socket.subscribe.subscribe feeds

        socket.subscribe.on 'message', (channel, message) ->
            console.log " ---> [#{username}] Update on #{channel}: #{message}"
            socket.emit 'feed:update', channel

    socket.on 'disconnect', () ->
        socket.subscribe?.end()
        console.log " ---> [] Disconnect, there are now" +
                    " #{io.sockets.clients().length-1} consumers."

That’s all there is to it. There a lot going on, but it’s effectively a small circle composed of subscribers and publishers, using Redis to maintain pubsub connections between the many clients and their many feeds.

Hey NewsBlurians, I’m applying for a grant from the Knight Foundation. In a previous life I worked on DocumentCloud, a successful Knight grantee, building open-source libraries. I’m looking to continue the fine tradition of building for both users and for other developers.

I’m asking for enough to fund a year of development with the help of another engineer. Please vote for the NewsBlur grant application on Knight’s website, reblogged below.

Knight News Challenge Entry:

1. What do you propose to do? [20 words]

To build an intelligent social news reader for web and mobile called NewsBlur.

2. Is anyone doing something like this now and how is your project different? [30 words]

RSS feed readers exist, none have the sharing model, original site view, and intelligence classifiers of NewsBlur. Since Google Reader phased out sharing, there’s a sizable community looking to share and discuss news.

3. Describe the network with which you intend to build or work. [50 words]

NewsBlur will surface stories shared by friends and friends of friends by combining the imported networks of Twitter/Facebook with communities on NewsBlur. These communities make it easy to expand your network by showing popular comments from outside your network. NewsBlur also has intelligence classifiers which allow the user to filter and highlight comments across all networks.

4. Why will it work? [100 words]

Because it’s worked before, just under a different model in Google Reader’s now defunct all-or-nothing community. NewsBlur’s network will be oriented more towards showing relationships and distance between you and the other active commenters on a story. NewsBlur will capitalize on the value of pre-existing networks with an intuitive and clean interface that highlights the distance between users. Surfacing relative connections between people will result in a more active community and increased engagement between like-minded readers. NewsBlur further benefits newspapers, publishers, and individual writers by showing the original site (including ads and design), as well as encouraging reading through NewsBlur’s intelligence filters.

5. Who is working on it? [100 words]

I started working on NewsBlur as a side project in June 2009. Over the past 2.5 years, NewsBlur has become self-sustainable through organic growth (word-of-mouth, blog posts, github activity). Because NewsBlur is open-source, a number of contributors from the NewsBlur community have developed their own pet features which have been integrated back into the website. This also works well for finding typos in documentation and allowing users to submit a simple pull request to get it fixed.

6. What part of the project have you already built? [100 words]

On the back-end: distributed feed fetchers and parsers. On the front-end: the feed reader itself, intelligence training, and an iPhone app. There is an actively used API, on top of which NewsBlur’s users have built a mobile website, an Android app, a Windows Phone app, and a Nokia MeeGo app. What’s not built is the entire social layer. A prototype has already been developed to surface any network relationships on comments and shared stories.

7. How would you sustain the project after the funding expires? [50 words]

NewsBlur is free, but there is also a premium subscription that costs between $1 - $3 per month. Users can choose how much they’d like to pay, but that means that NewsBlur is able to pay for its 8 servers. The gap between costs and revenue (also known as profit) is increasing every day.

Requested amount from Knight News Challenge: $150,000
Expected amount of time required to complete project: 1 year

And what a gorgeous mobile app it is. App developer Róbert Márki just released Web Feeds, the first NewsBlur app for Nokia MeeGo. Take a look at these screenshots:

Feed list

Story list

Story view

Intelligence training

Adding a site

Intelligence control

Just stellar work and it has all the features of the website. Web Feeds is free and available from the Nokia Store.

Exactly four months ago, Jason Kottke found my project, NewsBlur, and tweeted:

In that time, every measure of traffic, paid subscriptions, and community feedback has cleared new heights on my revered munin charts. But I kept my full-time job and only worked on NewsBlur as a side-project, as I have been doing for the past two years.

I’m at a slow but steady trickle of users, enough of whom convert into paying premium subscribers to pay for the increasing number of servers (now costing almost $800/month!). But a vocal support forum and an ever-increasing trickle of payments aren’t what convinced me to dedicate myself full-time to my humble project.

What convinced me is that life’s too short.

The only way to do great work is to love what you do

I’m following the better-half of my inner-Jobs and choosing to follow the path of doing what I love to do.1 It’s going to be expensive on my part. There will be no paycheck, health insurance runs out in a few months, and even on an exceptionally good day, premium subscriptions bring in only about half of my former salary.

Paying customers are keeping things afloat

It’s not that big a risk. Users are choosing to pay and choosing how much to pay. Some users opt to pay a $12/year at a buck a month, while some choose to pay $36/year at $3/month. The servers cost roughly .75x what premium subscriptions bring in. That gap between cost and revenue is growing steadily, as the servers I have are running at 25% capacity.

Maciej Ceglowski, of Pinboard fame, has written extensively about the scourge of free services not taking payments from users. From a recent February 2012 presentation, he writes of the three types of services [pdf]:

Business Model #1: Charge Money

This is a groundbreaking new approach where a site accepts fungible tokens of value in return for a product or service.

Business Model #2: Burn Money

Find a sponsor with deep pockets and run at a loss indefinitely.

Business Model #3: Offer a Free Service and Fail

The most popular business model has been to offer a free service and then shut down (or transition to model #1).

You can look forward to new features, bug fixes, and an unending stream of progress, all because you’re paying me for the work I do. It’s a simple setup that helps me sleep at night. I’m building an honest product. You know what you’re paying for. And NewsBlur will continue to be ad-free, because ads are not part of the product that I want to use myself.

So much time, so little to do. Strike that! Reverse it.

In what I can only consider a bizarrely wonderful coincidence, I see this the morning after I declared freedom:

The wind’s at my back, and I’m feeling pretty good about this.2


  1. The worse-half of my inner-Jobs insists on perfection and never gets anything shipped. The balancing act between better and done is about the only constant in this crazy dream to be a self-sustaining indie dev. 

  2. You should follow NewsBlur’s development on Github. I live on props, so that’d make me feel even better. 

Two big announcements today:

  1. SSL is now available. You can use https://www.newsblur.com if that’s your thing.
  2. Stripe.js integration means you can use a credit card to pay for a premium account.

In order to integrate Stripe.js into NewsBlur, I had to add SSL support, which was not the easiest task. Generating signing requests, signing certifications, and providing additional security is no basic task.

Since you’re reading this, it would be a safe assumption that you have already gone premium. To which, firstly, I thank you. And secondly, if you’re invested in the success of a product, you want to watch it grow. But that means you also will not have a chance to see the new payment form, so here’s the new premium upsell:

And the new payment form:

I’m extraordinarily pleased with how much easier this was relative to setting up PayPal about a year ago for recurring payments. Thanks to Stripe.js for being such a hassle-free payment provider.

Twelve months can be a quick flyby if you don’t stop to write everything down. Luckily, a habit I’ve kept since July 2009, when I started recording monthly goals for my project, is still going strong.

This gives me the opportunity to review everything that we’ve accomplished in the past year. There were numerous new features, about half of which were created as a result of feedback from the community. You asked for it, I found the time.

It’s been a pretty good cycle that looks like it’s going to keep going for years.

Premium accounts are keeping NewsBlur alive

At only a $1/month, nearly 800 people have subscribed to NewsBlur. Not much of it goes back to me, since the servers, between the eight of them, cost $650 every month. But that gap is increasing and my prediction is that these servers should be good for more than a doubling of traffic.

Social is coming

In October Google Reader dropped its social features and there was an enormous influx of new users. Unfortunately, NewsBlur doesn’t have any social features that could be considered a replacement for the community lost in the Reader sunsetting.

But that’s changing soon. There’s only a couple more weeks of development needed before the Social branch is ready for beta testers. And I’ll be taking feedback on the changes, iterating until sharebro communities are able to form. It’ll be a great time, watching a favorite service evolve into a social community.

If you’re a developer, you might enjoy watching the development of NewsBlur on GitHub.

2011 Calendar

The biggest features built this year were the River of News, the iPhone app, and the API. There were tons of other big features, but these three top the list for their importance.

January 2011

  • River of News - read all stories in a folder
  • “Show N hidden stories” button
  • Bookmarklet - browser bookmark that allows you to subscribe to a site from the site itself
  • Favicon fetching and color decomposing - the color gradient that matches the site’s favicon.

February 2011

  • New dashboard
  • Replicated PostgreSQL - a slave relational DB for redundancy
  • Firefox browser extension - handles RSS feeds by forwarding them to NewsBlur
  • Fabric scripts - bootstrapping new servers now takes 5 minutes vs. 3 hours
  • Hiding the sidebar - hit the u key

March 2011

  • Recommended sites - built entirely to be able to link to the hilarious Ten Sexy Ladies
  • New logo - actually, a logo, period.
  • Inline classifiers - click on an author or tag in the Feed view
  • Create a new folder directly from the bookmarklet

April 2011

  • The NewsBlur API
  • Hide and show story changes - see the story being modified after being published
  • Graphs on the dashboard
  • Upgraded to Django 1.3
  • Feed fetch histories in Site Statistics

May 2011

  • Send story by email - the original social network
  • Community feedback on the dashboard
  • The tutorial - built because blogger Louis Gray didn’t easily grasp how to get started with NewsBlur, and he’s an early adopter, so good luck to any regular users without the help of a tutorial

June 2011

  • Prototype mobile site - using jQuery UI, was a disaster in terms of performance
  • Took a 2 week long vacation to the Yucatan

July 2011

  • Account dialog
  • Major work on the iPhone app

August 2011

  • the iPhone app work continues
  • Python API
  • Major speed-ups in feed loading and fetching

September 2011

  • Submit the iPhone app to the App Store - still version 1.0, not polished enough to launch
  • New users and new premium users now get emailed a pretty template
  • Allowing the moving of the story titles pane - detailed on the blog

October 2011

  • iPhone App version 1.1 - adding sites from the app
  • Planning out Social - see the blog post

November 2011

  • Mark story as unread - #1 feature request since 2010
  • Move to folder - moving folders and sites to other folders without having to drag-and-drop.
  • Sharing stories - added Pinboard and Google+
  • Site settings - change a site’s feed address

December 2011

  • Launched the iPhone App - polished enough at version 1.2
  • MongoDB is now replicated - for higher availability
  • Lots of backend changes to account for increased load - increased speed with more load using better code

You can follow updates in real-time on Twitter by following both @samuelclay and @newsblur. Lots more to come.

Keep the feedback, ideas, praise, and bugs coming. I couldn’t do this without the external motivation coming from dozens of your voices talking about NewsBlur.