The NewsBlur Blog

Jan 16, 2012
2011: Year in Review
Twelve months can be a quick flyby if you don’t stop to write everything down. Luckily, a habit I’ve kept since July 2009, when I started recording monthly goals for my project, is still going strong.

This gives me the opportunity to review everything that we’ve accomplished in the past year. There were numerous new features, about half of which were created as a result of feedback from the community. You asked for it, I found the time.

It’s been a pretty good cycle that looks like it’s going to keep going for years.

Premium accounts are keeping NewsBlur alive

At only a $1/month, nearly 800 people have subscribed to NewsBlur. Not much of it goes back to me, since the servers, between the eight of them, cost $650 every month. But that gap is increasing and my prediction is that these servers should be good for more than a doubling of traffic.

Social is coming

In October Google Reader dropped its social features and there was an enormous influx of new users. Unfortunately, NewsBlur doesn’t have any social features that could be considered a replacement for the community lost in the Reader sunsetting.

But that’s changing soon. There’s only a couple more weeks of development needed before the Social branch is ready for beta testers. And I’ll be taking feedback on the changes, iterating until sharebro communities are able to form. It’ll be a great time, watching a favorite service evolve into a social community.

If you’re a developer, you might enjoy watching the development of NewsBlur on GitHub.

2011 Calendar

The biggest features built this year were the River of News, the iPhone app, and the API. There were tons of other big features, but these three top the list for their importance.

January 2011
- River of News - read all stories in a folder
- “Show N hidden stories” button
- Bookmarklet - browser bookmark that allows you to subscribe to a site from the site itself
- Favicon fetching and color decomposing - the color gradient that matches the site’s favicon.
February 2011
- New dashboard
- Replicated PostgreSQL - a slave relational DB for redundancy
- Firefox browser extension - handles RSS feeds by forwarding them to NewsBlur
- Fabric scripts - bootstrapping new servers now takes 5 minutes vs. 3 hours
- Hiding the sidebar - hit the u key
March 2011
- Recommended sites - built entirely to be able to link to the hilarious Ten Sexy Ladies
- New logo - actually, a logo, period.
- Inline classifiers - click on an author or tag in the Feed view
- Create a new folder directly from the bookmarklet
April 2011
- The NewsBlur API
- Hide and show story changes - see the story being modified after being published
- Graphs on the dashboard
- Upgraded to Django 1.3
- Feed fetch histories in Site Statistics
May 2011
- Send story by email - the original social network
- Community feedback on the dashboard
- The tutorial - built because blogger Louis Gray didn’t easily grasp how to get started with NewsBlur, and he’s an early adopter, so good luck to any regular users without the help of a tutorial
June 2011
- Prototype mobile site - using jQuery UI, was a disaster in terms of performance
- Took a 2 week long vacation to the Yucatan
July 2011
- Account dialog
- Major work on the iPhone app
August 2011
- the iPhone app work continues
- Python API
- Major speed-ups in feed loading and fetching
September 2011
- Submit the iPhone app to the App Store - still version 1.0, not polished enough to launch
- New users and new premium users now get emailed a pretty template
- Allowing the moving of the story titles pane - detailed on the blog
October 2011
- iPhone App version 1.1 - adding sites from the app
- Planning out Social - see the blog post
November 2011
- Mark story as unread - #1 feature request since 2010
- Move to folder - moving folders and sites to other folders without having to drag-and-drop.
- Sharing stories - added Pinboard and Google+
- Site settings - change a site’s feed address
December 2011
- Launched the iPhone App - polished enough at version 1.2
- MongoDB is now replicated - for higher availability
- Lots of backend changes to account for increased load - increased speed with more load using better code
You can follow updates in real-time on Twitter by following both @samuelclay and @newsblur. Lots more to come.

Keep the feedback, ideas, praise, and bugs coming. I couldn’t do this without the external motivation coming from dozens of your voices talking about NewsBlur.
Oct 26, 2011
A Social Feed Reader

NewsBlur was released exactly one year ago. You can read the initial reaction on Hacker News: http://news.ycombinator.com/item?id=1834305. Since then, so much has changed and all for the better. Usage is up—way, way up. Premium users are helping the site run. Load times are approaching the goal of less than 100 ms (0.10 sec) per page. In short, things couldn’t be better.

Recently, my main competitor, Google Reader, announced plans to decommission the social features of the site and integrate them into Google+. While Google is busy refocusing on what’s important to them, many users feel left behind. My goal with NewsBlur is to make a better, complete experience for reading sites. This includes social features that make reading a social experience.

Social is a major planned feature. It’s highly prioritized right after I build two other big ticket items: mobile and search. The mobile iPhone app is wrapping up and is already at version 1.1 on the App Store, although I have not publicly launched it because it still needs a few more features (specifically, training and the river of news) to be considered feature-complete.

Once that’s out the door, I have to build search to be able to support social. Search won’t be impossible, since the UI design decisions are fairly straight-foward, and the backend is a no-brainer in terms of design. But it’ll take some time to get right, make fast, and get integrated into the massive database that is quickly accumulating.

After that, I have so many social ideas swimming in my head that I’m ready to drown. I’ve had these social features planned since day one. But I never built them because I needed to make a proof-of-concept first, to be sure that social features would be even feasible.

Also, you can’t discount the immense network effort that building social features would bring. I’m a solo indie developer working on this part-time. Having social features would crush my servers if used as much as I think they would. But know that I am planning some wonderful features to be released as soon as the pieces are in place. Look for the first inkling in the new year.

Let me know on Twitter what your ideas of a good social experience are: @samuelclay.

Sam Clay
Sep 30, 2011
Customizing the reader, step 1: story titles

The iPhone app is now only a few days away from launching. But it took 3 weeks of sitting around in the App Store approval queue before getting here. During that time, I started working on the new customizations that folks have been asking for.

The first of these customizations is the story title pane. Most other readers allow you to position it wherever you like, and NewsBlur is now joining them in giving you that freedom.

You can change the positioning under Manage > Preferences.

Left positioning

Top positioning

Bottom positioning

The old tried and true, this is the default and what you’re already used to.

I hope you like these new positions. There are a few other niceties in this release, namely that resizing window panes is now a live operation, showing you the content stretching in realtime.

There’s a lot more customizations coming out soon. Follow me on GitHub to watch them happen: http://github.com/samuelclay.
Aug 9, 2011
Blar: A new Android app for NewsBlur

This Summer is shaping up to be the season for mobile apps. Blar, a new Android client for NewsBlur, has just been released. It’s available on the Android Market here: https://market.android.com/details?id=bitwrit.Blar. It is created by Harris Munir, who you can contact through his site.

The source code to Blar is also released under a fair license on Bitbucket: https://bitbucket.org/harris/blar/.

In other news, the official NewsBlur iPhone app is nearly finished and is ready to be beta tested. If you are a premium user, email me your UUID and username and I will send you your own free copy of the NewsBlur iPhone app. Just know that it is under heavy development and might change for the better in the near-term future.

Blar was created using the just released NewsBlur API. If you create your own mobile app, bookmarklet, browser extension, or even desktop client, I’ll feature it either on the dashboard of every user or the Goodies section. I’m also happy to assist you in making the app, and even making new API endpoints for you if you wish. Just let me know and keep the NewsBlur apps rolling.
Apr 26, 2011
Make your own feed reader with NewsBlur's new API
Please vote for this blog post on Hacker News: http://news.ycombinator.com/item?id=2485377.

It’s a big news day here at NewsBlur HQ. For those of you who don’t know, NewsBlur HQ is a comfy seat on the A train, part of the NYC subway system, since that’s where most of the code gets written anyhow.

I’m happy to announce NewsBlur’s brand-spanking-new API. It’s free, it comes with tons of juicy data, and it can be used to create your own feed reader. Let’s look at what this all means.

NewsBlur is a visual feed reader with intelligence. This API gives you access to all the moving parts that make up the flagship newsblur.com site. This includes the original site, stories, intelligence classifiers, statistics, and a really neat River of News view that aggregates multiple feeds into a single view.

Juicy Data from the API

Let’s take a quick look at a visualization of the data you can get out of the API. This is a screenshot from the Statistics dialog from a single feed:

To get this data, you would need to make two calls:
```
$ curl http://www.newsblur.com/rss_feeds/search_feed?address=techcrunch

{
    "feed_address": "http://feeds.feedburner.com/TechCrunch",
    "updated": "4 minutes",
    "subs": 2514,
    "feed_link": "http://techcrunch.com",
    "favicon_fetching": false,
    "feed_title": "TechCrunch",
    "favicon": "[BASE64 FAVICON GOODNESS DATA]",
    "favicon_color": "90b490",
    "id": 12,
    "result": "ok"
}

$ curl http://www.newsblur.com/rss_feeds/statistics/12

{
    "premium_subscribers": 66,
    "average_stories_per_month": 490.33333333333331,
    "subscriber_count": 2514,
    "last_load_time": 40,
    "update_interval_minutes": 4,
    "last_update": "5 minutes",
    "feed_fetch_history": [...],
    "result": "ok",
    "stories_last_month": 501,
    "active_subscribers": 43,
    "story_count_history": [["2010-11", 455.0], ["2010-12", 500.0], 
                            ["2011-1", 500.0], ["2011-2", 500.0], 
                            ["2011-3", 487.0], ["2011-4", 500.0]],
    "next_update": "3 minutes",
    "classifier_counts": {
        "feed": [{
            "neg": 5,
            "feed_id": 12,
            "pos": 40
        }],
        "title": [
        {
            "neg": 4,
            "tag": "techcrunch tv",
            "pos": 14
        },
        {
            "neg": 3,
            "tag": "iphone 4",
            "pos": 11
        }
        ...
        ],
        "author": [{
            "neg": 2,
            "pos": 24,
            "author": "MG Siegler"
        },
        {
            "neg": 3,
            "pos": 25,
            "author": "Michael Arrington"
        },
        {
            "neg": 4,
            "pos": 13,
            "author": "Paul Carr"
        }
        ...
        ]
    }
}
```
How about the intelligence classifiers for a site?
```
$ curl http://www.newsblur.com/rss_feeds/search_feed?address=louisgray

{
    "feed_address": "http://blog.louisgray.com/feeds/posts/default?alt=rss",
    "updated": "2 minutes",
    "subs": 130,
    "feed_link": "http://blog.louisgray.com/",
    "favicon_fetching": false,
    "feed_title": "louisgray.com",
    "favicon": "[FAVICON GOES HERE]",
    "favicon_color": "f96402",
    "id": 172,
    "result": "ok"
}

$ curl http://www.newsblur.com/reader/feeds_trainer?feed_id=172

[{
    "stories_last_month": 24,
    "feed_id": 172,
    "feed_tags": [["android", 13.0], ["google", 12.0], ["apple", 12.0], 
                  ["iphone", 9.0], ["twitter", 7.0], ["facebook", 5.0], 
                  ["sxsw", 5.0], ["media", 5.0], ["spotify", 4.0], 
                  ["itunes", 4.0], ["ipad", 4.0], ["silicon valley", 4.0], 
                  ["personal", 3.0], ["apple tv", 3.0], ["quora", 3.0], 
                  ["social networking", 3.0], ["yobongo", 3.0], 
                  ["entertainment", 3.0], ["seesmic", 3.0], ["search", 3.0], 
                  ["music", 3.0], ["beluga", 3.0], ["work", 3.0], 
                  ["chat", 3.0], ["samsung", 3.0]],
    "classifiers": {
        "authors": {},
        "feeds": {},
        "titles": {},
        "tags": {
            "personal": 1,
            "android": -1,
            "facebook": -1,
            "twitter": 1,
            "rss": 1
        }
    },
    "feed_authors": [["louisgray@gmail.com (Louis Gray)", 62]]
}]
```
Without going too deep into explaining various parts of the API, you can dive right into it by viewing the various API endpoints at http://www.newsblur.com/api. The API is meant to be used with an authenticated account, which can be created directly from the API. I spent a lot of time polishing the look and readability of the API, so I hope you like it, since if you’re going to be developing on top of it, you will probably spend a looong time working with the 30 endpoints.

I’d like to briefly note that I work on NewsBlur in my free time, and it’s entirely open-source on GitHub: http://github.com/samuelclay. This API is the result of 22 months of work and thought. But it’s not corporate, not funded, and certainly not battle-tested. Go easy on it for the time being. And offer feedback on Twitter.

Is making a NewsBlur client cannibalism?

Woah! How does that even work? There’s no endpoint for cannibalism. (OK, there is, it’s /reader/delete_feed.) But what does this API mean for you?

Twitter has a web client, but the real iceberg is the ecosystem built on their API. Google Reader has a website, but more than a handful of enormous venture-engorged companies make their living by using the Reader API.

If you choose to use the NewsBlur API, you are building complements to the website. You can mix-and-mash data as you please. But more importantly, you can use NewsBlur data in ways that are not even possible on the website today.

Want to mix up a social layer with the subscriptions in your NewsBlur account? Just use /reader/feeds, /reader/feed/:id, and /rss_feeds/statistics. Got an Android device and want to make a mobile client that you can charge money for and make a profit? The NewsBlur API supports that.

What you can’t do is wrap advertisements around NewsBlur data. Why? Because NewsBlur preserves the original site, so there are already ads on the page if the publisher chooses to have them. Additionally, ads degrade the NewsBlur experience. NewsBlur is not some fly-by-night air-ride-equipped service out to make a buck. NewsBlur is built to make life better, a small net-enabled reading experience at a time. Profit is great, but not at the cost of ads.

If you build it, they will come, and there’s no hard scaling part

You’re building on an API, not a service end-to-end. If you want to handle the scaling part, use the raw NewsBlur code on GitHub: http://github.com/samuelclay. NewsBlur.com will take care of the scaling and infrastructure, and you can take care of building a new layer that might make you famous/well-off/existentially-happy.

The API is ready for you at http://www.newsblur.com/api.

Questions? Feedback? Want to stay up to date when I drive cross country and take photos of the beautiful countryside between New York and California? Follow @samuelclay on Twitter.
- Samuel Clay, innocent Founder of NewsBlur
Apr 23, 2011
Where We Are in April
Hi readers, I want to take a moment to share what I’m working on for the month of April:
- The API: This is the biggest and baddest update since the River of News and premium accounts. The API includes some lightweight rewrites of how NewsBlur communicates to the back-end server. It’s going smoothly, and you can watch the progress on the less-than-aptly named branch dashboard_tweaks on GitHub.
The goal for the API is to have users build Android and iPhone apps on top of NewsBlur. Heck, even web apps which use NewsBlur data sound just fine. Whatever it is, I want to support it.
- Graphs on the Dashboard: I want to add some transparency to how the NewsBlur service is doing, both in terms of user growth and server status. I am adding a few graphs and numbers to the dashboard to show off how many premium and standard users have logged in during the past day, as well as how many feeds have been loaded, the response time for those feeds, and if there is a backlog for feed fetching.
- Bug fixes: If you watch how often I’m committing to NewsBlur on GitHub, you may notice that a handful of bugs are fixed every single day. If you see a bug and it hasn’t been fixed, let me know on Twitter at @samuelclay, or email me at samuel@ofbrooklyn.com.
- Press page: Along with the API there will be a press page to showcase the nearly 40 articles and reviews written about NewsBlur. This page will also have a press kit, since one of the easiest ways to get reviewed is to have a number of materials at the ready.
- Publisher information: If you’re a publisher and you want to make sure the NewsBlur experience is tip-top for your readers, this new page will highlight how to make the Original view work best.
The work is progressing and it won’t be long now until everything above is ready for you to use and enjoy. There are a few other features that may make it out if I find myself working through everything else quickly enough.

I recently launched a feature to show an aggregation of the intelligence classifiers that users are using on their sites. Soon, this data may come back to implicitly recommend stories for you. But in order for there to be enough data to make recommendations useful, more users needs to classify stories they like and dislike.

The chart below shows the growth of different intelligence classifiers. The trend is up for tags, authors, feeds, and titles. That’s very good news.

I’m also happy to say that as NewsBlur continues to grow, and with a few hundred premium users, it is becoming quite a big network of readers.
Apr 1, 2011
Explaining Intelligence

If you’re not using intelligence classifiers, you’re only getting half the value out of NewsBlur.

Intelligence classifiers are the phrases, tags, and authors that you like and dislike. Training your sites by choosing classifiers for each feed will automatically highlight the stories you want to read and hide the stories you don’t want to see.

How to train intelligence

To train your feeds, you have four options:

1) Train everything at once using the intelligence trainer, linked to from the Dashboard:

2) Train a feed individually:

3) Train a story:

4) Choose tags and authors in the Feed view:

What’s happening under the hood

To get a better picture of how stories are being classified into the red, yellow, and green states, we need to take a look at how NewsBlur is using your intelligence classifiers and applying them to stories.

When you select a tag, author, or phrase, NewsBlur looks for an exact match in all other stories in the same feed. It’s a very simple match, and nothing mysterious is happening without you being explicit about what you want to see.

Green always wins. If you have 2 green classifiers and 3 red classifiers on a single story, the story will show up green, since it’s clear there is at least *something* you like about the story.

However, classifying the publisher (i.e. the feed itself) works slightly differently. If you specify that you like or dislike the feed, all stories are automatically classified according to this preference, unless there is a tag, author, or title phrase that is classified, in which case it wins over the feed’s classification.

This offers a neat trick to hide most stories from a feed, even in the yellow intelligence state, except for the few stories that you want to watch. Simply train the feed to dislike the feed itself, but give a thumbs up to the tags/authors/phrases in the stories you want to read. This will result in all stories being either red or green, which keeps the site out of your yellow intelligence setting.

What’s in store for the future

Right now the intelligence classifiers are pretty naive. But the impetus for building NewsBlur was to passively train your feeds, just based on your implicit preferences. There’s a lot of work to be done to make this happen, and you can follow NewsBlur’s progress over on GitHub at http://github.com/samuelclay.
Mar 15, 2011
A New Logo for a New Blog

We’ve come a long way, readers. What started as a fun project to scratch an itch has become a fun project that pays for its ever-increasing self. This week I’m going to show how motivated I am about turning NewsBlur into a serious blog reader. And it starts with a collection of a circles:

What may take you, somebody who has done more than only crop images in an image editor, only a few minutes took me on the order of 1-2 hours a day over 5 days. It’s always like this. Slow as sin until you start to pick up how the editor wants you to work. Then it’s gravy. I could, in fact, completely redo the logo in 10 minutes on a blank canvas. That’s how it goes.

The biggest difference is not the new logo, which sits comfortably underneath your sites when reading, but the new favicon.

This little guy sits there, staring at you the whole time, and reminding you where you are. He had to be created separately, but once you know what the hell a mask is, resizing and showing exaggerated pixels is child’s play.