We’re happy to announce that we’ve added Activity streams to Kicksend’s web application.  You can now get a quick overview of all of the things you’ve sent and received from all of your friends.  Along with just viewing files, you can see messages that you or your friends have sent. Activity streams will soon make it to all of our other platforms.

To check out your activity stream just click on the Activity link on the sidebar.  This is just one of the many new features that we’re going to be rolling out within the next few weeks.

Last weekend, the Kicksend team took a break from our upcoming launch work to get some fresh air at Muir Woods, one of the most well preserved old-growth coastal redwood forests in the country. 

Coast redwoods are amazing – they’re some of the tallest and oldest trees on the planet and look stunningly graceful. Muir Woods, along with Big Basin Park in the Santa Cruz mountains, is one of the few places in the country that you can walk around near really old coast redwood trees.

Since it’s really close to San Francisco, the woods were pretty packed, but the whole experience was still very, very enjoyable. Brendan took some killer photos, which we’ve attached

By the way, the Kicksend team is comprised of fanatic foodies, so we couldn’t miss the opportunity to eat some awesome grilled cheese sandwiches and tomato soup at the Muir Woods Cafe (recommended!!)

One of our requirements at Kicksend is to get messages proxied out from our Rails app to the XMPP messaging server on-demand during a HTTP request. This is trickier than you would expect. A typical XMPP session consists of a multi-step authentication and then usually a persistent connection per-user, something that doesn’t play well within a typical short HTTP request cycle. While most of Kicksend’s messaging triggers are on the client-side, there are a few instances when we need to communicate server-side from the webapp to the XMPP server (for example, when adding a new connection), and doing an authentication each time produces a noticeable slowdown that degrades user-experience.

The solution to this problem involves setting up a XMPP component that hooks into the XMPP server directly, acting as a trusted proxy. Any messages you forward to this component will get passed on directly to the server. Indeed, many of the bigger XMPP-backed apps you’ve used have some custom components running in the background in addition to the XMPP server. 

XMPP components can be written in any language as long as they conform to the spec. At the same time, they should be able to handle a flood of messages that might occur at any time and also be easy enough to connect to from other sections of your app. While I was doing some poking around on Github, I noticed xmpp.js, an XMPP component library for Node.js – after doing a quick test, we realized this worked really well.

Next up, we need a way to reliably get messages to the new proxy-component. The naive implementation would be to maintain a persistent connection to the proxy-component from each instance of the Rails app – but this is a horrible idea in practice. Honestly, the less things you have connected persistent to your Rails instances, the better. We were already using Redis in our stack, so it made sense to use a Redis list as a queue instead of setting up another direct connection. Now the new architecture is as shown in the image below.

Now, whenever we need to pass a message to the XMPP server from the webapp, we stick it into a special Redis list. The proxy-component is now connected to the XMPP server and also connected to Redis. Using Redis’ BLPOP feature, the proxy-component ‘listens’ to the list, and forwards any new messages to the XMPP server. BLPOP is especially suited for this setup since it blocks the redis connection till a new item shows up in the list, making it quite zippy (otherwise you’d have to poll the list every few seconds, not as great or fast). 

We have this running internally on Kicksend, and it’s proving to be quite reliable. Sure, architecture-wise this adds a new point of failure but we’ve ensured that it’s isolated to a smaller section of our app and constantly monitored. At the same time, we’re also making sure that only messages that can afford to be lost run through this, and our users see direct speedup results in some of the more important UX elements of our app. We’re all about optimizing user-experience, so this setup is a definite win in our eyes.

Kicksend is beautiful, powerful realtime filesharing for your family and friends. Signup here.

PS: XMPP folks who want to use this setup, there are a couple of gotchas. Make sure your XMPP server isn’t checking “from” attributes in the component configuration. Also, older versions of Redis have issues with maintaining BLPOP connections alive. Make sure you’re on the latest.