Category : Open Source

in Open Source, Tech
jeff digsby

The Digsby Source Opens Up to the Public!

Today is a great day for the open source community. At Tagged we leverage a good deal of open source technologies to scale tagged.com and bring the best possible experience to our users. Today we are giving back to the community that has helped us in numerous ways by opening our Digsby source code to everyone. (Digsby, an application for IM, email and social network management, was brought under the Tagged brand last year.)

Over the past few months we have been working to bring Digsby into a more open-source friendly state. Our main goals were to make it easy to set up and develop. Furthermore, we wanted to sunset the Digsby servers as numerous services exist on the Internet that are more widely used and have very extensive APIs to accomplish the same synchronized experience.

We first started by removing the client’s dependency on our servers, but we also wanted to preserve all existing accounts for existing users. To do this properly we included a profile importer in the local account creation workflow to help ease the transitions of local accounts for our current users.

Next we wanted to make sure that we could set up a consistent environment for all developers. We tweaked our build process and made sure that everyone could build Digsby and all of its dependencies. This will allow a more powerful experience by letting anyone tinker with the building blocks of wxWidgets, WebKit and some of the other dependencies for Digsby.

We will be hosting Digsby on GitHub for anyone to access. From this point forward we will be building Digsby together, so your input will directly contribute to the future of Digsby!

Check out photos of the Digsby team and milestones on Flickr.

Jeffrey Rogiers was an original member of the Digsby team. He is now working on the Tagged mobile team as an Engineering Manager. You can follow him on Twitter.

 

in Big Data, Open Source, Stig

Five Open Source Technologies for 2012

Over at PC World, Joab Jackson, wrote an article on five open-source technologies to watch for in 2012 and Stig, our upcoming open-source graph database, was included! We’re featured along with Nginx, OpenStack, Linux Mint, and Gluster.

Stig is still a bit of a mystery, as it hasn’t been actually released yet. But observers are predicting it could fit a niche in the social networks and other applications that keep a wide range of data. The needs of social networking services are inherently different from other types of jobs, and would benefit from a database attuned to its needs, Lucas explained. “You can’t be a relevant service in this space without being able to scale to a planetary size,” he said.

Check out the entire article here and sign up for our newsletter over at stigdb.org for more information on when we launch!

in Open Source, Python, Tools

Python Interface to the HipChat JSON API

One of the many modern challenges to scaling a business is keeping employee accounts consistent. Traditionally, businesses would use a LDAP or Active Directory server and make all their tools plug in to those services. When someone was hired, they would receive a single user account that worked everywhere. At Tagged, we wrote a tool called SUMP: Simple User Management Portal. SUMP hooks into all the systems that we use, such as Google Apps for Businesses, our LDAP server and Hipchat, which we use for group chat.

There were already a few libraries available to access HipChat’s API but none had the user creation features we needed, and we preferred Python. So, we decided to roll our own library and release it as Open Source. Since many organizations have their own tool like SUMP, often written in Python, we figured this has a high value for other groups. Writing these libraries can be so simple that it’s easy for many people to create their own implementation. By releasing it as Open Source, we benefit because it pushes others to use the same HipChat API. This causes HipChat to maintain it for all their customers, and not just for us, so we don’t need to worry that SUMP might stop working one day. We also get the added benefit of possible outside contributors.

When we wrote this code, we decided to make it as Pythonic as possible. The HipChat API returned JSON objects, which are more like dictionary data types, but it was clear there were distinct data objects. Every API call gets wrapped into a typed class. Rather than building up JSON objects, function parameters are converted automatically so the library has a very object-oriented feel. Finally, there are some sample command line tools in the ‘commands.py’ file to demonstrate some common use cases for the package.

While the library is simple, it’s designed with a couple abstractions to make it extensible. Behind the scenes is a base class HipChatObject and a partially applied call_hipchat function for accessing any arbitrary API function. This makes it possible for other developers to add features to the library very rapidly when HipChat introduces new functionality.

The code is available on both GitHub and Gitorious and there have already been some pull requests over Gitorious. It’s just a small contribution, but along with other projects like the Node.js Memcached Library – we’re paving the way for Tagged to do even more Open Source contributions in the future.

Yaakov Nemoy is a Systems Administrator at Tagged and you can follow him on Twitter and on Google+.

in Open Source

Compression in the Node.js Memcached Library

Here at Tagged we use memcached extensively with our PHP layer, primarily to store compressed, serialized PHP Objects. I’ve been working on a project that utilizes Node.js to interact with these objects in memcached and I’ve run into a few issues since the PHP memcached extension compresses objects above a certain size and the Node.js memcached library doesn’t support compression. Fixing this issue was not as simple as just compressing the data before putting it in memcached or decompressing the data that comes back from memcached.

The issue is how Node.js handled strings with bytes above the regular ascii range. The Node.js memcached library used strings to read data coming back from memcached, however when a byte was encountered that was not in the regular ascii range the byte would be converted into an UTF8 error byte sequence. In addition, the memcached flags indicating the data was compressed had to be properly set.

Tip: To avoid problems when dealing with buffers with non-ascii bytes, keep it as a buffer as long as possible. If necessary, the buffer can be converted to a string using the ‘binary’ protocol. However, if something is appended to this string, the non-ascii bytes will get converted to UTF8 error byte sequences. Hence, if something is added to this string, it should be converted back to a buffer and the necessary bytes should be appended.

With this in mind, I helped create this Node.js memcached library to solve these issues. It is useful to anyone who wants to use compression with memcached in node, as there are currently no other memcached libraries for Node.js that support compression. This is also useful for people who want to be able to have Node.js applications that are interoperable with the same memcached stores as their PHP applications.

Let me know what you think and please post any questions you may have! This is just one of the many issues with memcached and Node.js we’ve encountered and solved along the way. As we continue to use these technologies more extensively, we will be able to share more project stories, tips and tricks on how to use them efficiently. Stay tuned.

Nalin DeZoysa is a Software Engineer at Tagged and you can follow this project on Github.

in Announcements, Open Source

Tagged on Github!

We’re excited to announce that Tagged is now on Github, the web-based distributed version control system.

You can follow some of the projects our engineers are contributing to such as a Node-Kafka, a node client for Kafka, LinkedIn’s disk based message queue, JHM, an Intelligent build system we’re developing here at Tagged and many more projects to come in the future. We’ll be writing about how we’re using open-source projects here on our blog.

We’re excited to work with and give back to the open-source community. Follow all the projects we’re contributing to here – https://github.com/tagged