Category : Tech

in Tech
clojure

Clojure: A better concurrent Java

Our mission at Tagged is to win social discovery, and in order to accomplish this we need robust server technology. The programming language Clojure reduces the risk in developing concurrent server software to help make this possible.

Clojure is a functional language that was designed to run on a JVM (Java Virtual Machine) and has language support for multi-threaded programming. It integrates seamlessly with Java. It is a dialect of Lisp and, as such, is easily extensible. A relatively new language, Clojure nevertheless has a large, growing, and enthusiastic user community. It is used on servers as a replacement for Java in highly concurrent environments, or as a general-purpose functional programming language.

Clojure is the most important development in the Lisp world since the development of Scheme in the 1980s. It solves several long-standing problems with Lisp in enterprise settings. First, it solves the foreign-function interface problem by running on the JVM, giving it access to the entire universe of Java libraries. Additionally, it solves the Lisp data structure problem by introducing vectors, maps and sets that implement a common sequence (first, rest, cons) interface like classic Lisp lists.  Previous dialects of Lisp have been based on this list data structure and have added vectors and maps as afterthoughts that do not share the same access interface and don’t necessarily nest; Clojure lists, vectors, maps, and sets nest arbitrarily. Lastly, it is functional by design (previous Lisps have been functional by convention) and all the data structures are immutable and persistent. This makes code much more robust because objects, once created, can never be modified.  Although new, modified versions can be cheaply created from old versions, the state of an object can never be modified “out from under you,” making syntax like public, private, protected, volatile, WeakReference, synchronized, getters, and setters superfluous.

When a new version is created from an old object, the new version shares the unchanged part with the old object; it doesn’t duplicate the unchanged data from the old object. Objects that can be shared by different threads are confined to three different reference types (vars, atoms and agents) that can only be accessed through controlled functional interfaces.

Clojure greatly reduces the risk involved in developing multi-threaded software. Concurrent programming in Java is notoriously difficult to get right, and even worse to maintain. Bugs can hide in seemingly correct code only to appear at random times, and working code can be easily broken if a maintainer forgets a volatile modifier or synchronized block. Clojure makes writing robust, bug-free, and concurrent software mind-blowingly simple.

in Tech

Development and Customer Support Tools

With over 330 million users worldwide, it is important for us developers at Tagged to find efficient ways to make the Tagged.com user experience the best it can be. My Customer Experience Tools team is the connector between the engineering efforts behind Tagged.com and the Customer Experience Support team who work with Tagged users directly. In order to adequately serve Tagged’s ever-growing global user base, my Tools team and the Customer Support team must work closely together with a united goal of facilitating customer satisfaction and providing a means for Tagged developers to achieve maxiumum productivity in this matter.

Currently my team utilizes over 120 tools on a day-to-day basis, and while all these tools are valuable, we are working a more centralized approach with a streamlined design.

Our new tool pages will derive from a base AdminPage class, which will provide many fundamental features:

  • Logging for each page view
  • Health check interface (default behavior returns 200 header, subclasses may override)
  • Default JS includes (jQuery, jquery.ui, tagged.js, adminpage.js, etc.)
  • Default CSS includes (admin-tools.css, jquery.ui.css, etc.)
  • Default access to commonly-used JavaScript variables (such as url_static)
  • Rendering of page headers (header.php files will go away) and other advanced UI elements (including an “omni-button” and “favorites”; see wireframes)
  • Permission checking
  • Convenience method hooks (ie. AdminPage::handlePost())

New tools will be created by subclassing the AdminPage class and overriding methods as needed, such as AdminPage::render().  The old *.html files go away, and page flow is driven by rewrite rules and an index page that instantiates the proper class based on tool identification.  Automatic alerts are sent to the page maintainer when a tool is down or for a variety of other reasons i.e. tool performance is falling below some threshold.

In order to classify our tools, we will use a tree-based organizational structure.  Tools can be grouped under sub-menus, and in principle there is no nesting limit (side note: we are thinking about enforcing or recommending a 2-level limit).  We will use the nested set categorical model, which allows one-line queries to retrieve trees and paths.  When viewing a tool, a “breadcrumb” widget will be displayed, and it will allow the user to quickly access related tools.

Tool configuration will be data-driven, meaning that the system controls high-level tool properties at the data level.  Additionally, by subclassing the base AdminPage, we ensure that all tools will inherit the same top-level header with a title, breadcrumbs, and other miscellaneous information e.g. the creation date and tool creator.  This will allow us to create a unified tool presentation.

My Tools team is constantly striving to develop a variety of products and services to serve our broader Tagged engineering teams, Customer Support team, and in the end, our global audience of users. The tools we use are essential to helping us make rapid decisions on any given product or service and get quick feedback to adjust our efforts as needed.


Rajesh Nanda is the Engineering Manager for the C.E. Tools and Payment team at Tagged with a passion for simple user-driven applications and an interest in mobile security.

 

Tech Talk: Tagged Deployment System

With the goal of making the lives of all engineers here at Tagged a bit easier, Corey and I from the SiteOps team have been working on an automated deployment system. This special tool will allow our developers to deploy new versions of their products without SiteOps’ interaction. Currently it manages only Java applications, but in the future it will be expanded to handle most (if not all) the different types of applications used for Tagged’s website and infrastructure.

This system transfers primary control to the developers for managing application deployments, improving response times, and allows our SiteOps team to focus on more complex requests from the developers. In particular, it allows me to devote more time to other projects to further assist our engineering team. I am excited to watch how this new system will improve the efficiency for making changes to the Tagged website.

Check out the video of my Tech Talk below for a more in-depth look at the new Tagged Deployment System.


Kenneth Lareau is a Senior Systems Engineer at Tagged with a passion for music, mathematics, and Git – a version control system; you can follow him on his website. Corey Hickey is also a Senior Systems Engineer on the Tagged SiteOps team.

 

jeff digsby

The Digsby Source Opens Up to the Public!

Today is a great day for the open source community. At Tagged we leverage a good deal of open source technologies to scale tagged.com and bring the best possible experience to our users. Today we are giving back to the community that has helped us in numerous ways by opening our Digsby source code to everyone. (Digsby, an application for IM, email and social network management, was brought under the Tagged brand last year.)

Over the past few months we have been working to bring Digsby into a more open-source friendly state. Our main goals were to make it easy to set up and develop. Furthermore, we wanted to sunset the Digsby servers as numerous services exist on the Internet that are more widely used and have very extensive APIs to accomplish the same synchronized experience.

We first started by removing the client’s dependency on our servers, but we also wanted to preserve all existing accounts for existing users. To do this properly we included a profile importer in the local account creation workflow to help ease the transitions of local accounts for our current users.

Next we wanted to make sure that we could set up a consistent environment for all developers. We tweaked our build process and made sure that everyone could build Digsby and all of its dependencies. This will allow a more powerful experience by letting anyone tinker with the building blocks of wxWidgets, WebKit and some of the other dependencies for Digsby.

We will be hosting Digsby on GitHub for anyone to access. From this point forward we will be building Digsby together, so your input will directly contribute to the future of Digsby!

Check out photos of the Digsby team and milestones on Flickr.


Jeffrey Rogiers was an original member of the Digsby team. He is now working on the Tagged mobile team as an Engineering Manager. You can follow him on Twitter.

 

in Tech
chris b

Using Hadoop @ Tagged

At Tagged, we’re extremely focused on data science, and this requires large volumes of data to be statistically relevant. We’re always looking to improve the speed of our data availability to keep up with the pace of the rest of Tagged, which is fast-moving and agile.

Hadoop is an open source software framework that supports distributed data-intensive storage and computation. It’s particularly strong in dealing with the large amounts of data generated by larger websites (like Tagged.com).

Hadoop consists of two main layers:

1) HDFS (Hadoop Distributed File System) – this is built to handle large quantities of data
2) Map/Reduce – this shines in doing computation and analysis on large data sets

Hadoop is important to Tagged not only because of its ability to store large amounts of data, but also because it works in concert with and is a foundation for real-time data analysis. Real-time data analysis is one of our big goals on the Analytics Team. Hadoop allows us to give decision makers immediate access to metrics so they can react at the speed of our business. The ability to store large amounts of data allows us to store the amount of history required for our data scientists and to draw meaningful insights about customer behavior. Hadoop is flexible; it’s not just about crunching big data slowly. Hadoop can be used for big data storage and retrieval and can also be leveraged for real time analytics.

I’m excited about this new trend in no relational (NoSQL) database technology and how it can be used to complement more traditional databases. In a personal sense, it gives me a chance to work with cutting edge technology that was specifically developed to deal with big data challenges that we encounter at companies like Tagged.


Chris Bunnell has over 10 years experience of working with big data in a variety of industries (banking, telecom and now social media). You can connect with him on LinkedIn.