in Tech
clojure

Clojure: A better concurrent Java

Our mission at Tagged is to win social discovery, and in order to accomplish this we need robust server technology. The programming language Clojure reduces the risk in developing concurrent server software to help make this possible.

Clojure is a functional language that was designed to run on a JVM (Java Virtual Machine) and has language support for multi-threaded programming. It integrates seamlessly with Java. It is a dialect of Lisp and, as such, is easily extensible. A relatively new language, Clojure nevertheless has a large, growing, and enthusiastic user community. It is used on servers as a replacement for Java in highly concurrent environments, or as a general-purpose functional programming language.

Clojure is the most important development in the Lisp world since the development of Scheme in the 1980s. It solves several long-standing problems with Lisp in enterprise settings. First, it solves the foreign-function interface problem by running on the JVM, giving it access to the entire universe of Java libraries. Additionally, it solves the Lisp data structure problem by introducing vectors, maps and sets that implement a common sequence (first, rest, cons) interface like classic Lisp lists.  Previous dialects of Lisp have been based on this list data structure and have added vectors and maps as afterthoughts that do not share the same access interface and don’t necessarily nest; Clojure lists, vectors, maps, and sets nest arbitrarily. Lastly, it is functional by design (previous Lisps have been functional by convention) and all the data structures are immutable and persistent. This makes code much more robust because objects, once created, can never be modified.  Although new, modified versions can be cheaply created from old versions, the state of an object can never be modified “out from under you,” making syntax like public, private, protected, volatile, WeakReference, synchronized, getters, and setters superfluous.

When a new version is created from an old object, the new version shares the unchanged part with the old object; it doesn’t duplicate the unchanged data from the old object. Objects that can be shared by different threads are confined to three different reference types (vars, atoms and agents) that can only be accessed through controlled functional interfaces.

Clojure greatly reduces the risk involved in developing multi-threaded software. Concurrent programming in Java is notoriously difficult to get right, and even worse to maintain. Bugs can hide in seemingly correct code only to appear at random times, and working code can be easily broken if a maintainer forgets a volatile modifier or synchronized block. Clojure makes writing robust, bug-free, and concurrent software mind-blowingly simple.

in Tech

Development and Customer Support Tools

With over 330 million users worldwide, it is important for us developers at Tagged to find efficient ways to make the Tagged.com user experience the best it can be. My Customer Experience Tools team is the connector between the engineering efforts behind Tagged.com and the Customer Experience Support team who work with Tagged users directly. In order to adequately serve Tagged’s ever-growing global user base, my Tools team and the Customer Support team must work closely together with a united goal of facilitating customer satisfaction and providing a means for Tagged developers to achieve maxiumum productivity in this matter.

Currently my team utilizes over 120 tools on a day-to-day basis, and while all these tools are valuable, we are working a more centralized approach with a streamlined design.

Our new tool pages will derive from a base AdminPage class, which will provide many fundamental features:

  • Logging for each page view
  • Health check interface (default behavior returns 200 header, subclasses may override)
  • Default JS includes (jQuery, jquery.ui, tagged.js, adminpage.js, etc.)
  • Default CSS includes (admin-tools.css, jquery.ui.css, etc.)
  • Default access to commonly-used JavaScript variables (such as url_static)
  • Rendering of page headers (header.php files will go away) and other advanced UI elements (including an “omni-button” and “favorites”; see wireframes)
  • Permission checking
  • Convenience method hooks (ie. AdminPage::handlePost())

New tools will be created by subclassing the AdminPage class and overriding methods as needed, such as AdminPage::render().  The old *.html files go away, and page flow is driven by rewrite rules and an index page that instantiates the proper class based on tool identification.  Automatic alerts are sent to the page maintainer when a tool is down or for a variety of other reasons i.e. tool performance is falling below some threshold.

In order to classify our tools, we will use a tree-based organizational structure.  Tools can be grouped under sub-menus, and in principle there is no nesting limit (side note: we are thinking about enforcing or recommending a 2-level limit).  We will use the nested set categorical model, which allows one-line queries to retrieve trees and paths.  When viewing a tool, a “breadcrumb” widget will be displayed, and it will allow the user to quickly access related tools.

Tool configuration will be data-driven, meaning that the system controls high-level tool properties at the data level.  Additionally, by subclassing the base AdminPage, we ensure that all tools will inherit the same top-level header with a title, breadcrumbs, and other miscellaneous information e.g. the creation date and tool creator.  This will allow us to create a unified tool presentation.

My Tools team is constantly striving to develop a variety of products and services to serve our broader Tagged engineering teams, Customer Support team, and in the end, our global audience of users. The tools we use are essential to helping us make rapid decisions on any given product or service and get quick feedback to adjust our efforts as needed.

Rajesh Nanda is the Engineering Manager for the C.E. Tools and Payment team at Tagged with a passion for simple user-driven applications and an interest in mobile security.

 

in Tech Talk Videos, Tech Talks
scala

Tech Talk: Scala

Tagged allows people to connect and meet each other on a global scale every single day. As part of this social discovery mission, we must create smart algorithms to process the data from our millions of users. Scala is a technology that helps our team tackle this challenge.

Scala is a functional object-oriented language and the driving technology behind Tagged’s PeopleRanker: people relevance scoring, models and algorithms. It can be a huge benefit to projects that focus on analyzing, transforming and processing collections of data.

Scala helped us build a rapid pipeline for building, training and deploying models. Its functional nature enables our engineers to smoothly transform ideas and algorithms to production code. Scala has turned out to be useful for standard web services as well. As a language, it is very concise and the compiler offers strong type inference, offering a seamless experience when working with collections and functions.

Because Scala runs on a JVM, we can also import our existing Java libraries, external packages like Trove, Guice and Jersey, and also use native IO. Scala “convertors implicits” make it easy to work with Java types as if they were Scala.

There are a few pitfalls to Scala, notably the more complex syntax and slow compilation time. However, my team’s experience has shown that it was easier for our engineers to transition from Java to Scala than go back to Java after working in Scala. The syntax and conciseness of the language are addicting and make previous Java patterns feel clunky and obtrusive.

The best example of how Scala has helped Tagged be more productive can be seen in our MeetMe application. We are now able to train models in the afternoon and have them run in production the next morning, serving recommendations to millions of users from our Scala-based server.

For a more in-depth look at Scala, check out the below video of my recent Tech Talk:

 

Michael Terkowski is a People Algorithms Engineering Manager at Tagged. When not obsessing over Scala, he teaches and performs dance, plays rock guitar and enjoys Battlestar Galactica.

in Tech Talk Videos, Tech Talks

Tech Talk: Matrix Factorization

There are millions of material varieties in the world such as sand, concrete, milk, cotton, etc., but only about a hundred elements make these materials in various combinations. Science helps us discover what things are made up of, and because of the recent exploration into matrix factorization techniques, we are able to apply this methodology with abstract concepts.

When given a set of people and their movie preferences, what is a common basic set of movie concepts that people perceive? We don’t just think of movies as simply action or adventure, but we sometimes come up with more specific categories such as “George Clooney movies.” In truth, the concepts we perceive are more nuanced and different than we actually have words for. Matrix factorization not only helps us find underlying idea sets, it also helps us define a set of component preferences a given person may have.

Matrix factorization is just what the name claims: taking a given matrix and factoring it into several smaller matrices. The term matrix factorization captures a wide variety of algorithms that have the same underlying idea: factoring a large matrix into smaller ones. Some of these are categorized as CF (Collaborative Filtering), SVD (Singular Value Decomposition), and NMF (Non negative matrix factorization).

When you look at the output of matrix factorization, it is easy to see that the results all have a lower dimension than the input as they allow a way to compress information, which is helpful in Data Analysis situations. And if you can represent some data with high dimensionality in lower dimensions, why not use just two dimensions? In this case, we can project the data onto a piece of paper, and this forms the basis of Data Visualization using matrix factorization.

At Tagged we are just scratching the surface of matrix factorization techniques -currently exploring applications for this in our popular Pets game and engaging our content teams to serve up the most exciting and relevant content for our users.

In closing, matrix factorization is a fundamental technique (like searching or sorting) and is much more than just a recommendation algorithm – I believe it can be of use in many situations.

Take a look at the video below for a more in-depth discussion on using Matrix Factorization at Tagged.

Senthil Gandhi is a Senior Scientist on the Analytics team at Tagged. His background is in delivering algorithmic solutions for massive data volume situations, numerical analysis, advanced algorithms and data structures. Senthil has a passion for photography (his galleries are here and here) and Deep Learning Algorithms – you can follow him on his blog too.

 

in Security
trackback-spam

Fighting Spam on Tagged

Spam is a problem that plagues most social websites, and Tagged is no exception. On the Security team, we think about spam all the time. For those who aren’t as familiar with the issue, I’d like to share details about our team’s motivation for building the solutions we have put in place.

Fighting spam is a constant battle between security agents and spammers. It’s an area of software development that is truly a virtuous cycle of advancement since both sides are actively trying to outsmart each other. These challenges have helped me grow professionally by forcing me to become accustomed with some of the leading-edge techniques being used in the industry right now. Machine learning in particular has taken the spotlight in the software industry as a means to present users with more relevant information. In the case of spam on Tagged, our system learns what content looks bad and then blocks or deletes it.

Spam detection and prevention is a perfect opportunity for investment at Tagged since the area directly ties back to providing a great user experience. Nobody should think that the Internet is a scary place where one wrong click can be a costly mistake; our goal is to create a safe environment for our users. We continue to have room for improvement in this area with blocking spam and removing fake profiles. At the core, Tagged should be a site for real people to come and connect with other real people.

The most important takeaway I’ve learned from working against spam has been that solutions do not need to be as daunting as the problem. It’s easy to get caught up in solving edge cases or trying to think too far ahead, and sometimes it doesn’t hurt to just try something and see how well it works. Some of the more successful techniques being used against spam are surprisingly intuitive and simple at their core. For example, rather than using some complicated heuristic for labeling what content should be called “spam,” we can just crowd-source the problem to our users by looking at the results of the spam reports they submit. Although there is a lot of noise in this data, the important cases do bubble up. These results can then be used to feed data into our automated systems.

The Security team is dedicated to protecting our users, and one of our core values at Tagged is “Users are #1.” With over 330 million members, we are always trying to stay ahead of the game so that anyone who visits Tagged can have a great experience.

Andrew Neilson is a Software Engineer on the Security team at Tagged with a passion for making ice cream and playing hockey. You can follow Andrew on Twitter.