A recent clipping in The Times sparked my interest. It talked of a software company in the US working with major brands to glean marketing information from photographs posted on social media. The results could, it said, be used to tailor advertising more accurately.
Reliability of photos
So far, so quant, but what, I wondered, was the qualitative aspect, if any? A photo may be worth more than a thousand words, but it can also lie, or at least obscure emotions. So I decided to learn more about Ditto Labs, the company in question. It mines photos posted on social media sites such as Twitter, Instagram, and Tumblr.
As photo sharing sites proliferate, other ways of harvesting consumer intelligence from social media are beginning to be offered in place of text mining. Ditto Labs specialises in scanning these photos for logos and distinctive brand-specific patterns to help clients refine their marketing. The data can be overlaid on others provided such as trends, client and competitor performance, plus demographic information.
“We’re always in touch with what’s happening in real time,” says Mary Tarcynski, its CMO. “Consumers, especially in the US and some other key markets, have a habit of documenting their behaviour online.”
Social media sites are a large source of data for researchers. They are also evolving as a communication medium, with people increasingly posting photos as a new form of conversation. The work done here is largely quantitative, but some companies may be tempted to replace qualitative research with the data it can provide.
The appeal is obvious: “You can just immediately go and tap into it so you don’t have to set up a focus group or online survey... they’ve already been there and have probably documented their interaction on anything,” says Mary.
These sorts of insights are provided within the context of how the brand appears in a person’s life, and what they are saying about the brands or their lifestyle. For example, if someone is posting a photo of themselves eating or drinking, it is possible to look at who they are eating with, the kind of place they are eating in, and whether (or not) they seem to be feeling good about the experience. Are they saying ‘I’m being healthy’ or ‘this is my guilty pleasure’? This kind of contextualised information gives companies a framework for how the brand is being used and perceived.
Geo-location information also shows the precise location where the photo was taken, where people are using the products, and potentially helping to identify trends. Companies can also use this sort of data for customer engagement by retweeting, favouriting, and following people who engage with their brand on social media.
Traditional Social Media Monitoring (SMM) mines text posted on social media. Here, though, photos are mined for occurrences of a brand, later filtering out false positives and incidentally occurring brands (‘photo-bombers’) by using machine learning. This, in turn, uses algorithms to improve precision in detecting logos, products, and patterns, so a crease in a shirt won’t be mistaken for a Nike Logo, say.
This is where it gets trickier. SMM has come under fire for not always reflecting real usage of brands or genuine sentiment, even when using ‘solutions’ to this problem such as Automated Sentiment Analysis. A blog post, or even a short caption in a status update, can contain multiple tonalities of such subtlety that even humans struggle to discern the intended tone or sentiment. So this begs the question, can human sentiment be analysed by something that is automated?
Ditto Labs uses Image Recognition (IR), the method of acquiring, analysing, and understanding images to produce numerical information. This uses three types of IR technology: facial recognition, optical character recognition, and pattern recognition.
The facial recognition software analyses facial expressions, claiming to ‘capture’ sentiment felt towards a brand. It builds a ‘smile score’ based on the intensity of emotion shown. Such analytics are more subjective, with the data providing a backbone for interpretation of sentiment. In other words, Ditto Labs provides the‘visual’ smile score data, but the client interprets it.
As well as using software to identify brands and assess people’s ‘sentiment’ within a photo, the company also provides textually-derived sentiment analysis in its basic form. This it also turns into a ‘smile score’.
It is here that the process may begin to be overstretched in terms of its capability. The analytics seem most unlikely to provide the real depth of qualitative research, even if able to analyse text in conjunction with the photos.
Overall, they may be able to analyse if a person is positive, negative, or neutral, but it is much more complicated to link these sentiments to any visible brand. They may be circumstantially related, but establishing any level of causation is a massive challenge and one which many informed observers would say is improbable, if not impossible.
Ditto Labs already filters out photos in which brands are ‘photo-bombing’ (where their appearance is ‘incidental’) to strengthen the link between sentiment and brand. But although it will be analysing photos in which the brands are more purposefully featured, it still cannot definitively state whether an individual’s sentiment is linked to the brand/product simply by analysing facial expressions and accompanying text.
Complementary tool
In this respect there seems to be a clear role for qualitative research as a complementary analytical tool rather than as a substitute. In sentiment analysis, both at the stage of benchmarking facial expressions for a ‘smile score’, and for exploring and understanding the nature of the relationships which appear to exist in social media between consumers and brands, allowing humans to decipher human sentiment may well remain the most reliable option, albeit not the least expensive.
