Interviews

Interview with Yahoo’s Research Scientist Miriam Redi

Miriam Redi is a Research Scientist at Yahoo Labs, London. Her research focuses on content-based social multimedia analysis. In particular, she explores ways to automatically assess image semantics, visual aesthetics, and multimedia creativity.

Miriam-Redi

Miriam developed a strong visual analysis background during her Ph.D. at EURECOM Sophia Antipolis, with a thesis on “Novel Methods for Semantic and Aesthetic Multimedia Retrieval” and publications in main Multimedia conferences. After obtaining her Ph.D. in May 2013, she was an intern at Yahoo Labs Barcelona, working on video semantics and creativity assessment. Miriam’s passion for visual analysis started during her master thesis on blind image manipulation detection, developed at University of California Santa Barbara. I did an interview with her, recently. You can read the interview below:
1. As a vision researcher at Yahoo Labs, you developed a machine learning algorithm (CrowdBeauty) that can distinguish beautiful portraits from ugly ones. How does it work?
The CrowdBeauty algorithm can identify beautiful images depicting different subjects, including people, animals, nature, and urban scenery. To develop such an efficient computational aesthetic algorithm, we needed to teach a machine how to distinguish between good and bad images using an algorithmic approach.
To do so, we created a data set of 10,000 Flickr images and asked a large number of crowdsourced contributors to judge the beauty of the photos. (The data set is publicly available at di.unito.it). Next, from each picture we extracted a set of visual features that described the image’s visual properties, such as the light contrast, the color distribution, and the compositional uniqueness.
We then ingested the image features and the corresponding aesthetic judgments into a machine learning framework. The machine learned how to associate the photographic properties of an image with its degree of beauty. This would allow the algorithm to automatically score the beauty of new, unseen Flickr images.
2. Is it the same algorithm that identifies the unpopular Flickr images and digs out the un-noted photographic gems, hidden within this long tail of unpopularity?
Once trained, CrowdBeauty is able to score the aesthetic degree of any image in the Flickr network. To surface the hidden photographic gems, we isolated a large collection of around 10M unpopular Flickr images, and employed CrowdBeauty to compute a beauty score for each of them.
We then ranked the images in terms of their computed aesthetic score. The images at the top of the ranking corresponded to the most beautiful images in the collection, according to the CrowdBeauty algorithm. These were the hidden gems we were looking for: they were beautiful, yet unpopular images that would remain hidden from our eyes if we did not have a computational aesthetic algorithm to detect their presence in the long tail of unpopularity.
3. How about the accuracy of the results? How close is it to the human judgement of beauty?
To evaluate the reliability of the algorithm, and verify the high degree of beauty of the surfaced photographic gems, we again resorted to a crowd sourcing experiment. We asked annotators to evaluate the aesthetic degree of a new set of images, which included the images detected by the algorithm, and groups of Flickr images ranked as very popular, average popular and unpopular.
We were very happy to see that, according to this evaluation, CrowdBeauty was able to detect photos whose median perceived beauty score was equal to the most popular ones, and whose average was lower by only 1.5 points (out of a scale of 5).
4. So, there is hope for all those unappreciated photographers out there, right? Particularly with the new Flickr Beauty Explorer page?
With our work, we want to stress the importance of promoting content quality, as opposed to the widel used popularity-based metrics, including number of views, favorites and comments. Many talented photographers do not receive the attention they deserve due, in part, to the low popularity-based scores the receive with photo sharing sites.
With CrowdBeauty, we are able to discover high quality pictures, independent of their social signals. Such hidden gems could be collected in a new Flickr Beauty Explorer page showcasing the most beautiful yet undiscovered images, thus drawing people’s attention towards talented but underrepresented photographers. This would complement the current Flickr Explorer image collection, whose stunning pictures are characterized by very high social engagement.
5. What are the other exciting possibilities of this technology?
Our work promotes the democratization of photo sharing platforms, and creates an opportunity to balance the visibility of popular and beautiful photos with those that are as beautiful but with less social exposure.
This technology allows to increase the payoff of the photo service by uncovering valuable content, which can be used for promotion, advertising, mashup, or any other commercial service, that would have otherwise gone unnoticed.
On a different note, our algorithm rewards users based on their photographic talent rather than their popularity, thus fostering both the engagement and the artistic production of less popular contributors.
Moreover, while social dynamics require a large time span to establish a judgement on users’ talents, our algorithm could assess photographers’ skills as soon as they join the network by simply analyzing their image collections. This new perspective could motivate new users to contribute more to photo sharing sites, and improve the overall quality of the content.
Thank you for spending your valuable time with us, Miriam.
Miriam Redi spoke at the RE.WORK Deep Learning Summit, London, in September 2015. The next Deep Learning Summit takes place in San Francisco on 28-29 January. Use our discount code BDMS20 for 20% off tickets!

9 Comments
  1. minecraft free 7 months ago
    Reply

    Hi there! I know this is somewhat off topic but I was wondering which blog platform are you using for this site?
    I’m getting fed up of WordPress because I’ve
    had problems with hackers and I’m looking at options for another platform.

    I would be great if you could point me in the direction of a good platform.

  2. top ps4 games 2019 6 months ago
    Reply

    I’m extremely pleased to discover this page. I need to to thank you for ones time for this
    particularly fantastic read!! I definitely appreciated every bit of
    it and i also have you saved to fav to look at new stuff on your site.

  3. quest bars cheap 6 months ago
    Reply

    When I originally commented I seem to have clicked the -Notify me when new comments are added-
    checkbox and now each time a comment is added I receive 4 emails
    with the same comment. Perhaps there is a way you are able to remove me from that
    service? Cheers!

  4. descargar facebook 4 months ago
    Reply

    WOW just what I was looking for. Came here by searching for descargar facebook

  5. quest bars cheap 3 months ago
    Reply

    Appreciate the recommendation. Let me try it out.

  6. coconut oil are 4 weeks ago
    Reply

    Good day! I just want to offer you a huge thumbs up for the great info you have
    here on this post. I am coming back to your
    blog for more soon.

  7. quest bars cheap 1 week ago
    Reply

    Good post but I was wanting to know if you could write a litte more on this topic?
    I’d be very thankful if you could elaborate a little bit further.
    Kudos!

  8. quest bars cheap 1 week ago
    Reply

    This text is worth everyone’s attention. Where can I find
    out more?

  9. ps4 games 5 days ago
    Reply

    It’s really a great and useful piece of information. I am satisfied that you simply shared this useful info with us.
    Please stay us informed like this. Thanks for sharing.

Leave a Comment

Your email address will not be published.

You may also like

Pin It on Pinterest