Main description:

This book presents a summary of the multimodal analysis of user-generated multimedia content (UGC). Several multimedia systems and their proposed frameworks are also discussed. First, improved tag recommendation and ranking systems for social media photos, leveraging both content and contextual information, are presented. Next, we discuss the challenges in determining semantics and sentics information from UGC to obtain multimedia summaries. Subsequently, we present a personalized music video generation system for outdoor user-generated videos. Finally, we discuss approaches for multimodal lecture video segmentation techniques. This book also explores the extension of these multimedia system with the use of heterogeneous continuous streams.

Contents:

1 Introduction 1.1 Background and Motivation

1.2 Overview

1.3 Acronyms and Notations

1.4 Roadmap

2 Literature Review

2.1 Event Understanding

2.2 Tag Recommendation and Ranking

2.3 Soundtrack Recommendation for UGVs

2.4 Lecture Video Segmentation

3 Event Understanding

3.1 Introduction

3.2 System Overview

3.2.1 EventBuilder

3.2.2 EventSensor

3.3 Evaluation

3.3.1 EventBuilder

3.3.2 EventSensor

3.4 Summary

4 Tag Recommendation and Ranking

4.1 Introduction

4.1.1 Tag Recommendation

4.1.2 Tag Ranking

4.2 System Overview

4.2.1 Tag Recommendation

4.2.2 Random Walk based Relevance Scores

4.2.3 Fusion of Different Tag Recommendation Approaches

4.2.4 Tag Ranking

4.3 Evaluation

4.3.1 Tag Recommendation

4.3.2 Tag Ranking

4.4 Summary

5 Soundtrack Recommendation for UGVs

5.1 Introduction

5.1.1 Increasing Popularity of User-Generated Videos

5.1.2 Challenges with User-Generated Videos in Viewing and Sharing

5.1.3 Motivation for Generating Music Videos for Outdoor User-Generated Videos

5.2 Music Video Generation

5.2.1 Scene Moods Prediction Models

5.2.2 Music Retrieval Techniques

5.2.3 Automatic Music Video Generation Model

5.3 Evaluation

5.3.1 Dataset and Experimental Settings

5.3.2 Evaluation Metrics

5.3.3 Objective Evaluation

5.3.4 Subjective Evaluation

5.3.5 Experimental Results

5.3.6 Comparison with State-of-the-arts

5.3.7 Discussion of Results

5.4 Summary

6 Lecture Video Segmentation

6.1 Introduction

6.2 Lecture Video Segmentation

6.2.1 Prediction of Video Transition Cues using Supervised Learning

6.2.2 Computation of Text Transition Cues using N-gram based Language Model

6.2.3 Computation of SRT Segment Boundaries using the state-of-the-art

6.2.4 Computation of Wikipedia Segment Boundaries

6.2.5 Transition File Generation

<6.3 Evaluation

6.3.1 Dataset and Experimental Settings

6.3.2 Results from the ATLAS System

6.3.3 Results from the TRACE System

6.4 Summary

7 Conclusions and future work