{"id":35295,"date":"2018-08-22T18:59:04","date_gmt":"2018-08-22T15:59:04","guid":{"rendered":"https:\/\/www.altoros.com\/blog\/?p=35295"},"modified":"2018-08-23T09:38:47","modified_gmt":"2018-08-23T06:38:47","slug":"multimodal-sentiment-analysis-with-tensorflow","status":"publish","type":"post","link":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/","title":{"rendered":"Multimodal Sentiment Analysis with TensorFlow"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_79_2 counter-hierarchy ez-toc-counter ez-toc-transparent ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Beyond_conventional_sentiment_analysis\" >Beyond conventional sentiment analysis<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#What_is_a_multimodal_sentiment_analysis\" >What is a multimodal sentiment analysis?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Multimodal_data_set\" >Multimodal data set<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Training_the_model_with_TensorFlow\" >Training the model with TensorFlow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Want_details_Watch_the_video\" >Want details? Watch the video!<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Related_slides\" >Related slides<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#Further_reading\" >Further reading<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#About_the_expert\" >About the expert<\/a><\/li><\/ul><\/nav><\/div>\n<h3><span class=\"ez-toc-section\" id=\"Beyond_conventional_sentiment_analysis\"><\/span>Beyond conventional sentiment analysis<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Sentiment analysis is a widespread approach to gaining valuable insights based on mostly textual data. The approach has become one of the essential ingredients while developing <a href=\"https:\/\/www.altoros.com\/blog\/tensorflow-for-recommendation-engines-and-customer-feedback-analysis\/\" >recommendation engines<\/a> and <a href=\"https:\/\/www.altoros.com\/blog\/building-a-chatbot-with-tensorflow-and-keras\/\">chatbots<\/a>, analyzing <a href=\"https:\/\/www.altoros.com\/blog\/machine-learning-for-automating-a-customer-service-chatbots-and-neural-networks\/\">customer feedback<\/a>, etc. Sentiment analysis is also used to enable <a href=\"https:\/\/www.altoros.com\/blog\/tensorflow-in-finance-discussing-predictive-analytics-and-budget-planning\/\">predictive analytics<\/a> and estimate budgets in finance, for instance.<\/p>\n<p>While textual sentiment analysis is heavily populated and actively researched, sentiment analysis based on visual or audio data gets far less attention. As it might be fundamentally different from simply classifying images, visual sentiment analysis also imposes certain difficulties like:<\/p>\n<ul>\n<li>a high level of abstraction required to grasp the message behind an image<\/li>\n<li>implicit knowledge of culture involved<\/li>\n<li>human subjectivity implying that two individuals may use the same image to express different emotions<\/li>\n<\/ul>\n<div id=\"attachment_35557\" style=\"width: 160px\" class=\"wp-caption alignright\"><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/anthony-hu-spotify-tensorflow-meetup.jpeg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-35557\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/anthony-hu-spotify-tensorflow-meetup.jpeg\" alt=\"\" width=\"150\" height=\"150\" class=\"size-full wp-image-35557\" \/><\/a><p id=\"caption-attachment-35557\" class=\"wp-caption-text\"><small>Anthony Hu<\/small><\/p><\/div>\n<p>With the proliferation of social media, traditional text-based sentiment analysis adopted more complex models to aggregate data in its diversity of forms\u2014text, still images, audio, videos, GIFs, etc. At the <a href=\"https:\/\/www.meetup.com\/ru-RU\/Artificial-Intelligence-Experience-London\/events\/251511314\/\" rel=\"noopener\" target=\"_blank\">recent TensorFlow meetup<\/a> in London, <a href=\"https:\/\/www.linkedin.com\/in\/anthhu\/\" rel=\"noopener\" target=\"_blank\">Anthony Hu<\/a> of Spotify introduced the audience to a novel approach of <em>multimodal sentiment analysis<\/em>. With deep neural networks combining visual analysis and <a href=\"https:\/\/www.altoros.com\/blog\/natural-language-processing-saves-businesses-millions-of-dollars\/\">natural language processing<\/a>, the goal behind this approach is to infer the latent emotional state of a user.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"What_is_a_multimodal_sentiment_analysis\"><\/span>What is a multimodal sentiment analysis?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Apart from textual data, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Multimodal_sentiment_analysis\" rel=\"noopener\" target=\"_blank\">multimodal sentiment analysis<\/a> inspects other modalities, specifically audio and visual data. So, the method can be bimodal or trimodal featuring different combinations of either two or three modalities\u2014text, audio, and video\u2014respectively.<\/p>\n<p>As with the traditional sentiment analysis, one of the most basic tasks in multimodal sentiment analysis is classification of different sentiments into such categories as positive, negative, or neutral. However, various combinations of text, audio, and visual features are analyzed under the multimodal approach. For this reason, there exist three fusion techniques that help to execute analysis: <em>feature-level<\/em>, <em>decision-level<\/em>, and <em>hybrid<\/em> fusion.<\/p>\n<p><em>Feature-level<\/em> fusion, aka early fusion, aggregates all the available features from each modality and puts them together into a single feature vector, which is then fed into a classification algorithm. Though, one may experience certain difficulties with integrating heterogeneous features under this technique.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/tensorflow-multimidal-analysis-co-memory-network-with-memory-hops.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/tensorflow-multimidal-analysis-co-memory-network-with-memory-hops-1024x563.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-35940\" \/><\/a><small>A co-memory network with memory hops for multimodal sentiment analysis (<a href=\"https:\/\/www.researchgate.net\/publication\/325357998_A_Co-Memory_Network_for_Multimodal_Sentiment_Analysis\" rel=\"noopener\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p><em>Decision-level<\/em> fusion, aka late fusion, treats data from each modality independently into its own classification algorithm. The final classification results are acquired through fusing each result into a single decision vector. Unlike the feature-level fusion, the decision-level one doesn&#8217;t involve the integration of heterogeneous data. This way, each of the modalities can use the most favourable classification algorithm.<\/p>\n<p>Naturally, <em>hybrid fusion<\/em> represents a combination of feature-level and decision-level techniques. Usually, it&#8217;s a  two-step procedure with the feature-level fusion initially performed between two modalities and the decision-level one applied as a second step to fuse the initial results with the remaining modality.<\/p>\n<p>In practice, a multimodal approach can be exhausted in the development of virtual assistants, analysis of YouTube video reviews, emotion recognition, etc. In the latter case, multimodal sentiment analysis might be of a great help to healthcare when detecting the symptoms of anxiety, stress, or depression.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Multimodal_data_set\"><\/span>Multimodal data set<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>In his research on applying multimodal approach, Anthony and his fellow team members built a data set based on Tumblr\u2014a microblogging service where users post multimedia<br \/>\ncontent usually accompanied by images, text, and tags. These tags can be viewed as a an emotional self-report of a user. This is essential to the research, as its goal was to predict an emotional state of a user rather than sentiment polarity.<\/p>\n<p>Using the Tumblr API, the team searched for emotions through each tagged post from 2011 to 2017. By filtering out non-English blog posts, the team has come up with a total of approximately 250,000 articles. As a result, the data set comprised 15 different emotions form the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Positive_and_Negative_Affect_Schedule\" rel=\"noopener\" target=\"_blank\">PANAS-X<\/a> scale and the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Robert_Plutchik#Plutchik's_wheel_of_emotions\" rel=\"noopener\" target=\"_blank\">Plutchik\u2019s wheel of emotions<\/a>.<\/p>\n<p>To deal with the visual component of a data set, <a href=\"https:\/\/www.altoros.com\/blog\/what-is-behind-deep-reinforcement-learning-and-transfer-learning-with-tensorflow\/\">transfer learning<\/a>, and the was Inception neural network in particular, was applied. With an architecture comprising 22 layers, the Inception neural network is a common choice as it&#8217;s one of the most advanced and precise neural networks.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analysis-with-tensorflow.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analysis-with-tensorflow-1024x575.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-35555\" \/><\/a><small>A high-level architecture of a multimodal neural network (<a href=\"https:\/\/www.slideshare.net\/seldon_io\/tensorflow-16-multimodal-sentiment-analysis-with-tensorflow\" rel=\"noopener\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>The text-based data was aggregated using the means of <a href=\"https:\/\/www.altoros.com\/blog\/natural-language-processing-saves-businesses-millions-of-dollars\/\">natural language processing<\/a>. The recurrent neural networks utilized had a temporal awareness of what was said before in a context.<\/p>\n<blockquote><p><em>&#8220;To understand someone, you&#8217;ve got memory, and you know the context, so you are aware of what the person said before. So, you have to keep track of that, and we use recurrent neural networks for that.&#8221; \u2014Anthony Hu, Spotify<\/em><\/p><\/blockquote>\n<p>In the very case, the text of a Tumblr blog post was broken down into a sequence of words that were embedded into a high-dimensional space to preserve semantic relationship and then fed<br \/>\ninto a <a href=\"https:\/\/www.altoros.com\/blog\/using-long-short-term-memory-networks-and-tensorflow-for-image-captioning\/\">long short-term memory<\/a> layer.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Training_the_model_with_TensorFlow\"><\/span>Training the model with TensorFlow<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>To read data in a parallel thread, thus ensuring it is always ready to be consumed, <a href=\"https:\/\/www.tensorflow.org\/api_docs\/python\/tf\/data\/TFRecordDataset\" rel=\"noopener\" target=\"_blank\">TFRecords<\/a>\u2014a binary file format used by TensorFlow were used for fast data processing. The <code style=\"color: #222222; background-color: #e6e6e6; padding: 1px 2px;\">TFRecordDataset<\/code> consumes <code style=\"color: #222222; background-color: #e6e6e6; padding: 1px 2px;\">TFRecords<\/code> and automatically creates a queue for training.<\/p>\n<p>Training a model, which involved only a text-based data set, resulted in 69% of test accuracy. It was twice higher than the result of training the image-based model, which means text is a better input for recognizing emotions. The multimodal approach combining both text and images achieved 80% of training accuracy and 72% of test accuracy.<\/p>\n<p>Before experimenting with multiple modalities, the team first utilized random guessing, which resulted in 11% of accuracy, and trained a purely image model, which achieved 36% of test accuracy and 43% of train accuracy. This way, the multimodal approach significantly outplays all the other models employed.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analysis-with-tensorflow-train-accuracy-results.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analysis-with-tensorflow-train-accuracy-results.png\" alt=\"\" width=\"640\" height=\"360\" class=\"aligncenter size-full wp-image-35620\" \/><\/a><small>Train accuracy results (<a href=\"https:\/\/arxiv.org\/pdf\/1805.10205.pdf\" rel=\"noopener\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>To validate how well the system recognizes different shades of emotions, the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Emotion_classification\" rel=\"noopener\" target=\"_blank\">circumplex model<\/a> was applied. This model suggests that emotions are distributed in a two-dimensional circular space with arousal and valence dimensions. The arousal dimension represents the vertical axis, while the valence one embodies the horizontal axis, and the center of the circle is a neutral valence and a medium level of arousal.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/a-circumplex-model-tensorflow-multimodal-analysis.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/a-circumplex-model-tensorflow-multimodal-analysis-1024x554.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-35925\" \/><\/a><small>A circumplex model of emotions (<a href=\"https:\/\/www.slideshare.net\/seldon_io\/tensorflow-16-multimodal-sentiment-analysis-with-tensorflow\" rel=\"noopener\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>It&#8217;s important to note that no validation set was used to tune hyperparameters, meaning that better performance could be reached. Anthony and his colleagues published a <a href=\"https:\/\/arxiv.org\/abs\/1805.10205\" rel=\"noopener\" target=\"_blank\">research<\/a>, where they detail the approach and the findings.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Want_details_Watch_the_video\"><\/span>Want details? Watch the video!<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<table width=\"100%\">\n<tbody>\n<tr>\n<td>\n<div style=\"float:right; width:50%; padding-left:15px; font-size:14px;\">\n<p><strong>Table of contents<\/strong><\/p>\n<ol>\n<li style=\"margin-bottom: 16px;\">Early research in sentiment analysis  (00:40)<\/li>\n<li style=\"margin-bottom: 16px;\">Building a multimodal data set (02:25)<\/li>\n<li style=\"margin-bottom: 16px;\">An architecture of multimodal neural network (06:16)<\/li>\n<li style=\"margin-bottom: 16px;\">Training a model with TensorFlow (11:26)<\/li>\n<li style=\"margin-bottom: 16px;\">Training results (13:05)<\/li>\n<li style=\"margin-bottom: 16px;\">A circumplex model of emotions (18:01)<\/li>\n<li style=\"margin-bottom: 16px;\">Hierarchical clustering of emotions (20:17)<\/li>\n<\/ol>\n<\/div>\n<div class=\"video-container\"><iframe loading=\"lazy\" title=\"TensorFlow London: Multimodal Sentiment Analysis with TensorFlow\" width=\"1200\" height=\"675\" src=\"https:\/\/www.youtube.com\/embed\/Ll8aUTrzjR8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Related_slides\"><\/span>Related slides<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><center><iframe loading=\"lazy\" src=\"\/\/www.slideshare.net\/slideshow\/embed_code\/key\/4YHgdY3aYB7QKj\" width=\"595\" height=\"485\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" style=\"border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;\" allowfullscreen> <\/iframe><\/center><\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Further_reading\"><\/span>Further reading<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/tensorflow-for-recommendation-engines-and-customer-feedback-analysis\/\">TensorFlow for Recommendation Engines and Customer Feedback Analysis<\/a><\/li>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/natural-language-processing-and-tensorflow-implementation-across-industries\/\">Natural Language Processing and TensorFlow Implementation Across Industries<\/a><\/li>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/approaches-and-models-for-applying-natural-language-processing\/\">Approaches and Models for Applying Natural Language Processing<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"About_the_expert\"><\/span>About the expert<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/anthony-hu-spotify-icon.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/anthony-hu-spotify-icon.png\" alt=\"\" width=\"150\" height=\"150\" class=\"alignright size-full wp-image-35308\" \/><\/a><\/p>\n<p><small><a href=\"https:\/\/www.linkedin.com\/in\/anthhu\/\" rel=\"noopener\" target=\"_blank\">Anthony Hu<\/a> has joined Machine Intelligence Laboratory at the University of Cambridge to work on computer vision and machine learning applied to autonomous vehicles, more precisely in scene understanding and vehicle&#8217;s interpretability. Previously, he served as a research scientist at Spotify where he worked on identifying musical similarities at a large scale using audio. Anthony holds MSc in Applied Statistics from the University of Oxford. Prior to that, he went to Telecom ParisTech, one of the top French public institutions of higher education and research engineering.<\/small><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Beyond conventional sentiment analysis<\/p>\n<p>Sentiment analysis is a widespread approach to gaining valuable insights based on mostly textual data. The approach has become one of the essential ingredients while developing recommendation engines and chatbots, analyzing customer feedback, etc. Sentiment analysis is also used to enable predictive analytics and estimate budgets in [&#8230;]<\/p>\n","protected":false},"author":3,"featured_media":35948,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[7],"tags":[748,749],"class_list":["post-35295","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news-and-opinion","tag-machine-learning","tag-tensorflow"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Multimodal Sentiment Analysis with TensorFlow | Altoros<\/title>\n<meta name=\"description\" content=\"A multimodal sentiment analysis poses an alternative to a conventional one, capable of processing text, visual, and audio data in a single workflow.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Multimodal Sentiment Analysis with TensorFlow | Altoros\" \/>\n<meta property=\"og:description\" content=\"Beyond conventional sentiment analysis Sentiment analysis is a widespread approach to gaining valuable insights based on mostly textual data. The approach has become one of the essential ingredients while developing recommendation engines and chatbots, analyzing customer feedback, etc. Sentiment analysis is also used to enable predictive analytics and estimate budgets in [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/\" \/>\n<meta property=\"og:site_name\" content=\"Altoros\" \/>\n<meta property=\"article:published_time\" content=\"2018-08-22T15:59:04+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-08-23T06:38:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"640\" \/>\n\t<meta property=\"og:image:height\" content=\"360\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Sophia Turol\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sophia Turol\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/\",\"url\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/\",\"name\":\"Multimodal Sentiment Analysis with TensorFlow | Altoros\",\"isPartOf\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif\",\"datePublished\":\"2018-08-22T15:59:04+00:00\",\"dateModified\":\"2018-08-23T06:38:47+00:00\",\"author\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/58194952af19fe7b2b830846e077a58e\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage\",\"url\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif\",\"contentUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif\",\"width\":640,\"height\":360},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.altoros.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Multimodal Sentiment Analysis with TensorFlow\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#website\",\"url\":\"https:\/\/www.altoros.com\/blog\/\",\"name\":\"Altoros\",\"description\":\"Insight\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.altoros.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/58194952af19fe7b2b830846e077a58e\",\"name\":\"Sophia Turol\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2019\/05\/trello_card-96x96.jpg\",\"contentUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2019\/05\/trello_card-96x96.jpg\",\"caption\":\"Sophia Turol\"},\"description\":\"Sophia Turol is passionate about delivering well-structured articles that cater for picky technical audience. With 3+ years in technical writing and 5+ years in editorship, she enjoys collaboration with developers to create insightful, yet intelligible technical tutorials, overviews, and case studies. Sophie is enthusiastic about deep learning solutions\u2014TensorFlow in particular\u2014and PaaS systems, such as Cloud Foundry.\",\"url\":\"https:\/\/www.altoros.com\/blog\/author\/sophie-turol\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Multimodal Sentiment Analysis with TensorFlow | Altoros","description":"A multimodal sentiment analysis poses an alternative to a conventional one, capable of processing text, visual, and audio data in a single workflow.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/","og_locale":"en_US","og_type":"article","og_title":"Multimodal Sentiment Analysis with TensorFlow | Altoros","og_description":"Beyond conventional sentiment analysis Sentiment analysis is a widespread approach to gaining valuable insights based on mostly textual data. The approach has become one of the essential ingredients while developing recommendation engines and chatbots, analyzing customer feedback, etc. Sentiment analysis is also used to enable predictive analytics and estimate budgets in [...]","og_url":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/","og_site_name":"Altoros","article_published_time":"2018-08-22T15:59:04+00:00","article_modified_time":"2018-08-23T06:38:47+00:00","og_image":[{"width":640,"height":360,"url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif","type":"image\/gif"}],"author":"Sophia Turol","twitter_misc":{"Written by":"Sophia Turol","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/","url":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/","name":"Multimodal Sentiment Analysis with TensorFlow | Altoros","isPartOf":{"@id":"https:\/\/www.altoros.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage"},"image":{"@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage"},"thumbnailUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif","datePublished":"2018-08-22T15:59:04+00:00","dateModified":"2018-08-23T06:38:47+00:00","author":{"@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/58194952af19fe7b2b830846e077a58e"},"breadcrumb":{"@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#primaryimage","url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif","contentUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2018\/08\/multimodal-sentiment-analasis-with-tensorflow-v1.gif","width":640,"height":360},{"@type":"BreadcrumbList","@id":"https:\/\/www.altoros.com\/blog\/multimodal-sentiment-analysis-with-tensorflow\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.altoros.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Multimodal Sentiment Analysis with TensorFlow"}]},{"@type":"WebSite","@id":"https:\/\/www.altoros.com\/blog\/#website","url":"https:\/\/www.altoros.com\/blog\/","name":"Altoros","description":"Insight","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.altoros.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/58194952af19fe7b2b830846e077a58e","name":"Sophia Turol","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2019\/05\/trello_card-96x96.jpg","contentUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2019\/05\/trello_card-96x96.jpg","caption":"Sophia Turol"},"description":"Sophia Turol is passionate about delivering well-structured articles that cater for picky technical audience. With 3+ years in technical writing and 5+ years in editorship, she enjoys collaboration with developers to create insightful, yet intelligible technical tutorials, overviews, and case studies. Sophie is enthusiastic about deep learning solutions\u2014TensorFlow in particular\u2014and PaaS systems, such as Cloud Foundry.","url":"https:\/\/www.altoros.com\/blog\/author\/sophie-turol\/"}]}},"_links":{"self":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/35295","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/comments?post=35295"}],"version-history":[{"count":53,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/35295\/revisions"}],"predecessor-version":[{"id":35966,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/35295\/revisions\/35966"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/media\/35948"}],"wp:attachment":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/media?parent=35295"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/categories?post=35295"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/tags?post=35295"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}