Analyzing Text and Generating Content with Neural Networks and TensorFlow
Can convolutional neural networks, typically used for image processing, accelerate text processing? Where do word embeddings come in here to help? How to generate unique content by using TensorFlow? This blog post explores these questions as discussed at the recent TensorFlow meetup in Denver.
Natural language processing with neural networks
Ville Kallioniemi, a software engineer at Oracle, focused on word embeddings as a means of natural language processing (NLP). He explained that word embeddings represent a basic building block of NPL systems, which need numerical inputs. As an example, he mentioned a popular machine learning algorithm—word2vec. It helps to create embeddings using a shallow neural network that predicts the next word or surrounding words.
(Previously, we have briefly written about text analysis with word2vec, which can view a sequence of words as vectors.)
Though convolutional neural networks (CNN) are mostly used for image processing, Ville highlighted that one can get satisfying results when applying them for text classification. However, one has “to turn a word into something that a CNN will understand.” That’s where word embeddings come in to split text into “eatable” pieces.
Ville also exemplified a few scenarios where text classification can be of use:
- e-mail (ham or spam)
- tweet sentiment (positive, negative, or neutral)
- social media posting (a topic company X is interested in)
You can find Ville’s presentation below.
Generating content with TensorFlow
In his session, Martyn Garcia of 255 BITS overviewed a number of tools that emerge from TensorFlow and help to generate unique content. Explaining how these tools work, he divided them into two groups:
- Sequence: char-rnn, sketch-rnn, and wavenet
- Real-valued: autoencoders, generative adversarial networks, vae, and hybrid
Take a look at Martyn’s presentation here.
Join our group to stay tuned with the upcoming events.
Want details? Watch the video!
- Using Convolutional Neural Networks and TensorFlow for Image Classification
- Image and Text Recognition with TensorFlow Using Convolutional Neural Networks
- Text Prediction with TensorFlow and Long Short-Term Memory—in Six Steps
- How TensorFlow Can Help to Perform Natural Language Processing Tasks
About the experts
Ville Kallioniemi is a software engineer for Oracle, working on deep learning applications. His professional experience ranges from embedded systems to machine learning.
Martyn Garcia is a full-stack engineer focusing on machine learning. When not consulting, he contributes to open-source projects, TensorFlow in particular.