\(~~~~~~~~~~\)Text Mining\(~~~~~~~~~~\)

Asst. Prof. Dr. Somsak Chanaim

International College of Digital Innovation, CMU

October 15, 2026

Learning Objectives

By the end of this chapter, students are able to…

  1. Explain the basic concepts of Text Mining, NLP, and a text corpus.

  2. Match common NLP applications with their practical benefits.

  3. Distinguish among Standard Sentiment Analysis, Fine-Grained Sentiment Analysis, Emotion Detection, and Aspect-Based Sentiment Analysis.

  4. Explain the main steps of rule-based sentiment analysis: tokenization, stopword removal, keyword identification, and sentiment classification.

  5. Distinguish the training and prediction stages of machine-learning-based sentiment analysis.

  6. Interpret sentiment scores, word clouds, Bag of Words, and word-count vectors.

What is Text Mining

Text Mining (also called Text Data Mining or Text Analytics) is the process of extracting useful information, patterns, and knowledge from unstructured text data.

It combines techniques from natural language processing (NLP), machine learning, and statistics to transform text into structured data for analysis.

Application of Text Mining

Applications of Text Mining

Text Mining can be applied to many types of unstructured text data.

Domain Typical Text Data Main Tasks
🛍 Business Customer reviews Sentiment, topics, keywords
🏥 Healthcare Clinical notes, medical reports NER, classification
💰 Finance News, reports, complaints Sentiment, NER
🎓 Education & Research Essays, forums, articles Similarity, summarization

Common Idea

Different domains use different text sources, but the goal is similar:

Text Data → Text Mining Technique → Insight → Decision / Action

NER = Named Entity Recognition

🛍 Business: Customer Review Analysis

Scenario: An e-commerce company receives thousands of customer reviews every day. The goal is to understand what customers like, dislike, and frequently discuss.

📄 Text Data

  • Product reviews
  • Customer feedback
  • Social media comments
  • Support messages

🔍 Text Mining Tasks

  • Sentiment Analysis → Positive / Neutral / Negative

  • Topic Modeling → Delivery, price, quality

  • Keyword Extraction → Frequently discussed issues

💡 Business Action

  • “late delivery” → investigate logistics problems
  • “battery problem” → review product quality
  • “great packaging” → identify product strengths

🏥 Healthcare: Mining Clinical Text

Scenario: Hospitals generate large amounts of unstructured medical text that may contain important information about diseases, symptoms, treatments, and patient risks.

📄 Text Data

  • Clinical notes
  • Discharge summaries
  • Radiology reports
  • Medical records

🔍 Text Mining Tasks

  • Named Entity Recognition (NER) → Diseases, symptoms, treatments

  • Text Classification → Categorize clinical documents

  • Pattern Mining → Discover common relationships

🩺 Healthcare Applications

  • Clinical decision support
  • Patient-risk identification
  • Medical-record summarization
  • Detection of recurring health patterns

💰 Finance: Mining Financial Text

Scenario: Financial organizations analyze large volumes of news, reports, and textual records to identify useful signals and potential risks.

📄 Text Data

  • Financial news
  • Analyst reports
  • Earnings-call transcripts
  • Customer complaints
  • Transaction descriptions

🔍 Text Mining Tasks

  • Sentiment Analysis → Market or company sentiment

  • Named Entity Recognition (NER) → Companies, people, financial entities

  • Text Classification → Categorize reports or suspicious text

  • Topic Analysis → Emerging financial themes

📈 Applications

  • Market sentiment monitoring
  • Fraud investigation support
  • News analysis
  • Earnings-call analysis

🎓 Education & Research

Scenario: Universities and researchers work with large amounts of textual information, including student work, discussions, and academic publications.

📄 Text Data

  • Student assignments
  • Discussion forums
  • Open-ended responses
  • Research articles

🔍 Text Mining Tasks

  • Text Similarity → Identify similar documents

  • Summarization → Generate concise article summaries

  • Sentiment / Topic Analysis → Analyze student feedback

  • Text Classification → Categorize responses or documents

📚 Applications

  • Academic integrity support
  • Learning analytics
  • Student-feedback analysis
  • Literature analysis

Comparing Text Mining Applications

Domain Text Data Main Techniques Example Insight
🛍 Business Customer reviews Sentiment, Topic Modeling Customer complaints
🏥 Healthcare Clinical notes NER, Classification Diseases and symptoms
💰 Finance News, reports Sentiment, NER Market sentiment
🎓 Education Essays, forums Similarity, Classification Learning patterns

Key Takeaway

The text-mining workflow is similar across domains,
but the data, analytical task, and interpretation depend on the problem.

\[ \text{Text} \rightarrow \text{Numerical Representation} \rightarrow \text{Analysis} \rightarrow \text{Insight} \]

NLP Applications and Their Benefits

The mock exam may ask you to match an NLP task with its practical benefit.

NLP Task Main Purpose Example Benefit
Spam Detection Classifies unwanted or suspicious email Helps manage inboxes efficiently
Voice Search Converts speech into a search/query action Supports hands-free control of devices
Machine Translation Translates text or speech between languages Enables real-time multilingual communication
Chatbot Automatically responds to user questions Reduces waiting time for product/order inquiries

Exam Tip

Focus on the purpose of each task. For example, Voice Search is about speech-based interaction, while Chatbots are about conversational responses.

Text Corpus

A text corpus is a collection of written or spoken texts assembled for language analysis or machine-learning tasks.

A useful corpus should represent the kind of real-world language relevant to the problem. Researchers can use corpora to study language patterns and to train or evaluate text-mining models.

Do Not Confuse

A text corpus is not simply a public library collection. It is a dataset of language samples prepared for analysis.

Natural Language Processing

Natural Language Processing (NLP) is a field of artificial intelligence (AI) that focuses on enabling computers to understand, interpret, and generate human language (spoken or written).

It combines techniques from linguistics, computer science, and machine learning to bridge the gap between human communication and computer understanding.

Key Capabilities of NLP

  1. Text Preprocessing → Tokenization, stemming, lemmatization, stop-word removal.
  2. Text Classification → Spam detection, sentiment analysis, topic labeling.
  3. Named Entity Recognition (NER) → Identifying people, places, dates, organizations in text.
  4. Machine Translation → Google Translate, DeepL.
  5. Sentiment Analysis → Detecting emotions (positive, negative, neutral) in text.
  6. Speech Recognition → Turning speech into text (e.g., Siri, Alexa).
  7. Text Generation → Chatbots, large language models (chatGPT, Gemini ✨).

Sentiment analysis

Sentiment Analysis is a text-mining technique used to identify the polarity or attitude expressed in text. A common task is to classify text as positive, negative, or neutral. More advanced approaches may also estimate sentiment intensity, emotions, or sentiment toward specific aspects.

Example Sentence and Sentiment Value

Some sentiment-analysis systems assign a sentiment score to text.

A common normalized scale is –1 (very negative) to +1 (very positive), but there is no universal sentiment-score scale. Different tools may use labels, probabilities, raw lexicon scores, or other scoring systems.

Sentence:

“The movie was fantastic and inspiring.”

  • Sentiment Value: +0.85 (strongly positive)

Sentence:

“The service was terrible and disappointing.”

  • Sentiment Value: –0.80 (strongly negative)

Sentence:

“The food was okay, nothing special.”

  • Sentiment Value: 0.05 (neutral / slightly positive)

1. Standard Sentiment Analysis

Task: Classify text into broad categories → Positive, Negative, or Neutral.

Example

  • Text: “The food was delicious.” → Positive
  • Text: “The service was slow.” → Negative

2. Fine-Grained Sentiment Analysis

Task: Break down sentiment into levels of polarity

  • Very Positive: 😍 / 🤩 / 🥳 / ⭐⭐⭐⭐⭐

  • Positive: 🙂 / 😊 / ⭐⭐⭐⭐

  • Neutral: 😐 / 😶 / ⭐⭐⭐

  • Negative: 🙁 / 😟 / ⭐⭐

  • Very Negative: 😡 / 😠 / 😭 / ⭐

Sentence

  • Text: “The movie was absolutely amazing!” → Very Positive
  • Text: “The product is okay.” → Neutral
  • Text: “This was the worst experience ever!” → Very Negative

3. Emotion Detection (identifies specific emotions)

Task: Use NLP and psychology-based models to classify emotions.

  • Happiness
  • Anger
  • Sadness
  • Fear
  • Surprise
  • Disgust
  • etc.

  1. I’m so excited for my new job! → Joy/Excitement 😀🤩

  2. I’m scared about the results. → Fear 😨

  3. This food tastes terrible. → Disgust 🤢

  4. Wow, I didn’t expect that surprise party! → Surprise 😲

4. Aspect-Based Sentiment Analysis (ABSA)

Looks at specific aspects/features of a product or service.

Task: Identify what part of the product/service the sentiment refers to.

Aspect Sentiments:

  • Camera → Positive

  • Battery → Negative

  • Price → Neutral

Four Sentiment Analysis Types — Mock Exam Summary

Type What it identifies Typical Output
Standard Sentiment Analysis Overall polarity Positive / Negative / Neutral
Fine-Grained Sentiment Analysis Strength or level of sentiment Very good / Good / Okay / Bad / Very bad
Emotion Detection Specific emotion Joy / Anger / Sadness / Fear / etc.
Aspect-Based Sentiment Analysis (ABSA) Sentiment toward a specific aspect Camera = Positive, Battery = Negative

Quick Recognition

  • Five levels of opinion → Fine-Grained Sentiment Analysis
  • Positive / Negative / Neutral → Standard Sentiment Analysis
  • Joy / Anger / Sadness → Emotion Detection
  • Opinion about product features or service aspects → ABSA

Interactive Sentiment Analysis (Demo)

Teaching Demo

This interactive uses a simplified lexicon-based approach to demonstrate how sentiment scores can be constructed from positive/negative words, negation, intensifiers, and emoji cues. It is not a production NLP model. Thai word segmentation is intentionally simplified for classroom demonstration.

Example

  • I absolutely love this product—super easy to use! 🙂

  • The app is good, but the battery life is not great.

  • This update is incredibly fast and really impressive.

  • It’s not bad, just a bit slow sometimes.

  • The UX is terrible… I’m so disappointed. 👎

  • ใช้งานง่ายมาก ชอบฟีเจอร์ใหม่ที่สุด!

  • ไม่ดีเท่าไหร่ แถมค้างบ่อยๆ จนหงุดหงิด 😡

  • บริการโอเคนะ แต่ไม่ได้เร็วมาก

  • ราคาแพงไปนิด แต่คุณภาพก็ดีมากจริงๆ

  • Nothing special—works as expected.

Sentiment Analysis Workflows

Rule-Based Sentiment Analysis — Mock Exam Focus

Consider the sentence:

The movie was absolutely fantastic.

A simplified rule-based workflow can be read as follows:

\[\begin{aligned} &\text{Sentence} \rightarrow \text{Tokenization} \rightarrow \text{Stopword Removal}\\ &\rightarrow \text{Keyword Identification} \rightarrow \text{Sentiment Classification} \end{aligned} \]

Step Example Result Meaning
1. Text tokenization The, movie, was, absolutely, fantastic, . Split the sentence into tokens
2. Stopword removal movie, absolutely, fantastic Remove common words that add little information
3. Keyword identification fantastic Identify the sentiment-bearing word
4. Sentiment classification Positive Assign the final sentiment class

Mock Exam Tip

If a sentence changes from a complete sentence to a list of separate words, that is tokenization.
If common words such as the or was disappear, that is stopword removal.

Machine-Learning-Based Sentiment Analysis

Machine-learning sentiment analysis has two major stages: Training and Prediction.

Training Stage

\[\begin{aligned} &\text{Labeled Text} \rightarrow \text{Feature Extraction} \rightarrow \text{Numerical Features}\\ &\rightarrow \text{Machine-Learning Algorithm} \rightarrow \text{Trained Classifier Model} \end{aligned} \]

  • The machine-learning algorithm is the method that learns from labeled examples.
  • The classifier model is the trained model produced by the learning process.

Prediction Stage

\[\begin{aligned} &\text{New Text} \rightarrow \text{Feature Extraction} \rightarrow \text{Numerical Features} \rightarrow \\ &\text{Classifier Model} \rightarrow \text{Predicted Tag} \end{aligned}\]

Training vs. Prediction

Training uses labeled examples to build the model.
Prediction uses the trained model to classify new text.

Workflow of Sentiment Analysis

\[\begin{aligned} &\text{Raw Text} \rightarrow \text{Preprocessing} \rightarrow \text{Feature Representation} \rightarrow\\ &\text{Sentiment Model} \rightarrow \text{Sentiment Output} \end{aligned}\]

Example

Review:
“The battery is not good.”

Output:
Negative sentiment

Interpreting Sentiment Scores

A sentiment score is a numerical summary produced by a sentiment-analysis system. The exact scale depends on the tool, so interpretation should focus on direction and relative strength.

  • A positive score generally indicates positive sentiment.
  • A negative score generally indicates negative sentiment.
  • A score near zero is often interpreted as neutral or mixed, depending on the model.

Important for the Mock Exam

Do not judge a model only by the sign of the score. Read the original sentence and ask whether the assigned score is reasonable.

Example:

“I was hesitant to order because it looked poorly designed, but the product ended up surpassing my expectations in every way.”

The final clause is strongly positive. A clearly negative score would therefore be suspicious and should be examined carefully.

Preprocessing: Cleaning, Normalizing, and Structuring

Key Idea: Preprocessing Is Task-Dependent

There is no single preprocessing pipeline that should always be applied. A useful preprocessing step for Bag of Words or TF–IDF may remove information that is important for sentiment analysis.

  1. Tokenization

    • Break text into smaller units such as words, subwords, or phrases.
    • Example: “The movie was great” → [“The”, “movie”, “was”, “great”]

  2. Lowercasing / Normalization

    • Standardize text to reduce unnecessary variation.
    • Example: “Great” and “great” may be treated as the same token.

  1. Stop-Word Removal

    • Remove very common words when they add little information for the task.
    • Use with caution: words such as not can be important for sentiment.

  2. Stemming / Lemmatization

    • Reduce related word forms to a common base form.
    • Stemming example: “running”, “runs” → “run”
    • Lemmatization example: “am/are/is” → “be”

  3. Punctuation, Symbols, and Emoji

    • Remove them only when they are not useful for the analysis.
    • In sentiment analysis, !!!, ?, and emoji such as 🙂 or 😡 may carry useful information.

  1. Handling Negation

    • Preserve expressions such as “not good” because negation can reverse sentiment.

  2. Domain-Specific Normalization

    • Handle abbreviations, spelling variants, hashtags, or technical terminology when necessary.

Example: Choose Preprocessing for the Task

Technique BoW / TF–IDF Sentiment Analysis
Lowercasing Often useful Often useful
Stop-word removal Often useful Use with caution
Remove punctuation Often useful Use with caution
Remove emoji Sometimes Usually avoid
Handle negation Useful Very important

🔎 Feature Extraction

Feature Extraction is the process of transforming preprocessed text into numerical vectors that machine learning or deep learning models can understand.

From Text to Numbers

Bag of Words → How often does a word appear?
TF–IDF → How important is the word in this document?
Word Embeddings → What does the word mean?
Contextual Embeddings → What does the word mean in this context?

Main Techniques

1. Bag of Words (BoW)

  • Concept: Represents text by counting how many times each word appears, ignoring grammar and word order.

  • Pros: Simple, easy to implement.

  • Cons: Loses word context, results in sparse data.

Example:

  • Text: “The movie was great, great acting”
  • Features: {the:1, movie:1, was:1, great:2, acting:1}

Document–Term Matrix (DTM)

When we apply BoW or TF–IDF to many documents, the result can be organized as a Document–Term Matrix.

Document good bad service food
D1 2 0 1 0
D2 0 1 1 0
D3 1 0 0 2

Rows = documents, columns = terms, and cells = counts or TF–IDF weights.
This converts text into the same kind of numerical table used by many machine-learning algorithms.

2. TF–IDF

Beyond the core mock-exam scope: useful for deeper understanding of text representation.

TF–IDF = Term Frequency–Inverse Document Frequency

  • TF (Term Frequency): Measures how often a term appears in a document.
  • IDF (Inverse Document Frequency): Gives more weight to terms that appear in fewer documents.

High TF–IDF = frequent in this document, but relatively rare across other documents.

  • Pros: Highlights terms that help distinguish one document from others.
  • Cons: Does not capture word meaning or semantic context.

Example:

  • D1: battery battery excellent
  • D2: screen excellent
  • D3: camera excellent

The word excellent appears in every document → lower IDF.

The word battery appears only in D1 → higher TF–IDF weight in D1.

3. Word Embeddings

Beyond the core mock-exam scope: useful for deeper understanding of semantic representation.

  • Concept: Represents each word as a dense numerical vector, where words with similar meanings or usage tend to have similar vectors.

  • Models: Word2Vec, GloVe, fastText.

  • Pros: Captures semantic relationships between words.

  • Cons: Traditional word embeddings assign one fixed vector to each word, regardless of context.

Example:

  • king – man + woman ≈ queen

4. Contextual Embeddings

Beyond the core mock-exam scope: useful for deeper understanding of modern NLP.

  • Concept: Represents a word using a vector that changes according to the surrounding context.

  • Models: BERT, RoBERTa, and other transformer-based language models.

  • Pros: Captures context-dependent meanings and handles ambiguous words better than traditional word embeddings.

  • Cons: Requires more computational resources.

Example:

  • “bank” in “river bank”
    ≠
    “bank” in “financial bank”

The word bank receives different representations because its meaning changes with context.

From Text to Machine Learning Models

A typical text-mining workflow is:

\[\begin{aligned} &\text{Raw Text} \rightarrow \text{Preprocessing} \rightarrow \text{Numerical Representation} \rightarrow\\ &\text{Machine Learning Model} \rightarrow \text{Output} \end{aligned}\]

Key Idea

Machine learning models generally work with numerical features, not raw text.

Examples of numerical text representations:

  • Bag of Words (BoW)
  • TF–IDF
  • Word Embeddings
  • Contextual Embeddings

Regression

  • Input to the model: Numerical text features.
  • ⚙ Models: Linear Regression, Ridge/Lasso, SVR, Neural Networks.
  • Output: Continuous values, e.g., Predicted Rating = 4.2 or Expected Engagement Score = 2,000.

Clustering

  • Input to the model: Numerical text features.
  • ⚙ Models: k-Means, Hierarchical Clustering, DBSCAN.
  • Output: Groups of similar documents, e.g., Delivery Issues, Price Concerns, Product Quality.

Topic Modeling

Topic modeling is also an unsupervised text-mining approach, but it is not the same as clustering. It aims to discover latent themes or topics in a collection of documents.

  • Example method: Latent Dirichlet Allocation (LDA)
  • Example topics: delivery & shipping, price & value, product quality

Output & Visualization

After preprocessing, feature extraction, and modeling, a text-mining system produces results that can be interpreted and visualized.

Key Outputs

  1. Sentiment Label

    • The main classification result.
    • Categories: Positive, Negative, Neutral (or Very Positive → Very Negative in fine-grained analysis).
    • Example: “The product is excellent” → Positive

  1. Sentiment Score

    • A numeric value representing sentiment direction or intensity.
    • Some systems normalize the score to –1 to +1, but the scale depends on the tool or model.
    • Example: “I love this phone” → +0.85

  2. Class Probability / Confidence

    • A probability is always between 0 and 1 (or 0% and 100%).
    • Example: \(P(\text{Positive})=0.92\)

  1. Aspect-Based Sentiment

    • Sentiment toward specific product features.

    • Example:

      • “The phone’s camera is great but the battery is bad”

        • Camera → Positive (+0.8)
        • Battery → Negative (–0.7)

Reading a Word Cloud — Mock Exam Focus

In a typical word cloud, larger words appear more frequently in the source text. Therefore, ranking words by frequency can usually be done by comparing their displayed sizes.

Example

If Dog is the largest word, followed by Baby, Pet, Cat, and Whiskers, the frequency ranking is:

  1. Dog
  2. Baby
  3. Pet
  4. Cat
  5. Whiskers

Visualization Techniques

  1. Pie Charts: Show the proportion of Positive / Neutral / Negative reviews.

  2. Bar Charts: Compare sentiment across products, brands, or time periods.

  3. Time-Series Plots: Track sentiment trends over time.

  4. Word Clouds: Highlight frequently occurring words or phrases for exploratory analysis.

  5. Dashboards: Combine charts and KPIs to support decision-making.

Interpreting Word Clouds

Word clouds are useful for exploratory visualization, but word size usually represents frequency, not statistical importance or significance.

Use KWIC (Key Word in Context) to examine how important words are actually used.

Bag of Words — Mock Exam Focus

Bag of Words: Vocabulary

A Bag of Words (BoW) representation uses a vocabulary of distinct tokens and records how often each token occurs. Word order is ignored.

Consider:

birds often fly around trees
a plane flies around the world

If the question states that singular/plural forms and different verb forms are treated as different words, then:

  • tree and trees are different
  • fly and flies are different

The vocabulary is:

birds, often, fly, around, trees, a, plane, flies, the, world

Exam Tip

Do not stem or lemmatize words unless the question explicitly tells you to do so.

Bag of Words: Count Vector

Suppose the vocabulary order is:

cars, love, to, race, through, streets, truck, trucks, enjoys, a, vehicle

For the sentence:

trucks race through streets enjoys a vehicle race cars

count each vocabulary term in exactly that order:

cars love to race through streets truck trucks enjoys a vehicle
1 0 0 2 1 1 0 1 1 1 1

The word race appears twice, so its value is 2.

Bag of Words: Same Vector, Different Word Order

A key property of BoW is that word order is ignored. Two sentences can therefore have the same BoW vector even when their word order is different, as long as they contain the same vocabulary words with the same counts.

How to Answer a Vector Question

  1. Use the vocabulary order exactly as given.
  2. Count each word.
  3. Ignore word order in the sentence.
  4. Ignore words that are not in the given vocabulary.
  5. Do not merge singular/plural or verb forms unless instructed.

Interactive Bag of Words

Interactive Word Cloud (demo)