# UAS — Artificial Intelligence (CSB) — Model Answers

> Final Exam · Odd Semester 2024/2025 · Universitas Gadjah Mada
> Answers to `UAS - AI - 2025_1.md` (CSB paper) — the same answers apply to the identical CSA paper (`UAS AI.md`) and the Indonesian KOMB counterpart.

| Field | Value |
|---|---|
| **Course** | Artificial Intelligence (CSB) |
| **Exam type** | Final Exam (UAS) |
| **Semester** | Odd Semester 2024/2025 |
| **Day, Date** | Monday, 2 December 2024 |
| **Time** | 100 minutes |
| **Exam mode** | Open one A4/F4 sheet |

**Institution:** Program Studi S1 Ilmu Komputer, FMIPA, Universitas Gadjah Mada

---

## Question 1 — Expert System Architecture (PLO 2 & 3 – CO 4, 7%)

### A. Role of the knowledge base, inference engine, and user interface

An **expert system** is an AI program that emulates the decision-making ability of a human specialist in a narrow domain. It captures scarce human expertise in machine-readable form and applies it consistently, at scale, and with explanations. Three components are essential:

**1. Knowledge Base (KB) — the "library" of expertise**
- A **passive repository** that stores everything the system knows about the domain:
  - **Facts** — ground truths of the domain (e.g., *"fever is a symptom of infection"*).
  - **Rules** — heuristic IF–THEN statements elicited from experts (e.g., *IF fever AND cough AND fatigue THEN influenza is likely (0.7)*).
  - **Relationships / frames / ontologies** — how concepts connect (disease–symptom, cause–effect).
- Built through **knowledge acquisition (knowledge engineering)**: interviews, protocol analysis, and case reviews with domain experts.
- **The system's ceiling**: the KB's *completeness and accuracy* directly bound the quality of the advice. Wrong or stale knowledge ⇒ wrong conclusions, no matter how good the reasoning. It is deliberately separated from reasoning logic so that the same shell can host different domains.

**2. Inference Engine (IE) — the "brain"**
- A domain-independent reasoning program that **applies the knowledge to the current problem**. Its tasks:
  - **Pattern matching** — compare rule antecedents (IF parts) against facts in **working memory**.
  - **Conflict resolution** — when several rules match, pick by specificity, recency, or explicit priority.
  - **Rule execution** — fire the chosen rule, asserting new facts or actions.
  - **Control/search** — iterate until a goal is reached or no more rules fire.
- Two reasoning strategies:
  - **Forward chaining (data-driven):** start from known facts, derive conclusions. Good for monitoring/triage.
  - **Backward chaining (goal-driven):** start from a hypothesis (e.g., *"patient has pneumonia"*), work backwards asking for evidence that would prove it. Good for diagnosis.
- Separating the IE from the KB is the architectural essence of an expert system: knowledge about *what* is in the KB, logic about *how* is in the engine.

**3. User Interface (UI) — the bridge to the user**
- Lets a **non-expert** user: enter observations/symptoms, answer the system's questions, and receive recommendations.
- Presents results, and (in most systems) supports the **explanation facility**: *"Why did you ask me that?"* (shows the goal being pursued) and *"How did you reach this conclusion?"* (shows the chain of fired rules). This transparency is what makes expert systems trustworthy in high-stakes domains like medicine and finance.

**Worked example — medical diagnosis (MYCIN-style):**

| Step | What happens | Component |
|---|---|---|
| 1 | Doctor enters: *fever 39°C, productive cough, crackles heard* | UI → working memory |
| 2 | Engine matches facts against rules: `R12: IF fever>38.5 AND cough AND crackles THEN pneumonia-suspected (0.8)` | Inference engine ↔ KB |
| 3 | Rule fires; system needs more evidence, so it **asks** via the UI: *"Is the chest X-ray infiltrate present?"* | IE → UI (backward chaining) |
| 4 | User answers *yes*; `R15: IF pneumonia-suspected AND X-ray-infiltrate THEN diagnosis = pneumonia (0.95)` fires | IE ↔ working memory |
| 5 | System outputs: *"Diagnosis: bacterial pneumonia (certainty 0.95). Recommend culture + amoxicillin."* | UI |
| 6 | User asks *HOW?* — system lists R12, R15 and the facts used | Explanation facility (UI + IE trace) |

The three components form a loop: **UI feeds facts in → IE reasons over the KB → conclusions and explanations come back out through the UI.** The KB holds *what* is known, the IE decides *how* to use it, and the UI makes the whole interaction possible for a non-expert.

### B. Challenges in maintaining the knowledge base over time

1. **Knowledge acquisition bottleneck** — experts are scarce, expensive, and often cannot articulate their tacit knowledge; eliciting and codifying new rules is slow and labour-intensive.
2. **Knowledge currency (obsolescence)** — domains evolve: new diseases, drugs, regulations, fraud patterns. An unmaintained KB silently gives outdated advice (e.g., recommending a withdrawn drug). Rules need continuous review cycles.
3. **Consistency and conflicts** — as hundreds/thousands of rules accumulate, newly added rules can contradict old ones (R5 says "risky", R80 says "safe"). Contradiction detection and rule-priority management become hard.
4. **Verification & validation (V&V)** — every change can break previously correct behaviour; KBs must be regression-tested against a library of solved cases. Unlike ordinary software, errors in encoded knowledge are *semantic*, not just syntactic, and are hard to spot automatically.
5. **Scalability & performance** — inference cost grows with rule count (the matching problem); large KBs become slow and un-auditable.
6. **No automatic learning** — classical expert systems do not learn from experience; every improvement requires a human edit. This contrasts with ML systems, and motivates modern **hybrid neuro-symbolic** designs.
7. **Loss of the expert** — when the domain expert retires or leaves, tacit knowledge behind the rules is lost, making future maintenance harder.
8. **Change management & audit** — in regulated domains (medicine, finance), updates must be versioned, approved, and traceable; an audit trail of who changed which rule and when is required.
9. **Domain drift in the environment** — the world changes faster than the KB (new fraud typologies, new guidelines), so a once-valid system degrades (concept drift).

**Mitigations:** a dedicated **knowledge-acquisition module**, expert-in-the-loop update workflows, automated rule-consistency checkers, case-based regression testing, periodic domain review boards, and hybrid architectures that let ML propose new rules for expert approval.

---

## Question 2 — Natural Language Processing (PLO 2 & 3 – CO 5, 8%)

### A. Importance of text preprocessing in NLP tasks

Text preprocessing is the **cleaning, normalization, and standardization of raw text** before it reaches a model. It matters because:

1. **Garbage in, garbage out.** Real-world text (tweets, chats, reviews) is messy: typos, slang, elongations ("sooo goood!!!"), URLs, HTML, emojis, hashtags, mixed languages. A model fed this raw sees noise, not language.
2. **Better signal-to-noise ratio.** Removing/normalizing irrelevant variation lets the model focus on genuinely discriminative content — empirical multilingual studies report ≈7% F1 improvement on social-media sentiment/emotion tasks from proper preprocessing alone.
3. **Consistent vocabulary → less sparsity.** Lowercasing, stemming/lemmatization, and slang normalization map many surface forms to one token, shrinking the vocabulary, reducing the curse of dimensionality, and improving generalization to unseen data.
4. **Task-specific information must be *deliberately* preserved or destroyed.** For grammar parsing, punctuation is structure; for emotion recognition, an emoji can flip the label of a sentence ("Great job 😂" is sarcastic-negative). Preprocessing is where these decisions are made — deleting the wrong thing destroys the very signal the task needs.
5. **Fair comparison and reproducibility.** A defined pipeline makes experiments repeatable and models comparable.
6. **Efficiency.** Smaller, cleaner vocabularies train faster and need less memory.

In short: modern models are only as good as the tokens they receive; preprocessing converts chaotic human text into a representation a model can actually learn from.

### B. Preprocessing pipeline for an emotion recognition system

An emotion recognition system (e.g., classifying posts into *anger, fear, joy, sadness, …*) typically consumes noisy social-media text. Pipeline steps:

**Step 0 — Data collection & language handling**
- Collect texts with emotion labels (self-reported hashtags, survey annotation, or existing corpora like GoEmotions).
- Run **language identification** and handle **code-switching** (mixed-language posts), e.g., keep mixed text and use a multilingual model.

**Step 1 — Noise cleaning (careful, emotion-aware)**
- Strip HTML, boilerplate, bot spam; replace URLs with a `<URL>` token (do not just delete — a link can signal sharing behaviour).
- Replace @mentions with `<USER>`; keep hashtags but **segment** them into words (`#feelingblessed` → "feeling blessed") — hashtags carry emotion labels.
- Keep exclamation/question marks and quoting — they are prosodic emotion cues.

**Step 2 — Lexical normalization (noisy-data handling)**
- **Lowercasing** (optionally keep an *all-caps flag* — SHOUTING is an anger/joy cue).
- **Elongation collapsing:** "sooooo" → "soo"/"so" plus an `<ELONG>` feature flag; repetition encodes intensity.
- **Slang/abbreviation dictionary expansion:** "lol", "wkwk" (Indonesian laughter), "ngab", "gt" → full forms. Maintain a domain slang lexicon.
- **Spelling correction** (SymSpell / edit-distance-1 to a vocabulary), kept conservative so names/neologisms survive.
- Deduplicate near-identical posts to avoid label bias.

**Step 3 — Emoji & emoticon handling (critical for emotion)**
- **Never delete emojis.** Convert them to textual descriptions or emotion tokens: `❤️` → `<heart>`/"love", `😭` → `crying` (note ambiguity: tears-of-joy vs grief — resolved by context, i.e., the model).
- Optionally map emoticons/emojis to **emotion-lexicon polarity scores** as extra features.

**Step 4 — Tokenization (the hard part in morphologically rich languages)**
- Problem: in morphologically rich languages (Turkish, Finnish, Hungarian, Arabic; Indonesian also has productive affixation, e.g., *memperbincangkan* = *mem- + per- + bincang + -kan*), one word encodes many morphemes. Word-level tokenization explodes the vocabulary and floods the model with out-of-vocabulary (OOV) variants of the same root.
- Solutions, in increasing sophistication:
  1. **Morphological analyzers / lemmatizers** (Zemberek for Turkish, MorphInd for Indonesian) that split words into root + affixes or map to the lemma;
  2. **Subword tokenization — the modern default:** WordPiece / Byte-Pair Encoding / SentencePiece (Unigram LM). Subwords learn frequent character units ("bincang", "mem", "-kan"), so unseen inflected forms decompose into known pieces → OOV is nearly eliminated;
  3. **Byte-level BPE** additionally handles typos, mixed scripts, and emoji bytes gracefully — ideal for noisy user-generated content.
- Transformers ( IndoBERT, mBERT, XLM-R) come with their own subword tokenizer — the pipeline should match the model's tokenizer rather than fight it.

**Step 5 — Stop-word removal (with caution)**
- Standard stop lists remove "not", "no", "very" — but **negation flips emotion** ("not happy") and intensifiers scale it ("very sad"). Use an emotion-aware stop list or skip stop-word removal entirely with subword models.

**Step 6 — Stemming / lemmatization**
- Prefer **lemmatization** (context-aware, dictionary-correct) over crude stemming ("studies"→"studi"); for rich morphologies use a proper morphological analyzer rather than suffix chopping.

**Step 7 — Label & data-level steps**
- Align labels to a taxonomy (Ekman's 6, Plutchik's 8, or fine-grained 27).
- Handle **class imbalance** in emotion labels (joy usually dominates) with class weights or stratified sampling; report **macro-F1**.
- Optional augmentation: back-translation, emoji synonym swapping.

**Step 8 — Representation & model**
- Baseline: TF-IDF + linear SVM/logistic regression (fast, interpretable).
- Strong model: fine-tuned multilingual transformer (XLM-R / mBERT / IndoBERT) with subword tokenization — empirically these dominate on noisy multilingual emotion data (XLM-R ≈ 90% F1 in recent studies vs ≈ 84–87% for mBERT/T5).

**Summary of the two named challenges:**
- *Noisy data* → normalization dictionary + conservative spell-correction + elongation flags + emoji-to-token conversion + byte-level subword tokenizer.
- *Tokenization in morphologically rich languages* → subword tokenization (BPE/SentencePiece) and/or morphological analyzers + lemmatization instead of word-level vocabularies, keeping OOV and sparsity under control.

### C. TF-IDF — and why it will not perform well

**The representation.** TF-IDF (Term Frequency – Inverse Document Frequency) represents a document as a vector with one dimension per vocabulary term:

- **Term Frequency** — how often term *t* occurs in document *d* (raw count, or normalized: `tf(t,d) = count(t,d) / len(d)`).
- **Inverse Document Frequency** — how *rare/distinctive* *t* is across the corpus of *N* documents:
  `idf(t) = log( N / df(t) )` where `df(t)` = number of documents containing *t*.
- **TF-IDF weight:** `tfidf(t,d) = tf(t,d) × idf(t)`.

**Interpretation:** a term gets a high weight when it is frequent *in this document* but rare *across the corpus* — i.e., distinctive of this document. Common words ("the", "and") are crushed by IDF. It is a purely **statistical, count-based, bag-of-words** method: no training, no learning, words are independent dimensions.

**Why TF-IDF will not perform well (especially on emotion/semantic tasks):**

1. **No notion of meaning/semantics.** "I love it" and "I adore it" share no terms → their vectors are near-orthogonal. Synonyms are invisible; TF-IDF cannot capture similarity of meaning.
2. **Bag-of-words destroys word order and structure.** "great camera, terrible battery" vs "terrible camera, great battery" produce essentially the same vector. Negation ("not good"), sarcasm, and scope — all crucial for emotion — are lost.
3. **Sparse, high-dimensional vectors.** One dimension per vocabulary term (10⁴–10⁶) makes vectors mostly zeros: memory-hungry, distance measures degrade (curse of dimensionality), and models generalize poorly.
4. **Out-of-vocabulary failure.** A word unseen in training has no dimension → weight 0. On noisy user text (typos, slang, new coinages) this is fatal, and TF-IDF has no subword/character fallback.
5. **Context-blind (static).** "Bank" (river) and "bank" (money) get the *same* column; each word has one fixed representation regardless of sentence. TF-IDF cannot represent polysemy, unlike contextual embeddings.
6. **Rarity ≠ importance on noisy text.** IDF assumes rare terms are informative — so typos and one-off slang receive *inflated* weights, exactly the opposite of what is needed on social-media data.
7. **No relational/relational reasoning.** Word similarities, analogies, or composition cannot be expressed; every term is an isolated axis.

**What works instead:**
- **Static dense embeddings** — Word2Vec / GloVe: learned, low-dimensional, semantically smooth ("king − man + woman ≈ queen"); **fastText** adds subword units, fixing the OOV problem.
- **Contextual embeddings** — BERT/IndoBERT/XLM-R: a word's vector depends on the full sentence (polysemy handled), pre-trained on huge corpora, fine-tunable with little labelled data.
- TF-IDF remains a reasonable **baseline** for keyword search/IR and small-data text classification with linear models, but for emotion recognition it systematically underperforms because the task is *semantic*, not *lexical*.

---

## Question 3 — Pattern Recognition & ML: Customer Fraud-Risk Classification Proposal (PLO 4 & 5 – CO 6, 10%)

**Problem recap.** Binary classification of e-commerce customers into **High-Risk / Low-Risk** for fraud using numerical behavioural features (transaction amounts, purchase frequency, account age, refund rate, …). Two error types matter: **false positives** (legitimate customers flagged → friction, lost trust/revenue) and **false negatives** (fraud missed → direct losses). Fraud datasets are typically **severely imbalanced** (fraud ≪ legitimate).

### 1. Data Exploration and Preprocessing

**1.1 Exploratory data analysis (EDA) first**
- Check dataset size, feature types, **class ratio** (expect heavy imbalance → this drives all later choices).
- Plot distributions/histograms per class, correlation heatmaps (amount ↔ refund rate?), temporal patterns (fraud tactics drift over time).
- Hunt for **leakage**: drop any feature that is only knowable *after* the fraud outcome (e.g., "was charged back") — otherwise evaluation is meaningless.
- Remove exact duplicates; define the unit of analysis (per-customer aggregate vs per-transaction).

**1.2 Missing values**
- **Diagnose the mechanism** (MCAR / MAR / MNAR) with missingness counts/plots. In fraud data, missingness is often *informative* (e.g., absent billing/phone data correlates with fake accounts) — treat it as signal, not noise.
- **Numeric imputation:** median (robust to skew) or KNN / iterative (MICE) imputation for inter-correlated features.
- **Add missing-indicator flags** (`was_amount_missing`) so the model can exploit informative absence.
- **Avoid row deletion** — it throws away rare positive cases. Fit imputers on **training folds only** to avoid leakage.

**1.3 Outliers**
- Key insight: **in fraud detection, outliers may be the signal** (unusually large amounts, extreme refund rates). Never blindly delete.
- **Detect:** IQR rule (outside 1.5×IQR), z-score (|z| > 3), robust Mahalanobis distance, or Isolation Forest for multivariate anomalies.
- **Treat:**
  - **Winsorize / cap** at percentiles (e.g., 1st/99th) so extremes stay but don't dominate;
  - **Log-transform** heavy-tailed features so extremes compress naturally;
  - **Keep an `is_extreme` flag** as an explicit feature;
  - Use **tree-based models**, which are intrinsically robust to monotone extremes.
  - Only *delete* an outlier when it is a provable data-entry error (e.g., negative account age).

**1.4 Skewed distributions**
- **log1p transform** for monetary amounts (classic in fraud data);
- **Box-Cox / Yeo-Johnson** for other skewed features (Yeo-Johnson handles zeros/negatives);
- **Quantile (rank) transform** to Gaussian as a stronger option;
- **Scaling:** RobustScaler (median/IQR-based) for distance-based models (LR, SVM, kNN); tree ensembles need **no scaling**.

**1.5 Feature selection / dimensionality reduction (interpretability + performance)**
- **Filter methods:** variance threshold, correlation pruning (drop one of highly-correlated pairs), mutual information / ANOVA F-score vs the target — cheap, model-free.
- **Wrapper methods:** Recursive Feature Elimination (RFE) — accurate but expensive.
- **Embedded methods:** L1-regularized logistic regression (sparse coefficients), tree/boosting **feature importances** refined with **SHAP values** (also yields per-prediction explanations reviewers can read).
- **Dimensionality reduction:** PCA decorrelates and compresses (e.g., the famous credit-card fraud benchmark uses 28 PCA components) — but note the **interpretability trade-off**: "PC7 = 0.3·amount − 0.2·refund_rate" means nothing to a fraud analyst. For this task, prefer **SHAP-guided feature selection on original features** over PCA whenever interpretability is a stated requirement; use PCA only if features are many and strongly redundant.

### 2. Model Selection and Training

**2.1 Candidate comparison**

| Model | Strengths | Weaknesses for this task | Verdict |
|---|---|---|---|
| **Logistic Regression** | Interpretable coefficients, calibrated-ish probabilities, fast | Linear boundary; needs scaling, log-transforms, careful feature engineering | **Baseline / benchmark** |
| **Decision Tree** | Fully interpretable rules | High variance, unstable | Use only as an explanation aid |
| **Random Forest** | Robust, non-linear, insensitive to scaling/outliers, low tuning effort, class-weighted | Larger memory; less accurate than boosting on tabular | Strong mid-complexity option |
| **Gradient Boosting (XGBoost / LightGBM / CatBoost)** | **State-of-the-art on tabular data**; non-linear interactions; handles skew; `scale_pos_weight` for imbalance; early stopping; SHAP-compatible | Needs tuning; overfits if unchecked | **Primary recommendation** |
| **SVM (RBF)** | Strong boundaries on small data | Very slow at scale; needs scaling; awkward probability outputs | Not preferred |
| **k-NN** | Simple | Collapses under imbalance + high dimensions | Avoid |
| **MLP / deep net** | Flexible | Needs far more data; least interpretable; overkill for a small numeric feature set | Not preferred |

**2.2 Training protocol**
- **Split:** stratified train/validation/test (e.g., 70/15/15) preserving the class ratio; use a **time-based split** if fraud behaviour drifts (train on past, test on future).
- **Cross-validation:** stratified k-fold on the training portion for all model selection; keep the test set untouched until the very end.
- **Probability calibration:** Platt scaling or isotonic regression on held-out predictions — thresholds and expected-cost decisions depend on honest probabilities.

**2.3 Hyperparameter tuning**
- **Search strategies:** Grid/Random search for small spaces; **Bayesian optimization (Optuna)** for efficiency; **early stopping** for boosting (stop when validation PR-AUC stops improving).
- **Representative parameters:**
  - *Logistic Regression:* `C`, penalty (L1/L2), `class_weight='balanced'`.
  - *XGBoost/LightGBM:* `max_depth` / `num_leaves`, `learning_rate` (0.01–0.2), `n_estimators` (with early stopping), `subsample`, `colsample_bytree`, `min_child_weight`, `scale_pos_weight` (≈ n_negative/n_positive), `reg_lambda`.
  - *Random Forest:* `n_estimators`, `max_depth`, `min_samples_leaf`, `max_features`, `class_weight`.
- **Tuning objective:** optimize a **threshold-independent, imbalance-aware metric — PR-AUC (average precision)** or log-loss, *not* accuracy. Tune the **decision threshold separately** on validation data after the model is fixed.
- Use **nested CV** if an unbiased performance estimate of the whole pipeline (tuning included) is required.

### 3. Evaluation and Class Imbalance

**3.1 Metrics — choices and justification**

First, what *not* to use: **accuracy** fails the "accuracy paradox" — if 1% of customers are fraudulent, a model that predicts "Low-Risk" for everyone scores 99% while catching zero fraud. ROC-AUC is also treacherous: recent research (2026, *Technologies*/MDPI, on a 0.17%-positive credit-card fraud benchmark) shows ROC-AUC saturates at 0.90–0.99 even for classifiers with thousands of false positives, because the vast negative class dominates the false-positive-rate denominator.

Recommended metric bundle:

| Metric | What it captures | Why it fits this task |
|---|---|---|
| **Precision** (TP/(TP+FP)) | Cost of **false positives** | Each FP = a legitimate customer wrongly blocked/reviewed → friction, support cost, churn |
| **Recall** (TP/(TP+FN)) | Cost of **false negatives** | Each FN = a fraudster waved through → direct monetary loss |
| **F2-score** (β=2) | Recall-weighted F | Use when missing fraud is costlier than flagging a legitimate user; the Fβ family lets β encode the real cost ratio (β ∈ [2,4]) |
| **PR-AUC (average precision)** | Threshold-free ranking quality; baseline = prevalence | Strictly more informative than ROC-AUC under severe imbalance (Davis & Goadrich; Saito & Rehmsmeier) — the right objective for model selection |
| **MCC** (Matthews correlation) | Symmetric, prevalence-robust single number over the whole confusion matrix | Recommended primary metric alongside F2 in the 2026 study; stays honest when classes are tiny |
| **Confusion matrix + cost-based expected loss** | Actual business impact | Put currency on FP (review cost, customer lifetime value lost) and FN (average fraud loss) and report **expected monetary loss** at the chosen threshold |
| **Recall @ fixed alert budget** | Operational reality | Analysts can review only *k* accounts/day — report recall among the top-k riskiest |

Reporting recipe: **PR-AUC + MCC + F2** as the core bundle, precision/recall and expected cost at the operating threshold, ROC-AUC only as a caveat-edged supplementary number.

**3.2 Handling class imbalance**

1. **Algorithm level (preferred first):**
   - **Class weighting / cost-sensitive learning** — `class_weight='balanced'`, XGBoost `scale_pos_weight`, or a custom cost matrix in the loss. Directly encodes that FN costs more than FP.
   - **Focal loss** (down-weights easy majority examples) in neural/boosting variants.
2. **Data level (resampling):**
   - **Oversampling:** random (duplicates → overfitting risk) or **SMOTE** (synthetic interpolation between minority neighbours), plus variants: **Borderline-SMOTE, SVM-SMOTE, ADASYN** (focus synthesis on the decision boundary), **SMOTE + Tomek/ENN** (clean overlap).
   - **Undersampling** the majority (risk: discards information) or ensemble variants (**EasyEnsemble, Balanced Random Forest**) that train many learners on balanced sub-samples.
   - **Critical rule:** apply any resampling **only inside training folds** (never before splitting) — otherwise synthetic points leak into validation/test and inflate scores.
3. **Decision level:**
   - **Threshold tuning:** after training, choose the cut-off on validation data to satisfy a business constraint (e.g., "recall ≥ 0.85 at precision ≥ 0.30" or minimum expected cost). Resampling and threshold shifting are theoretically equivalent ways of moving the decision boundary (Elkan's cost-sensitive result), and threshold tuning is the more controlled route.
4. **Label level (often the highest-leverage fix):**
   - Fraud labels arrive **late** (chargebacks take weeks): build a delayed-feedback retraining loop and let analysts' investigation outcomes relabel the queue (**active learning**).
5. **Modern empirical caveat:** with strong gradient-boosting models, aggressive SMOTE/resampling frequently adds *nothing* and can wreck probability calibration — best practice is to first train a strong model on the **original distribution**, tune with log-loss/PR-AUC, calibrate, and then test resampling as an *experiment* that must beat the baseline in a fair comparison.

**3.3 Deployment & monitoring (completing the proposal)**
- Serve calibrated **risk probabilities**, not just labels, so operations can flex the threshold.
- Attach **SHAP explanations** to each flag so analysts see *why* ("refund-rate 9× normal, account 3 days old").
- Monitor **input drift and precision/recall decay** in production; retrain on fresh (delayed) labels on a schedule.
- Keep a **human-in-the-loop review queue** for borderline scores; feed outcomes back as new training data.

**Final recommendation (one paragraph).** Preprocess with median/KNN imputation + missing-indicators, winsorize or log1p-transform skewed features, select features via mutual information + SHAP rather than PCA; train a **class-weighted gradient-boosting model (XGBoost/LightGBM)** against a logistic-regression baseline; tune with Bayesian search + early stopping using **PR-AUC**; calibrate probabilities and set the threshold from the FP/FN cost ratio; report **MCC, F2, PR-AUC, and expected monetary cost**, treat resampling as an optional experiment inside CV folds only, and monitor for drift with a delayed-label retraining loop.

---

## References (web sources consulted)

1. Imani et al. (2026). *Why ROC-AUC Is Misleading for Highly Imbalanced Data: In-Depth Evaluation of MCC, F2-Score, H-Measure, and AUC-Based Metrics Across Diverse Classifiers.* Technologies (MDPI), 14(1), 54 — basis for the metric bundle (MCC + F2 + PR-AUC), ROC-AUC saturation on 0.17%-positive fraud data, and resampling-only-in-training-folds protocol.
2. GeeksforGeeks / OER Commons lessons on **Expert Systems in AI** (components, forward/backward chaining, MYCIN/DENDRAL/XCON, explanation module).
3. Almalki (2025). *Sentiment Analysis and Emotion Detection Using Transformer Models in Multilingual Social Media Data.* IJACSA 16(3) — preprocessing (emoji normalization, code-switching handling) improving F1 ≈7%; XLM-R vs mBERT/T5 on multilingual emotion data.
4. TrainInData / imbalanced-learn guidance on imbalanced data: strong-model-first, calibration, threshold tuning, resampling ≡ cost-sensitive threshold shift (Elkan 2001).
5. Classic citations referenced: Chawla et al. (2002) SMOTE; Davis & Goadrich (2006), Saito & Rehmsmeier (2015) on PR vs ROC; Chicco & Jurman (2020) on MCC; Dal Pozzolo et al. (2018) on realistic credit-card fraud modelling.

---

*Model answers prepared for revision; the CSA and KOMB copies of this exam carry identical questions, so these answers cover all three papers.*