note

08 Metacognition And Advanced

Metacognition and Advanced Topics

Prerequisites

Learning Goals

After completing this module, you will be able to:

  1. Understand metacognition as reasoning about reasoning
  2. Implement strategy selection based on problem characteristics
  3. Identify and address knowledge gaps
  4. Apply visuospatial reasoning to problems like Ravens Matrices
  5. Understand design and creativity in AI systems
  6. Synthesize all KBAI concepts into integrated cognitive systems

1. Meta-Reasoning

What is Metacognition?

Metacognition is thinking about thinking—reasoning about one’s own cognitive processes.

In the Cognitive Architecture:

        Percepts
           ↓
    ┌─────────────┐
    │ METACOGNITION│ ← Monitors and adjusts reasoning
    │ (Thinking    │
    │  about       │
    │  thinking)   │
    ├─────────────┤
    │ DELIBERATION│ ← Reasoning, learning, memory
    ├─────────────┤
    │ REACTION    │ ← Direct percept-action mapping
    └─────────────┘
           ↓
        Actions

Metacognition Layer:

Beyond Mistakes: Knowledge Gaps

We covered learning from mistakes earlier (error detection, explanation, correction). Metacognition goes further:

Types of Knowledge Gaps:

1. Missing Knowledge

Don't know: Certain facts or rules
Example: Don't know that birds migrate
Result: Can't answer "Where do swallows go in winter?"
Fix: Acquire new knowledge

2. Incorrect Knowledge

Believe wrong information
Example: "All birds fly" (false: penguins, ostriches)
Result: Incorrect inferences
Fix: Correct existing knowledge (as we learned)

3. Inaccessible Knowledge

Have knowledge but can't retrieve it
Example: Know facts about topic but can't remember
Result: Retrieval failure despite having information
Fix: Improve indexing, add retrieval cues

4. Inefficient Knowledge

Have knowledge but using wrong form
Example: Have cases but need rules; have rules but need cases
Result: Correct but slow/inefficient reasoning
Fix: Transform knowledge representation

5. Insufficient Meta-Knowledge

Don't know WHEN to use what knowledge
Example: Have multiple strategies but don't know which fits problem
Result: Use wrong method, waste time
Fix: Acquire meta-level knowledge about knowledge

Strategy Selection

Problem: Multiple Reasoning Strategies Available

Strategies we’ve learned:

Question: Which strategy for which problem?

Meta-Knowledge for Strategy Selection

Strategy Characteristics:

Generate & Test:
- Good for: Small search spaces, well-defined goals
- Poor for: Large spaces, expensive evaluation
- Cost: O(n) generation + O(n) testing

Means-Ends Analysis:
- Good for: Clear goal, measurable distance, decomposable
- Poor for: Local minima, global constraints
- Cost: Depends on heuristic quality

Case-Based Reasoning:
- Good for: Repetitive problems, past experience relevant
- Poor for: Novel problems, small case library
- Cost: O(n) retrieval + adaptation cost

Planning:
- Good for: Sequential problems, known operators
- Poor for: Uncertain environments, dynamic worlds
- Cost: Exponential in plan length (without heuristics)

Meta-Strategy Selection:

Step 1: Analyze Problem

Characteristics:
- Search space size?
- Goal clarity?
- Past similar cases?
- Constraints type?
- Time available?
- Solution quality needed?

Step 2: Match to Strategy

If search space small AND goal clear:
  → Generate & Test

If past cases exist AND problem similar:
  → Case-Based Reasoning

If sequential actions AND operators known:
  → Planning

If constraints dominate AND local propagation works:
  → Constraint Propagation

If novel problem BUT analogous domain exists:
  → Analogical Reasoning

Step 3: Monitor Execution

While solving:
- Is strategy making progress?
- Is cost acceptable?
- Are assumptions holding?

If not:
- Switch strategy
- Combine strategies
- Adjust parameters

Step 4: Learn from Experience

After solving:
- Was strategy choice good?
- What cues predicted success/failure?
- Update meta-knowledge

Store:
- Problem characteristics
- Strategy used
- Success/failure
- Time/resource cost

The Process of Meta-Reasoning

Monitoring:

Am I making progress?
- Distance to goal decreasing?
- New information being discovered?
- Time/resources being used efficiently?

Evaluation:

Is my current approach working?
- Compare expected vs. actual progress
- Check if assumptions still hold
- Assess if goal still achievable

Regulation:

What should I do differently?
- Continue current strategy?
- Switch to different strategy?
- Adjust parameters (e.g., search depth)?
- Seek external help/information?

Learning:

What did I learn about my own reasoning?
- Which strategies work for which problems?
- What are my strengths/weaknesses?
- How can I improve meta-knowledge?

Meta-Meta-Reasoning?

Question: Can we have meta-meta-reasoning? (Reasoning about reasoning about reasoning?)

Answer: Theoretically yes, practically limited

Meta-Reasoning (Level 2):

Monitors: Deliberation (Level 1)
Reasons about: Which strategy to use, when to switch

Meta-Meta-Reasoning (Level 3):

Monitors: Meta-reasoning (Level 2)
Reasons about: Is meta-reasoning itself working? Should I think less and act more?

Practical Considerations:

Higher levels:
+ More adaptive
+ Better long-term performance
- More computational cost
- Diminishing returns
- Risk of infinite regress

Usually stop at meta-reasoning (Level 2)

2. Visuospatial Reasoning

The Challenge

Visuospatial reasoning involves reasoning with visual and spatial representations, not just symbolic/propositional representations.

Ravens Progressive Matrices: Quintessential visuospatial reasoning task

Problem:

2x2 Matrix:
A | B
--+--
C | ?

Given A, B, C, find ? from options 1-6

Challenge:

Two Views of Reasoning

Propositional/Symbolic View:

Knowledge represented as:
- Symbols (words, predicates)
- Relationships (links, rules)
- Abstract structures

Example:
- On(Block-A, Block-B)
- Shape(Object-X, Circle)
- Above(Y, Z)

Depictive/Analogical View:

Knowledge represented as:
- Images
- Spatial layouts
- Analogical models

Example:
- Actual visual representation of blocks
- Mental image of circle
- Spatial diagram showing positions

KBAI Challenge: Bridge both views

Symbol Grounding Problem

Problem: How do symbols get their meaning?

Example:

Symbol: "Circle"

Propositional representation:
- Shape(X, Circle)
- Closed-curve
- Points-equidistant-from-center

But what does "Circle" MEAN?
How does it connect to actual circles in world?

Symbol Grounding:

Symbols must be grounded in:
- Perceptual experience (seeing circles)
- Motor experience (drawing circles)
- Embodied interaction (tracing circles)

Not just symbol-to-symbol definitions!

For Ravens Matrices:

Need to ground:
- "Triangle" in actual triangular shapes
- "Rotation" in actual rotational transforms
- "Inside" in actual spatial containment

Cannot solve purely symbolically—need perceptual connection

Approach to Ravens Matrices

Hybrid Approach: Propositional + Depictive

Step 1: Visual Processing

Input: Images A, B, C, Options 1-6
Output: Structural descriptions

Extract:
- Objects (shapes with properties)
- Relationships (inside, above, left-of)
- Transformations (rotate, scale, translate)

Step 2: Propositional Representation

Frame representation:
A: {Circle: large, Square: small, Relation: inside}
B: {Circle: large, Square: small, Relation: above}
C: {Triangle: large, Circle: small, Relation: inside}

Step 3: Relationship Detection

A → B transformation:
- Position change: inside → above
- Shapes unchanged
- Sizes unchanged

Pattern: "Move inner object to above outer object"

Step 4: Transfer and Match

Apply pattern to C:
C has: Triangle (large), Circle (small), inside

Expected ?: Triangle (large), Circle (small), above

Match against options:
Option 1: ✗ Wrong shapes
Option 2: ✗ Wrong relationship
Option 3: ✓ Triangle large, Circle small, above
Option 4: ✗ Wrong sizes
...

Select: Option 3

Step 5: Verify Multiple Interpretations

Alternative patterns:
- Horizontal pattern (A→B)?
- Vertical pattern (A→C)?
- Diagonal pattern (A→?)?

Check consistency:
If multiple patterns agree → High confidence
If patterns conflict → Lower confidence, need tie-breaking

Advanced Ravens Strategies

Strategy 1: Affine Transformations

Detect geometric transformations:
- Translation (x, y shifts)
- Rotation (θ degrees)
- Scaling (size changes)
- Reflection (mirroring)

Parameterize and transfer

Strategy 2: Rule Induction

From examples, induce rules:
IF row-1-has-pattern-P
THEN row-2-should-have-pattern-P
THEN row-3-should-have-pattern-P

Test consistency across rows/columns

Strategy 3: Generate-and-Test

For each option:
- Assume it's correct
- Work backward to infer rule
- Check if rule explains all given images
- Select option with most consistent rule

Strategy 4: Fractured Problems

Break complex problem into parts:
- Solve for shape changes
- Solve for size changes
- Solve for position changes
- Solve for count changes

Combine partial solutions

3. Design and Creativity

Design as AI Task

Design: Creating artifacts to meet requirements

Characteristics:

- Ill-defined goals (multiple valid designs)
- Open-ended exploration
- Constraints from multiple sources
- Creativity valued
- Iteration and refinement
- Evaluation subjective

Types of Design:

Defining Creativity

Creativity involves:

1. Novelty

New, original, not mere copy
- But how novel?
- Novel to individual? To community? To world?

2. Value

Useful, elegant, meaningful
- Creativity isn't random novelty
- Must have purpose or beauty

3. Unexpectedness

Surprising, non-obvious
- Not just incremental improvement
- Leap rather than step

4. Appropriate

Fits context and constraints
- Not just bizarre
- Makes sense in domain

Creative = Novel + Valuable + Unexpected + Appropriate

Computational Creativity

Can AI be creative?

Arguments FOR:

- AI can generate novel combinations
- AI can evaluate against criteria
- AI can learn from creative examples
- AI can explore vast design spaces

Examples:
- AI-composed music
- AI-generated art
- AI-designed circuits
- AI-discovered proofs

Arguments AGAINST:

- AI lacks intentionality (no "meaning")
- AI lacks consciousness (no subjective experience)
- AI lacks emotions (no aesthetic feeling)
- AI optimizes rather than innovates

Counterpoint: Humans also inspired by examples, follow patterns, generate-and-test

Pragmatic View:

AI can augment human creativity:
- Generate variations
- Explore design space
- Suggest novel combinations
- Evaluate against constraints

Human + AI collaboration > either alone

Design by Composition

Basic Creative Strategy:

Step 1: Retrieve Components

From memory/case library:
- Past designs
- Design patterns
- Component types
- Successful solutions

Step 2: Compose/Combine

Methods:
- Merge features from different designs
- Substitute components
- Add/remove elements
- Transform parameters

Step 3: Evaluate

Against criteria:
- Functional requirements
- Aesthetic qualities
- Novelty measures
- Feasibility constraints

Step 4: Iterate

If not satisfactory:
- Try different combinations
- Adjust parameters
- Relax constraints
- Seek new components

Design by Analogy

Use analogical reasoning for creative design:

Example: Velcro

Problem: Need fastener that's easy to use, reusable

Observation: Burrs stick to dog fur

Analogy:
- Burrs (hooks) ← → Velcro hooks
- Fur (loops) ← → Velcro loops
- Sticking ← → Fastening

Transfer:
Create artificial burr-and-fur system
→ Velcro invented!

Process:

1. Define problem/need
2. Search for analogous situations (often in nature)
3. Map structure from source to target
4. Transfer and adapt solution
5. Prototype and test

Design by Transformation

Systematic creativity through transformations:

SCAMPER Framework:

S - Substitute: Replace component
C - Combine: Merge elements
A - Adapt: Adjust for new context
M - Modify: Change properties
P - Put to other use: New purpose
E - Eliminate: Remove component
R - Reverse: Invert relationship

Example: Designing New Chair

Start: Traditional 4-leg chair

Substitute: Metal legs → Molded plastic single piece
Combine: Chair + table → Desk-chair combo
Adapt: Office chair → Gaming chair (lumbar support, headrest)
Modify: Rigid back → Flexible mesh
Put to other use: Chair → Stepladder (when turned over)
Eliminate: Remove armrests → Stackable chairs
Reverse: Sitting → Kneeling (kneeling chair)

Each transformation → New design variation

4. Systems Thinking and Integration

Connections Across KBAI

Core Principle: Everything Connects

Knowledge Representations ←→ Reasoning Methods
         ↕                        ↕
    Learning Methods ←→ Memory Organization
         ↕                        ↕
    Meta-Reasoning ←─────────→ Problem Types

Example Integration: Ravens Matrices Project

Uses:

  1. Semantic Networks (fundamentals) - Represent figures
  2. Frames (language/common sense) - Structured object descriptions
  3. Generate & Test (core reasoning) - Try options
  4. Analogical Reasoning (advanced) - A:B :: C:?
  5. Case-Based Reasoning (learning) - Remember similar problems
  6. Production Systems (core reasoning) - Rule-based transformations
  7. Constraint Propagation (applied) - Consistent interpretations
  8. Metacognition (advanced) - Strategy selection
  9. Visuospatial Reasoning (advanced) - Visual processing

Single Project Integrates Entire Course!

Principles Underlying KBAI

Seven Fundamental Principles:

1. Knowledge Representations are Central

Right representation makes problem easier
Different representations enable different reasoning
Multiple representations often beneficial

2. Reasoning, Learning, Memory are Integrated

Not separate modules but unified system
Reasoning drives learning
Learning fills memory
Memory enables reasoning

3. Cognitive Architectures Provide Structure

Architecture + Content = Behavior
Fixed architecture allows flexible behavior through knowledge
Separates mechanism from knowledge

4. Analogy Enables Transfer

Map from known to unknown
Surface → Structural → Pragmatic similarity
Cross-domain transfer most creative

5. Generate-and-Test Underlies Many Methods

Generate candidates, test against criteria
Balance generator/tester intelligence
Ubiquitous pattern in AI

6. Meta-Reasoning Enables Adaptation

Thinking about thinking
Strategy selection
Error detection and correction
Knowledge gap identification

7. Human Cognition Informs AI Design

Cognitive psychology → AI architectures
AI systems → Cognitive models
Bidirectional relationship
Goal: Human-level, human-like AI

Future Directions

Current Research in KBAI:

1. Commonsense Reasoning at Scale

Challenge: Capture vast human commonsense
Approaches:
- Large knowledge bases (Cyc, ConceptNet)
- Learning from text (language models)
- Crowdsourcing (human computation)

2. Integrated Cognitive Architectures

Examples: SOAR, ACT-R, ICARUS, SIGMA
Goal: Single architecture for all cognition
Challenges: Integration, scaling, learning

3. Computational Creativity

AI that generates novel, valuable artifacts
Domains: Music, art, design, science
Challenge: Evaluation of creativity

4. Explanation and Transparency

AI that explains its reasoning
Important for trust, debugging, learning
KBAI's structured knowledge helps explainability

5. Hybrid Systems

Combine:
- Symbolic AI (KBAI) + Statistical ML
- Top-down reasoning + Bottom-up learning
- Propositional + Depictive representations
Best of both approaches

Summary

Key Takeaways

  1. Metacognition is reasoning about reasoning—monitoring deliberation, selecting strategies, detecting knowledge gaps, and adapting approaches. Goes beyond error correction to proactive self-improvement.

  2. Strategy Selection requires meta-knowledge about when to use which reasoning method. Analyze problem characteristics, match to strategy strengths, monitor execution, learn from experience.

  3. Knowledge Gaps come in five types: missing (don’t have), incorrect (wrong info), inaccessible (can’t retrieve), inefficient (wrong form), insufficient meta-knowledge (don’t know when/how to use).

  4. Visuospatial Reasoning bridges symbolic and depictive representations. Ravens Matrices require extracting structure from images, representing relationally, detecting patterns, transferring to novel cases.

  5. Design and Creativity involve novelty + value + unexpectedness + appropriateness. AI can augment creativity through composition, analogy, and systematic transformation (SCAMPER).

  6. Integration is key: KBAI’s power comes from combining representations (semantic networks, frames, production rules), reasoning methods (generate-test, CBR, analogy), learning mechanisms, and metacognition into unified cognitive systems.

  7. Seven Principles: Knowledge representations central, reasoning-learning-memory integrated, cognitive architectures provide structure, analogy enables transfer, generate-test ubiquitous, meta-reasoning enables adaptation, human cognition informs AI.

Essential Principles

Course Synthesis

Module 1 (Fundamentals)
  ↓ Provides foundation
Module 2 (Core Reasoning)
  ↓ Enables problem-solving
Module 3 (Learning)
  ↓ Enables improvement
Module 4 (Logic & Planning)
  ↓ Formalizes reasoning
Module 5 (Language & Common Sense)
  ↓ Enables understanding
Module 6 (Advanced Reasoning)
  ↓ Enables transfer
Module 7 (Applied)
  ↓ Demonstrates utility
Module 8 (Metacognition)
  ↓ Completes the cycle
Integrated Cognitive System ✓

Ravens Project as Capstone:

Final Reflection

What is KBAI?

Knowledge-Based AI is the study of artificial intelligence systems that:

  1. Use structured knowledge representations to explicitly capture what they know
  2. Employ deliberate reasoning methods to solve problems using that knowledge
  3. Learn from experience to improve performance over time
  4. Integrate reasoning, learning, and memory into unified cognitive architectures
  5. Exhibit meta-reasoning to monitor and adapt their own processes
  6. Draw inspiration from human cognition to achieve human-level, human-like intelligence

Why KBAI Matters:

The Vision:

Create AI systems that don’t just recognize patterns or optimize objectives, but that understand situations, reason about problems, learn from experience, explain their thinking, and adapt to new challenges—in short, AI systems that think.

See Also

The ultimate goal of KBAI: Create artificial intelligence systems that achieve human-level intelligence through human-like reasoning, learning, and memory—systems that truly think.

Thank you for learning KBAI. Now go build intelligent agents!