Multiple Outputs

Classification with Multiple Outputs

Optional Topic

Multi-Label Classification

New Concept

• Different from multi-class classification
• In multi-label classification, each input can have multiple labels simultaneously
• Example: Self-driving car image recognition system
• Each image might contain any combination of: cars, buses, pedestrians

Note

This differs from multi-class classification (like handwritten digit recognition) where each image belongs to exactly one class.

Real-World Example

Image 1

• Car: ✓ (yes)
• Bus: ✗ (no)
• Pedestrian: ✓ (yes)

Image 2

• Car: ✗ (no)
• Bus: ✗ (no)
• Pedestrian: ✓ (yes)

Image 3

• Car: ✓ (yes)
• Bus: ✓ (yes)
• Pedestrian: ✗ (no)

Caution

In multi-label classification, the target y is a vector of values (e.g., [1,0,1] indicating car=yes, bus=no, pedestrian=yes), rather than a single value.

Implementation Approaches

Approach 1: Separate Networks

• Treat as three completely separate machine learning problems
• Build separate neural networks for each detection task:
• Network 1: Car detection
• Network 2: Bus detection
• Network 3: Pedestrian detection
• “That’s actually not an unreasonable approach”

Approach 2: Single Network with Multiple Outputs

Preferred

• Train a single neural network to simultaneously detect all object types
• Network architecture:
• Input layer → Hidden layer(s) → Output layer with multiple units
• Example: 3 output units for car, bus, and pedestrian detection
• Each output unit uses a sigmoid activation function
• Output a³ is a vector [a₁³, a₂³, a₃³] representing probabilities for each class

Tip

The sigmoid activation is appropriate here because each output represents a separate binary classification problem (presence/absence of each object type).

Comparison with Multi-Class Classification

Multi-Class Classification

• Each input belongs to exactly one class
• Output y is a single value (even if it can take multiple possible values)
• Uses softmax activation in output layer
• Example: Digit recognition (each image is exactly one digit)

Multi-Label Classification

• Each input can belong to multiple classes simultaneously
• Output y is a vector of binary values
• Uses sigmoid activation for each output unit
• Example: Object detection (image may contain multiple object types)

Note

“Multi-class classification and multi-label classification are sometimes confused with each other” - understanding the distinction helps you choose the right approach for your specific application.

Looking Ahead

• Next topic: Advanced neural network concepts
• Introduction to optimization algorithms better than gradient descent
• These algorithms can help neural networks learn much faster

Multi-label classification allows neural networks to predict multiple outcomes simultaneously for a single input. This approach is particularly valuable in real-world applications like object detection, where multiple objects can appear in the same image. By using sigmoid activations in the output layer, a single network can make independent predictions for each possible label.