docs/CACHING_GUIDE.md

🚀 Caching System - Instant Playback for Examples

Overview

The Science Storyteller now includes an intelligent caching system that provides instant playback for pre-generated example topics. This dramatically improves user experience while maintaining full transparency.

How It Works

For Users

• ⚡ Click any example topic → Instant playback (< 1 second)
• ✍️ Enter custom topic → Fresh generation (~1.5 minutes)
• 🔍 Transparency → Status clearly shows if result is cached or freshly generated

For Developers

Cache Architecture

cache/
├── metadata.json                    # Topic → files mapping
├── {hash}_summary.txt              # Cached summaries
├── {hash}_script.txt               # Cached scripts
├── {hash}_paper.txt                # Cached paper info
└── (audio files stored in assets/audio/)

Cache Key Generation

• Topics are normalized (lowercase, stripped)
• MD5 hash ensures consistent lookup
• Example: "AlphaFold 3" → a1b2c3d4e5f6...

Implementation Details

Files Added/Modified

1. utils/cache_manager.py (NEW)

   • CacheManager class handles all caching logic
   • Methods: get(), set(), has(), list_cached()
   • Stores metadata in cache/metadata.json

2. app.py (MODIFIED)

   • Cache check at start of process_topic()
   • Instant return if topic is cached
   • Auto-cache new generations for future use
   • Status messages show cache/fresh indicator

3. generate_cache.py (NEW)

   • Pre-generates all 8 example topics
   • Runs automatically to populate cache
   • Takes ~15-20 minutes for all examples

4. test_cache.py (NEW)

   • Tests cache hit/miss scenarios
   • Measures speedup (typically 90s → <1s)
   • Validates transparency indicators

UI Changes

Header:

⚡ Click examples below for instant playback (pre-generated) 
| Custom topics take ~1.5 min to generate

Status Messages:

• Cached: ⚡ Instant result from cache! ... [Pre-generated example]
• Fresh: ✅ Success! Generated ... [Freshly generated]

Usage

Generate Pre-cached Examples

# Run once to pre-generate all 8 examples
uv run --env-file .env python generate_cache.py

This will:

• Generate podcasts for all example topics
• Store in cache/ directory
• Enable instant playback for examples

Test Cache Performance

# Test cache functionality
uv run --env-file .env python test_cache.py

Expected results:

• Fresh generation: ~90-120 seconds
• Cached retrieval: <1 second
• Speedup: ~100x faster

Check Cached Topics

from utils.cache_manager import CacheManager

cache = CacheManager()
print(cache.list_cached())
# Output: ['AlphaFold 3', 'transformer neural network', ...]

Performance Impact

Scenario	Without Cache	With Cache	Improvement
Example topics	~90 seconds	<1 second	~100x faster ⚡
Custom topics	~90 seconds	~90 seconds	Same (first time)
Repeat custom	~90 seconds	<1 second	~100x faster ⚡

Transparency Features

User Visibility

✅ Clear indicators in status message
✅ Different emojis (⚡ vs ✅)
✅ Explicit labels ([Pre-generated example] vs [Freshly generated])

Why This Matters

• Users understand what they're getting
• Meets hackathon requirement for transparency
• Builds trust with clear communication
• No "hidden magic"

Technical Benefits

1. Instant UX - Examples load immediately
2. Cost Savings - Reduces repeated API calls
3. Reliability - Examples always work
4. Scalability - Easy to add more cached topics
5. Transparency - Users know the difference

Example Topics (Pre-cached)

1. AlphaFold 3
2. transformer neural network
3. mRNA vaccine technology
4. tuberculosis vaccine
5. CRISPR/Cas9 gene editing applications
6. 3I/ATLAS
7. quantum entanglement
8. Ocean acidification

Maintenance

Adding More Cached Topics

1. Edit generate_cache.py:

   EXAMPLE_TOPICS = [
       # ... existing topics
       "your new topic here",
   ]
   

2. Run generator:

   uv run --env-file .env python generate_cache.py
   

3. Update UI examples in app.py to match

Clearing Cache

# Remove all cached content
rm -rf cache/

Cache will be regenerated on next generate_cache.py run.

Cache Storage

• Metadata: cache/metadata.json (~1KB per topic)
• Text files: ~5-10KB per topic
• Audio files: ~5-15MB per topic (WAV format)
• Total per topic: ~5-15MB
• 8 examples: ~40-120MB total

Future Enhancements

Possible improvements (post-hackathon):

1. LRU Cache - Auto-remove old entries
2. Compression - Convert WAV to MP3 (~10x smaller)
3. CDN Integration - Serve from edge locations
4. Smart Preloading - Predict likely user queries
5. Cache Analytics - Track hit/miss rates

Troubleshooting

Cache not working?

# Check if metadata exists
cat cache/metadata.json

# Verify files exist
ls -lh cache/

# Test cache manager
uv run --env-file .env python test_cache.py

Slow cache retrieval?

• Check file system performance
• Verify audio file exists at path
• Review logs for errors

Inconsistent results?

• Clear cache and regenerate
• Check topic normalization (case, whitespace)
• Verify hash generation

Summary

✅ Implemented: Full caching system with transparency
✅ Performance: ~100x faster for cached topics
✅ UX: Clear indicators for cached vs fresh content
✅ Hackathon Ready: Meets all requirements

The caching system provides the best of both worlds:

• Speed for common queries (examples)
• Freshness for custom topics
• Transparency for all users