feat: add logging and pathlib examples to test audio processor notebook

LEARNING_GUIDE.md

@@ -76,6 +76,383 @@ graph TD
 
 ---
 
+## Python Logging Module
+
+### What is Logging?
+
+Logging is Python's built-in system for tracking events, debugging, and monitoring your application. It's much better than using `print()` statements for debugging.
+
+### Basic Setup
+
+```python
+import logging
+
+# Create a logger instance specific to this module
+logger = logging.getLogger(__name__)
+
+# Configure logging to display messages
+logging.basicConfig(
+    level=logging.INFO,  # Show INFO and above (INFO, WARNING, ERROR, CRITICAL)
+    format='%(levelname)s - %(name)s - %(message)s'
+)
+
+# Now you can log messages
+logger.info("Audio processor functions module loaded.")
+```
+
+### Why Use `__name__` with Logger?
+
+**Benefits of `getLogger(__name__)`:**
+
+1. **Hierarchical organization**: If your code is imported as a module (like `utils.audio_processor`), the logger name will be `"utils.audio_processor"` instead of `"__main__"`. This creates a logger hierarchy that helps organize logs from different parts of your app.
+
+2. **Filtering by module**: You can configure different log levels for different parts of your application:
+   ```python
+   logging.getLogger("agents").setLevel(logging.DEBUG)  # Verbose for agents
+   logging.getLogger("utils").setLevel(logging.WARNING)  # Quiet for utils
+   ```
+
+3. **Identifies source**: In log output, you can see exactly which module generated each message, making debugging much easier.
+
+4. **Best practice**: Prevents logger name conflicts and follows Python conventions.
+
+### Log Levels
+
+From least to most severe:
+
+| Level | When to Use | Example |
+|-------|-------------|---------|
+| `DEBUG` | Detailed diagnostic information | `logger.debug(f"Variable x = {x}")` |
+| `INFO` | General informational messages | `logger.info("Processing started")` |
+| `WARNING` | Something unexpected, but not an error | `logger.warning("Cache miss, fetching from API")` |
+| `ERROR` | An error occurred, but app can continue | `logger.error(f"Failed to load file: {e}")` |
+| `CRITICAL` | Serious error, app may crash | `logger.critical("Database connection lost!")` |
+
+### Why Logging Doesn't Show by Default
+
+**The problem:** By default, loggers only show messages at WARNING level and above. Your `logger.info()` calls are ignored!
+
+**The solution:** Configure logging with `basicConfig()` to set the minimum level:
+
+```python
+logging.basicConfig(level=logging.INFO)  # Now INFO messages will appear
+```
+
+### Format String Explained
+
+```python
+format='%(levelname)s - %(name)s - %(message)s'
+```
+
+This creates output like:
+```
+INFO - __main__ - Audio processor functions module loaded.
+```
+
+- `%(levelname)s` → Log level (INFO, ERROR, etc.)
+- `%(name)s` → Logger name (from `__name__`)
+- `%(message)s` → Your actual message
+
+**Note:** You can add timestamps with `%(asctime)s` if you need them, but for simple learning it's cleaner without.
+
+### Practical Example
+
+```python
+import logging
+
+logger = logging.getLogger(__name__)
+
+def process_audio(file_path):
+    logger.debug(f"Starting audio processing for: {file_path}")  # Only in DEBUG mode
+    
+    try:
+        # Process the file
+        logger.info(f"Successfully processed: {file_path}")  # Normal operation
+        return True
+    except FileNotFoundError:
+        logger.error(f"File not found: {file_path}")  # Error, but continue
+        return False
+    except Exception as e:
+        logger.critical(f"Critical error processing {file_path}: {e}")  # Serious problem
+        raise
+```
+
+### Why Use Logging Instead of Print?
+
+| Feature | `print()` | `logging` |
+|---------|-----------|-----------|
+| **Control output** | ❌ Always prints | ✅ Can turn on/off by level |
+| **Timestamps** | ❌ Manual | ✅ Automatic |
+| **File output** | ❌ Manual redirection | ✅ Built-in handlers |
+| **Severity levels** | ❌ No distinction | ✅ DEBUG, INFO, WARNING, etc. |
+| **Production-ready** | ❌ Need to remove/comment | ✅ Just change log level |
+| **Module identification** | ❌ Manual | ✅ Automatic with `__name__` |
+
+### In Your Science Storyteller Project
+
+You'll use logging to track:
+- Which research papers were retrieved
+- API call successes/failures
+- Processing steps (search → summarize → TTS)
+- Errors during workflow
+- Performance timing
+
+**Example from your project:**
+```python
+logger.info(f"Searching for papers on topic: {topic}")
+logger.warning("No papers found, trying fallback query")
+logger.error(f"API call failed: {e}")
+```
+
+---
+
+## Working with File Paths: `pathlib.Path`
+
+### What is `pathlib`?
+
+`pathlib` is Python's modern, object-oriented way to work with file system paths. It was introduced in **Python 3.4** (2014) and is now the recommended approach for handling files and directories.
+
+### Why Use `Path` Instead of Strings?
+
+**Old way (strings and `os.path`):**
+```python
+import os
+
+path = "/home/user/audio.mp3"
+if os.path.exists(path):
+    dirname = os.path.dirname(path)
+    basename = os.path.basename(path)
+    new_path = os.path.join(dirname, "new_audio.mp3")
+```
+
+**New way (`pathlib.Path`):**
+```python
+from pathlib import Path
+
+path = Path("/home/user/audio.mp3")
+if path.exists():
+    dirname = path.parent
+    basename = path.name
+    new_path = path.parent / "new_audio.mp3"  # Use / operator!
+```
+
+**Benefits:**
+- ✅ More readable and intuitive
+- ✅ Works across Windows/Mac/Linux automatically
+- ✅ Chainable methods
+- ✅ Less error-prone than string concatenation
+- ✅ Object-oriented design
+
+### Creating Path Objects
+
+```python
+from pathlib import Path
+
+# From a string
+p = Path("/home/user/app/assets/audio/test.mp3")
+
+# From current directory
+p = Path.cwd()  # Current working directory. It does not need input path.
+
+# From home directory
+p = Path.home()  # User's home directory (~)
+
+# Relative paths
+p = Path("./assets/audio")
+```
+
+### Path Properties and Methods
+
+```python
+from pathlib import Path
+
+p = Path("/home/user/app/assets/audio/podcast_123.mp3")
+
+# Check existence and type
+p.exists()          # True/False - does it exist?
+p.is_file()         # True/False - is it a file?
+p.is_dir()          # True/False - is it a directory?
+
+# Get path components
+p.name              # 'podcast_123.mp3' - filename with extension
+p.stem              # 'podcast_123' - filename without extension
+p.suffix            # '.mp3' - file extension
+p.parent            # Path('/home/user/app/assets/audio') - parent directory
+p.parts             # ('/', 'home', 'user', 'app', 'assets', 'audio', 'podcast_123.mp3')
+
+# Path conversion
+str(p)              # Convert Path to string
+p.absolute()        # Get absolute path
+p.resolve()         # Resolve symlinks and make absolute
+```
+
+### Common Operations
+
+**1. Check if file exists:**
+```python
+path = Path("myfile.txt")
+if path.exists():
+    print("File found!")
+```
+
+**2. Create directories:**
+```python
+audio_dir = Path("./assets/audio")
+audio_dir.mkdir(parents=True, exist_ok=True)
+# parents=True: creates parent directories if needed
+# exist_ok=True: doesn't raise error if already exists
+```
+
+**3. Join paths (the smart way):**
+```python
+base = Path("./assets")
+audio_file = base / "audio" / "test.mp3"  # Use / operator!
+# Result: Path('./assets/audio/test.mp3')
+
+# Works with strings too!
+file_path = base / "audio" / f"podcast_{123}.mp3"
+```
+
+**4. Find files (glob patterns):**
+```python
+audio_dir = Path("./assets/audio")
+
+# All MP3 files in directory
+mp3_files = list(audio_dir.glob("*.mp3"))
+
+# All files recursively
+all_files = list(audio_dir.glob("**/*"))
+
+# Specific pattern
+podcasts = list(audio_dir.glob("podcast_*.mp3"))
+```
+
+**5. Read and write files:**
+```python
+path = Path("data.txt")
+
+# Write text
+path.write_text("Hello, world!")
+
+# Read text
+content = path.read_text()
+
+# Write bytes (for binary files)
+path.write_bytes(b'\x89PNG...')
+
+# Read bytes
+data = path.read_bytes()
+```
+
+**6. Get file metadata:**
+```python
+path = Path("myfile.txt")
+
+stats = path.stat()
+size_bytes = stats.st_size
+modified_time = stats.st_mtime
+```
+
+### Real Example from Your Project
+
+From `utils/audio_processor.py`:
+
+```python
+def process_audio_file(audio_path: str) -> Optional[str]:
+    """Validate an audio file using Path."""
+    
+    # Convert string to Path object
+    path = Path(audio_path)
+    
+    # Check if file exists
+    if not path.exists():
+        logger.error(f"Audio file not found: {audio_path}")
+        return None
+    
+    # Check file extension
+    if not path.suffix.lower() in ['.mp3', '.wav', '.ogg']:
+        logger.error(f"Invalid audio format: {path.suffix}")
+        return None
+    
+    # Convert back to string for return
+    return str(path)
+```
+
+**Why this is better than strings:**
+- `path.exists()` is clearer than `os.path.exists(audio_path)`
+- `path.suffix` is simpler than manually parsing the extension
+- Cross-platform compatible (Windows uses `\`, Unix uses `/`)
+- Type-safe with IDE autocomplete
+
+### Advanced Example: Cleanup Old Files
+
+```python
+from pathlib import Path
+
+def cleanup_old_files(directory: str, max_files: int = 10):
+    """Remove oldest audio files, keeping only max_files."""
+    
+    dir_path = Path(directory)
+    
+    if not dir_path.exists():
+        return
+    
+    # Get all MP3 files sorted by modification time
+    audio_files = sorted(
+        dir_path.glob('*.mp3'),              # Find all MP3s
+        key=lambda p: p.stat().st_mtime,     # Sort by modified time
+        reverse=True                          # Newest first
+    )
+    
+    # Remove oldest files beyond max_files
+    for old_file in audio_files[max_files:]:
+        old_file.unlink()  # Delete the file
+        logger.info(f"Removed old file: {old_file}")
+```
+
+### Path Version History
+
+- **Python 3.4** (2014): `pathlib` introduced
+- **Python 3.5** (2015): Bug fixes and improvements
+- **Python 3.6+** (2016+): Standard library functions accept `Path` objects
+
+**Backward compatibility:** If you need to support Python 2.7 or 3.3, use `pathlib2` package. But for modern projects (like yours), just use built-in `pathlib`.
+
+### Quick Reference Table
+
+| Task | Old Way (`os.path`) | New Way (`pathlib.Path`) |
+|------|---------------------|--------------------------|
+| Check exists | `os.path.exists(path)` | `Path(path).exists()` |
+| Get filename | `os.path.basename(path)` | `Path(path).name` |
+| Get directory | `os.path.dirname(path)` | `Path(path).parent` |
+| Join paths | `os.path.join(a, b)` | `Path(a) / b` |
+| Get extension | Manual string split | `Path(path).suffix` |
+| Create directory | `os.makedirs(path)` | `Path(path).mkdir(parents=True)` |
+| List files | `os.listdir(path)` | `Path(path).iterdir()` |
+| Read file | `open(path).read()` | `Path(path).read_text()` |
+
+### When to Convert Between Path and String
+
+**Rule of thumb:**
+- Use `Path` objects internally for all file operations
+- Convert to `str()` only when:
+  - Passing to APIs that don't accept Path
+  - Displaying to user
+  - Storing in JSON or database
+
+```python
+# Internal: use Path
+path = Path("./assets/audio") / "file.mp3"
+
+# External API: convert to string
+audio_url = upload_to_api(str(path))
+
+# Display to user: convert to string
+print(f"Audio saved to: {path}")  # Prints nicely automatically
+```
+
+---
+
 ## Python Function Basics
 
 Functions are the primary way to group code into reusable blocks. Let's break down a function from our codebase: `utils/audio_processor.py`.
@@ -1221,7 +1598,7 @@ The app uses Python's logging:
 ```python
 logging.basicConfig(
     level=logging.INFO,  # Change to DEBUG for more detail
-    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+    format='%(levelname)s - %(name)s - %(message)s'
 )
 ```
 

tests/retrieve.ipynb

@@ -20,7 +20,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "id": "3db54ea3",
    "metadata": {},
    "outputs": [
@@ -37,7 +37,7 @@
     }
    ],
    "source": [
-    "ArxivTool.connect(self)"
+    "ArxivTool.connect()"
    ]
   }
  ],

tests/test_audio_processor.ipynb

@@ -0,0 +1,183 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b561f992",
+   "metadata": {},
+   "source": [
+    "# Test functions "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4060c315",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "from pathlib import Path\n",
+    "from typing import Optional"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f652ec87",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import logging\n",
+    "\n",
+    "# A logger instance specific to the module\n",
+    "logger = logging.getLogger(__name__)\n",
+    "\n",
+    "# Only configure if not already configured (prevents duplicate handlers)\n",
+    "if not logging.getLogger().hasHandlers():\n",
+    "    logging.basicConfig(\n",
+    "        level=logging.INFO,  # Show INFO and above (INFO, WARNING, ERROR, CRITICAL)\n",
+    "        format='%(asctime)s - %(levelname)s - %(message)s',\n",
+    "        datefmt='%H:%M:%S'  # Only show hours:minutes:seconds\n",
+    "    )\n",
+    "\n",
+    "logger.info(\"Audio processor functions module loaded.\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "50c921a9",
+   "metadata": {},
+   "source": [
+    "## Download a sample audio file for testing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4e97b64a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "\n",
+    "test_audio_path = \"../assets/audio/test_sample.mp3\"\n",
+    "\n",
+    "if Path(test_audio_path).exists():\n",
+    "    logger.warning(f\"File already exists at {test_audio_path}, skipping download.\")\n",
+    "    pass\n",
+    "else:\n",
+    "    logger.info(f\"Downloading sample audio file to {test_audio_path}...\")\n",
+    "    url = \"https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3\"\n",
+    "    response = requests.get(url)\n",
+    "    with open(test_audio_path, 'wb') as f:\n",
+    "        f.write(response.content)\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "660c57af",
+   "metadata": {},
+   "source": [
+    "## Process audio file"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4fdd70ec",
+   "metadata": {},
+   "source": [
+    "## pathlib and Path()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "54d267ae",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Test if the audio file exists\n",
+    "test_audio_path = \"../assets/audio/test_sample.mp3\"\n",
+    "\n",
+    "if Path(test_audio_path).exists():\n",
+    "    logger.info(f\"Test audio file exists at {test_audio_path}.\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0b42d3f2",
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31mThe Kernel crashed while executing code in the current cell or a previous cell. \n",
+      "\u001b[1;31mPlease review the code in the cell(s) to identify a possible cause of the failure. \n",
+      "\u001b[1;31mClick <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. \n",
+      "\u001b[1;31mView Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details."
+     ]
+    }
+   ],
+   "source": [
+    "p = Path(test_audio_path)\n",
+    "\n",
+    "if p.exists():\n",
+    "    logger.info(f\"Audio file exists at: {p}\")\n",
+    "    \n",
+    "    # Path exists?\n",
+    "    logger.info(f\"Path exists? {p.exists()}\")\n",
+    "else:\n",
+    "    logger.error(f\"Audio file not found at: {test_audio_path}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ccc7fa24",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(Path.cwd())\n",
+    "print(Path.home())\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9452f2f7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "audio_path = \"../assets/audio/test_sample.mp3\"\n",
+    "\n",
+    "path = Path(audio_path)\n",
+    "print(path)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.19"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}