In this step-by-step tutorial, I show you how to create a Claude Code skill from scratch. You’ll learn how Claude Code skills work, where they live, and how to write instructions that make Claude follow proper red-green-refactor TDD.

I found some existing YNAB MCPs, but most were inactive with limited features. This seemed like a good opportunity to learn MCP design from scratch. What followed was a deep dive into context efficiency that changed how I think about building AI tools.

Understanding Model Context Protocols (MCPs)

The Model Context Protocol (MCP) is a standardized way to extend language models with external capabilities. Unlike traditional APIs where you write code to call endpoints, MCP servers allow AI models to discover and use tools autonomously. The protocol defines how models can:

• Call tools - Execute functions that interact with external systems
• Read resources - Access files, databases, or other data sources
• Receive prompts - Get specialized instructions for specific tasks

When you build an MCP server, you’re essentially creating a set of capabilities that any MCP-compatible AI assistant (like Claude) can use. The model sees your tool descriptions, understands what they do, and calls them as needed to fulfill user requests.

Understanding Context Windows

Before we dive into how to make our MCPs more efficient, it’s important to understand what we’re trying to optimize. When working with LLMs, the context window is the amount of content that the model can “pay attention” to at one time. Each model has a limit to the size of its context window. For example, Claude Sonnet 4.5’s context window is about 200,000 tokens.

What is a Token?

When you send text to an LLM, it doesn’t process words one at a time. Instead, text is broken into tokens—the fundamental units that language models read and generate. A token typically represents 3-4 characters, or roughly 0.75 words in English.

For API responses and JSON data (which is what MCPs work with), tokenization looks like this:

{"name": "Checking"}           // ~7 tokens
"transfer_payee_id"            // ~5 tokens
{"balance": 125000}            // ~6 tokens

The tokenizer breaks JSON into chunks: brackets, keys, values, and punctuation all consume tokens. Every field name has a cost. Field names like "debt_escrow_amounts" and "direct_import_in_error" cost ~4-6 tokens each. When you return an API response with 18 fields per object, you’re paying the token cost for every field name, every time—even when the model doesn’t need them.

Why Token Efficiency Matters

Given the limited size of the context window, it’s critical to consider how much you’re placing inside it. Models work best when they have good context, but there’s a balance to strike:

• Context limits are hard boundaries: Claude Sonnet 4.5’s 200k token limit sounds generous until you realize a naive MCP returning a year of transactions can consume 746,800 tokens—nearly 4x the entire context window. Your tool call would fail before the model could even process it.

• Real sessions are already crowded: In typical Claude Code sessions, MCP tool definitions, system prompts, and memory can consume 50-60% of the context window before you’ve had a single conversation. Every inefficient tool response eats into precious space needed for reasoning and multi-turn conversations.

• Noise degrades performance: Providing superfluous data doesn’t just waste tokens—it forces the model to parse irrelevant fields, increasing the chance of errors or confusion. A focused 262-token summary outperforms a noisy 4,890-token dump of raw data.

The goal isn’t to minimize tokens at all costs. It’s to give the model exactly what it needs, nothing more, nothing less. Let’s look at how this plays out in practice.

The Naive Approach: Direct API Wrapping

When I started building the YNAB MCP, I didn’t yet appreciate how much token efficiency would matter. I started with what seemed obvious: create a thin wrapper around the existing YNAB Python SDK. Each MCP tool would correspond to one API endpoint, passing through the full response. This is a common pattern I’ve seen in many MCP implementations.

# Naive approach: Just wrap the SDK
@mcp.tool()
async def get_budget(budget_id: str) -> str:
    response = ynab_client.budgets.get_budget(budget_id)
    return json.dumps(response.data.budget)  # Return everything

This approach works, technically. The model gets access to the data. But there’s a critical problem: API responses are designed for applications, not for AI context windows.

Traditional applications can process, filter, and cache data efficiently, so APIs return comprehensive data structures optimized for completeness, not token efficiency. A single endpoint might return thousands of fields because the API designers don’t know which specific fields your application needs.

AI models work differently. Every byte consumes precious context window space—space you could use for reasoning, conversation history, or additional tool calls. When you blindly pass through full API responses, you’re asking the model to pay the “context tax” for data it might not even need. Worse, the model has to analyze and determine which parts of that data are actually relevant—a cognitive load that can lead to errors or missed information.

To put this in perspective: tool results compete with everything else for context space. In a real Claude Code session (visible via /context), I saw the context window at 118k/200k tokens (59%)—before I’d even started a conversation. MCP tool definitions alone consumed 47.9k tokens (24%), system tools used 17.3k tokens (9%), custom agents took 2.4k tokens, and memory files added another 2.3k tokens. That’s 59% of the context window used just by the environment.

A naive MCP that returns 30k tokens for a budget overview would push that to 74% in a single tool call—leaving just 52k tokens for the actual conversation, reasoning, and additional tool calls. Every inefficient tool response eats into the space you need for multi-turn conversations.

This changed how I approached the design: MCPs need to be context-aware intermediaries, not transparent proxies. The question shifted from “How do I expose this API to the model?” to “What does the model actually need to help the user?”

Three Design Principles for Context-Efficient MCPs

One of the first things I wanted to do was check my budget overview - see my accounts, categories, and how I’m tracking for the current month. A straightforward use case that any budgeting tool should support.

My initial thought was to create tools that directly wrapped the YNAB API endpoints. Let’s take the accounts endpoint as an example. Here’s what the API returns:

{
  "data": {
    "accounts": [
      {
        "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
        "name": "Checking Account",
        "type": "checking",
        "on_budget": true,
        "closed": false,
        "note": "Primary checking",
        "balance": 125000,
        "cleared_balance": 120000,
        "uncleared_balance": 5000,
        "transfer_payee_id": "...",
        "direct_import_linked": true,
        "direct_import_in_error": false,
        "last_reconciled_at": "2025-11-05T18:27:20.140Z",
        "debt_original_balance": 0,
        "debt_interest_rates": {},
        "debt_minimum_payments": {},
        "debt_escrow_amounts": {},
        "deleted": false
      }
      // ... 46 more accounts
    ]
  }
}

For my 47 accounts, this API response contains 18 fields per account. Many of these fields are irrelevant for typical budget questions:

• Debt interest rates and minimum payments (only relevant for debt accounts)
• Direct import status (internal system state)
• Cleared vs uncleared balance breakdown (too granular for overview)
• Transfer payee IDs (internal references)
• Last reconciliation date (accounting detail)

When you’re answering budget questions, these internal bookkeeping details just create noise.

A naive wrapper would return all 18 fields × 47 accounts = 9,960 tokens.

But here’s what I actually need to answer questions like “What’s my checking account balance?” or “How much do I have across all accounts?”:

• Account name
• Account type
• Balance
• Whether it’s on-budget
• Whether it’s closed

That’s it. Just 6 fields. Here’s the filtered implementation:

For accounts (47 accounts):

# src/ynab_mcp/ynab_client.py
async def get_accounts(self, budget_id: str) -> list[dict[str, Any]]:
    """Get all accounts for a budget."""
    response = self.client.accounts.get_accounts(budget_id)
    accounts = []

    for account in response.data.accounts:
        # Skip deleted accounts entirely
        if account.deleted:
            continue

        # Return only the fields the model actually needs
        accounts.append({
            "id": account.id,
            "name": account.name,
            "type": account.type,
            "on_budget": account.on_budget,
            "closed": account.closed,
            "balance": account.balance / 1000 if account.balance else 0,
        })

    return accounts

Filtered approach (6 essential fields): 3,451 tokens for 47 accounts

Reduction: 65.4% by removing 12 unnecessary fields that the model doesn’t need for typical budget questions.

The Full Budget Overview

Of course, checking accounts is just one part of viewing your budget. A complete budget overview requires three tool calls:

Naive approach (no filtering):

• Accounts (all fields): 9,960 tokens
• Categories (all, including hidden): 12,445 tokens
• Monthly summary (estimated): ~8,000 tokens
• Total: ~30,405 tokens

Context-efficient approach:

• Accounts (filtered): 3,451 tokens
• Categories (visible only): 8,620 tokens
• Monthly budget summary: 6,808 tokens
• Total: ~18,879 tokens

Workflow reduction: 38% fewer tokens for the same functionality - a complete picture of my budget for the current month. This leaves plenty of room in Claude’s context window for conversation history, reasoning, and additional tool calls.

But it’s not just about saving tokens. By filtering out unnecessary data, I’m also improving model accuracy. When Claude doesn’t see fields like debt_escrow_amounts or direct_import_in_error, it can’t get confused by them or incorrectly incorporate them into calculations. The model focuses on exactly what matters: account names, balances, and budget status.

The key insight: the model doesn’t need to see all the data to work with it effectively. In fact, it works better with less data. By doing the filtering in the tool layer, I kept the context window lean while maintaining full functionality and improving reliability.

The filtering techniques I’d learned from accounts and categories immediately paid off when I tackled the next challenge: helping Claude categorize uncategorized transactions.

Categorizing Transactions

One of the workflows I wanted help with was taking uncategorized transactions and suggesting categories for them. This would help me ensure that my budget was accurate and up-to-date. To do this I created a tool that would fetch all uncategorized transactions from YNAB and get a list of the categories available in the budget.

On the first pass I noticed something unusual. Claude was recommending categories that were hidden in my budget - old categories I no longer used but hadn’t deleted. I quickly realized that the YNAB REST API doesn’t provide a way to exclude hidden categories in the API call itself. That meant I had two options:

1. Include instructions in my tool description telling Claude to ignore hidden categories
2. Filter them out in the tool code before returning data to Claude

I chose option 2. Here’s why: every instruction you add to a tool description consumes context window. More importantly, it puts the burden of filtering on the model. This means we’re knowingly giving the model more data than it needs, forcing it to do more work and potentially allowing the model to make mistakes.

It’s worth emphasizing: tool descriptions themselves consume context tokens. In my Claude Code session, MCP tool definitions consumed 47.9k tokens (24% of the context window) before any tools were even called. Every line of documentation, every parameter description, every usage instruction adds up. This creates a tension: you want clear, helpful descriptions, but verbose documentation eats into the space available for actual tool results and conversation.

The solution isn’t to write minimal descriptions—clarity matters. Instead, keep descriptions focused on what the tool does and its parameters, and handle behavior rules (like “ignore hidden items”) in your implementation code rather than in lengthy instructions. Instead, I implemented filtering at the tool layer:

def _filter_categories(
    self, categories: list[dict[str, Any]], include_hidden: bool = False
) -> list[dict[str, Any]]:
    """Filter categories to exclude hidden/deleted ones by default."""
    filtered = []
    for category in categories:
        # Always skip deleted categories
        if category.get("deleted"):
            continue
        # Skip hidden categories unless explicitly included
        if not include_hidden and category.get("hidden"):
            continue
        filtered.append(category)
    return filtered

Then I exposed this as a parameter in the tool:

@mcp.tool()
async def get_categories(budget_id: str, include_hidden: bool = False) -> str:
    """Get all categories for a budget.

    Args:
        budget_id: The ID of the budget (use 'last-used' for default budget)
        include_hidden: Include hidden categories and groups (default: False)

    Returns:
        JSON string with category groups and categories
    """

This approach gave me the best of both worlds: by default, Claude only sees active categories, but I can still access hidden categories when needed (like when I wanted to identify old balances that needed cleanup). This saved 30.7% of tokens per category list request (from 12,445 to 8,620 tokens by filtering out 69 hidden categories) while improving accuracy.

Design Principle #1: Filter at the Source

Do data filtering in your tool code rather than relying on prompt instructions. This saves tokens and prevents errors.

Historical Spending Analysis

Next, I wanted to be able to ask questions about my historical spending. Questions like “How much did I spend on groceries last month?” or “What categories am I overspending in?” would be really useful.

My first instinct was to create a tool that fetched all transactions for a date range and let Claude analyze them. But I quickly realized this approach had serious problems.

To illustrate, let me show you the real numbers for my 2024 transactions:

• Total transactions in 2024: 3,456
• Fields per transaction: 14 (id, date, amount, memo, account_name, payee_name, category_name, cleared, approved, etc.)
• Average per transaction: ~216 tokens

Now extrapolate this to common queries:

• 1 month of transactions (~284 txns): ~61,368 tokens
• 3 months of transactions (~852 txns): ~184,106 tokens
• 6 months of transactions (~1,704 txns): ~368,213 tokens
• 1 year of transactions (3,456 txns): ~746,800 tokens

The problems with this approach:

1. Token usage: A full year query would consume 746,800 tokens - that’s 3.7x larger than Claude Sonnet 4.5’s entire 200k context window! You literally couldn’t fit a year of transactions in a single request.
2. Speed: Transferring and parsing thousands of transaction objects is slow
3. Analysis burden: Claude would need to group, sum, and calculate averages on raw data
4. Wasted context: Most of those 14 fields per transaction aren’t relevant to “how much did I spend?”

Even a modest 3-month query would consume 184k tokens - using 92% of the available context window just for raw transaction data. This leaves almost no room for Claude to maintain conversation history, reason about the results, or make additional tool calls to answer follow-up questions.

Instead, I realized these calculations could easily be handled in the tool layer. Here’s what the aggregation logic looks like:

async def get_category_spending_summary(
    self,
    budget_id: str,
    category_id: str,
    since_date: str,
    until_date: str,
    include_graph: bool = True,
) -> dict[str, Any]:
    """Get spending summary for a category over a date range."""

    # Fetch transactions from API
    result = await self._make_request_with_retry("get", url, params=params)
    txn_data = result["data"]["transactions"]

    # Aggregate in tool layer
    total_spent = 0
    transaction_count = 0
    monthly_totals = {}

    for txn in txn_data:
        # Filter by category and date range
        if txn.get("category_id") != category_id:
            continue
        if txn["date"] > until_date:
            continue

        # YNAB stores amounts in milliunits (e.g., $125.00 = 125000)
        amount = txn["amount"] / 1000 if txn.get("amount") else 0
        total_spent += amount
        transaction_count += 1

        # Build monthly breakdown
        month_key = txn["date"][:7]  # YYYY-MM
        if month_key not in monthly_totals:
            monthly_totals[month_key] = 0
        monthly_totals[month_key] += amount

    # Calculate average per month
    num_months = len(monthly_totals) if monthly_totals else 1
    average_per_month = total_spent / num_months if num_months > 0 else 0

    # Return only the summary
    return {
        "category_id": category_id,
        "date_range": {"start": since_date, "end": until_date},
        "total_spent": total_spent,
        "transaction_count": transaction_count,
        "average_per_month": average_per_month,
        "monthly_breakdown": [
            {"month": month, "spent": amount}
            for month, amount in sorted(monthly_totals.items())
        ],
    }

The impact was dramatic. Let me show you a real example from my implementation:

Scenario: Analyze 6 months of spending for a single category (22 transactions)

• Before (returning raw transactions): 4,890 tokens
• After (pre-aggregated summary): 262 tokens
• Reduction: 94.6%

The aggregated response includes:

• Total spent
• Average per month
• Transaction count
• Monthly breakdown (array of {month, amount} objects)

That’s everything Claude needs to answer questions like “Am I spending more on groceries this year than last year?” without having to receive, parse, and aggregate dozens of individual transaction records.

Design Principle #2: Pre-Aggregate Data

Pre-calculate aggregations, summaries, and statistics in your tool code. Return insights, not raw data. This keeps your context window lean while still giving the model everything it needs to help users.

While filtering and aggregation solved the token efficiency problem, I ran into a different challenge: the API itself had limitations.

Building Tools for Unsupported Actions

While working on the workflow to have Claude help me categorize transactions, I realized I needed a way to split a transaction across multiple categories. For example, a Costco purchase might include $150 of groceries, $50 of household items, and $30 of gas.

Unfortunately, the YNAB API does not provide a way to convert an existing transaction into a split transaction. The API only allows creating NEW transactions with splits. This was a real limitation - but it presented an opportunity to think creatively about tool design.

Instead of telling users “sorry, the API doesn’t support this,” I created a tool that works within the API’s constraints:

@mcp.tool()
async def prepare_split_for_matching(
    budget_id: str,
    transaction_id: str,
    subtransactions: str,
) -> str:
    """Prepare a split transaction to match with an existing imported transaction.

    This tool fetches an existing transaction's details and creates a new UNAPPROVED split
    transaction with the same date, amount, account, and payee. You can then manually match
    them together in the YNAB web or mobile UI.

    Workflow:
        1. This tool fetches the existing transaction details
        2. Creates a new unapproved split transaction with those details
        3. You manually match them in the YNAB UI
        4. YNAB merges them into one split transaction

    Note:
        - The new split is created as UNAPPROVED for manual matching
        - The sum of subtransaction amounts should equal the original transaction amount
    """

The implementation:

async def prepare_split_for_matching(
    self,
    budget_id: str,
    transaction_id: str,
    subtransactions: list[dict[str, Any]],
) -> dict[str, Any]:
    # Fetch the original transaction details
    original = await self.get_transaction(budget_id, transaction_id)

    # Create a new split transaction with the same details but unapproved
    new_split = await self.create_split_transaction(
        budget_id=budget_id,
        account_id=original["account_id"],
        date=original["date"],
        amount=original["amount"],
        subtransactions=subtransactions,
        payee_name=original.get("payee_name"),
        memo=original.get("memo"),
        cleared=original.get("cleared", "uncleared"),
        approved=False,  # Key: create as unapproved for matching
    )

    return {
        "original_transaction": original,
        "new_split_transaction": new_split,
        "instructions": (
            "A new unapproved split transaction has been created. "
            "Go to YNAB and manually match these two transactions together. "
            "Look for the match indicator in the YNAB UI."
        ),
    }

This solution works because YNAB has a built-in “matching” feature where it can merge a manually-entered transaction with an imported one. By creating the split as unapproved, YNAB’s UI will detect the duplicate and offer to match them. When you accept the match, the imported transaction becomes a proper split transaction.

Is this ideal? No - I’d prefer a direct API endpoint. But it’s a pragmatic solution that works within the constraints of the underlying platform while still providing value to the user.

Design Principle #3: Work Within API Constraints

When an API doesn’t support something directly, look for workflows that combine available operations to achieve the desired outcome.

Real-World Token Reduction Results

Here’s a summary of the optimizations applied across the YNAB MCP, with real measured token counts:

When to Apply Each Technique

Field Filtering (accounts, categories)

• Use when: API returns many fields, but only subset is needed for common queries
• Savings: Moderate (30-65%)
• Complexity: Low - simple field selection
• Example: Remove debt details from non-debt accounts, skip internal IDs like transfer_payee_id, drop reconciliation timestamps

  # Instead of returning all 18 fields, return only what matters
  return {
      "id": account.id,
      "name": account.name,
      "balance": account.balance / 1000,
      "on_budget": account.on_budget
  }
  

Default Filtering with Parameters (hidden categories)

• Use when: Some data is rarely needed but occasionally useful
• Savings: Moderate (30-40%)
• Complexity: Low - add optional boolean parameter
• Example: Hide deleted/archived items by default, expose via include_deleted flag

Pre-aggregation (spending analysis)

• Use when: Model would need to compute summaries from raw data
• Savings: High (90-99%)
• Complexity: Medium - requires aggregation logic
• Example: Return monthly totals instead of individual transactions

Creative Workarounds (split transactions)

• Use when: API doesn’t support desired operation directly
• Savings: Enables new functionality (not about tokens)
• Complexity: High - requires understanding API constraints
• Example: Multi-step workflows that achieve goals indirectly

How to Measure Your Own MCP

You might be wondering: how did I get these specific numbers? Here’s my methodology—and how you can apply it to your own MCPs.

All the numbers in this post came from real measurements. Here’s how I validated the optimizations, and how you can do the same for your MCP:

1. Set Up Token Counting

Install tiktoken, OpenAI’s tokenizer library (Claude uses a similar tokenization scheme):

pip install tiktoken

Create a helper function to count tokens:

import tiktoken

def count_tokens(text: str) -> int:
    """Count tokens using tiktoken's cl100k_base encoding."""
    encoding = tiktoken.get_encoding("cl100k_base")
    return len(encoding.encode(text))

2. Measure API Responses

Create a script that fetches data both ways and compares:

import json

# Get raw API response
raw_response = api.get_accounts(budget_id)
raw_json = json.dumps(raw_response, indent=2)
raw_tokens = count_tokens(raw_json)

# Get your filtered response
filtered_response = your_mcp_tool.get_accounts(budget_id)
filtered_json = json.dumps(filtered_response, indent=2)
filtered_tokens = count_tokens(filtered_json)

# Compare
reduction = ((raw_tokens - filtered_tokens) / raw_tokens) * 100
print(f"Raw: {raw_tokens:,} tokens")
print(f"Filtered: {filtered_tokens:,} tokens")
print(f"Reduction: {reduction:.1f}%")

3. Test Real Workflows

Don’t just measure individual tools - measure complete workflows users will perform:

# Simulate a budget overview workflow
accounts = your_mcp.get_accounts(budget_id)
categories = your_mcp.get_categories(budget_id, include_hidden=False)
summary = your_mcp.get_budget_summary(budget_id, current_month)

total_tokens = (
    count_tokens(json.dumps(accounts)) +
    count_tokens(json.dumps(categories)) +
    count_tokens(json.dumps(summary))
)

print(f"Budget overview workflow: {total_tokens:,} tokens")

This revealed that my budget overview workflow uses ~19k tokens - well within Claude’s 200k context window with room to spare.

4. Watch for Runtime Warnings

Claude Code will warn you when tool responses exceed ~10k tokens. If you see these warnings frequently, it’s a signal to investigate:

⚠️ Large MCP response (~12.5k tokens), this can fill up context quickly.

These warnings helped me identify which tools needed optimization.

5. Validate Correctness

Token reduction means nothing if your tools return incorrect data. Always verify:

• Does the filtered data answer the user’s questions?
• Are calculations accurate? (spot-check aggregations against raw data)
• Does pagination work correctly? (ensure you’re not computing on partial datasets)

The goal isn’t to minimize tokens at all costs - it’s to return exactly what the model needs, nothing more, nothing less.

Limitations and Trade-offs

This context-efficient approach works well for YNAB, but it’s not without limitations. Before applying these patterns to your own MCP, consider these trade-offs:

Pre-aggregation assumes query patterns. If users ask questions that need raw transaction details (like “show me the memo for my largest grocery purchase”), the aggregated data won’t help. You’ll need additional tools that return raw data for those cases.

Filtering loses flexibility. By removing fields, you can’t answer questions that need those fields without making additional API calls. The key is knowing your use cases. For budget analysis and categorization, these trade-offs are worth it. For transaction-level forensics, you might need different tools.

Caching complexity. Pre-computed aggregations need invalidation strategies when data changes. If your underlying data updates frequently, you’ll need to think carefully about cache freshness and when to recompute.

Development overhead. Writing aggregation logic and filtering code is more work than simple pass-through wrappers. You’re trading implementation time for runtime efficiency. For frequently-used tools, this is usually worth it.

The goal isn’t to optimize every tool to the extreme. It’s to identify the high-impact workflows—the ones users will perform repeatedly—and optimize those intelligently.

Key Takeaways

• Context is expensive: In real Claude Code sessions, MCP tool definitions can consume 24% of the 200k context window before any tools are called
• Measure everything: Use tiktoken to count tokens on API responses before and after optimization
• Filter proactively: Removing 12 unnecessary fields from 47 accounts reduced tokens by 65.4%
• Aggregate strategically: Pre-computing spending summaries reduced a 6-month query by 94.6%
• Design for the model: Ask “What does the model need?” not “What does the API provide?”
• Default to minimal data: Return only what’s necessary by default, with optional parameters for edge cases
• Validate with real workflows: Test complete user flows, not just individual tools, to understand cumulative token impact

Conclusion

Building a context-efficient MCP requires a mindset shift: design for what the model needs, not just what the API provides. The three principles I learned—filter at the source, compute in tools, and work within constraints creatively—apply to any MCP wrapping an external API.

Through careful design, the YNAB MCP achieves dramatic efficiency:

• Budget overview: 38% reduction (30k → 19k tokens)
• Spending analysis: 94.6% reduction (4.9k → 262 tokens)
• Category filtering: 30.7% reduction (12.4k → 8.6k tokens)

These aren’t theoretical—they’re real measurements from actual usage. Context is expensive, and every token matters when you’re building tools for multi-turn conversations.

If you’re building an MCP, start by asking: “What does the model actually need to answer the user’s question?” Not “What does the API return?” That mindset shift makes all the difference.

Further Reading

If you want to dive deeper into MCP development and context optimization:

• Model Context Protocol Specification - Official MCP spec and documentation
• Code Execution with MCP - Anthropic’s engineering blog on building with MCP
• Most devs don’t understand how context windows work - Deep dive into context window fundamentals and practical management strategies
• YNAB API Documentation - The API this MCP wraps
• Anthropic’s Prompt Engineering Guide - Understanding context windows and token efficiency

The full YNAB MCP implementation is available at github.com/dgalarza/ynab-mcp-dgalarza if you want to dive deeper into the code. I’d love to hear about the context optimization techniques you’ve discovered in your own MCP projects.

Building MCP servers or designing Claude Code workflows for your team? I help engineers get this right from the start. Learn more.

For every Rails class we shipped, we had to trace requirements, detail business logic and interfaces, and map out risk controls — all in a format that is audit-ready for FDA review.

This post kicks off a series called ‘AI Prompts I Wish I Had’, sharing prompts that could’ve made our engineering workflows smoother while building regulated software.

The goal of this series is to offer practical, high-context AI prompts that help teams move faster toward an FDA submission — without compromising quality, traceability, or compliance.

🧠 The Prompt

# SDS Documentation Prompt

You are a SaMD software engineer generating regulatory documentation.
I will provide you with one or more Ruby/Rails classes used in a regulated
Software as a Medical Device (SaMD) product.

Based on the code I provide, generate a **Software Design Specification (SDS)** entry suitable
for inclusion in a Design History File (DHF).

Your output should include:

1. **Component Name**  
2. **Purpose / Role in the System**  
3. **Description of Logic / Behavior** – Provide detailed explanations including:
   - Specific business rules with exact criteria and thresholds  
   - All possible status values/enums with definitions of what each represents  
   - Step-by-step process flows with decision points  
   - Data validation rules and constraints  
   - Error handling and edge cases  

4. **External Interfaces** – Include:
   - Specific API endpoints and GraphQL mutations/queries  
   - Database table names and key fields  
   - Session storage keys and data structures  
   - Third-party service integrations  

5. **How It Satisfies Specific SRS Requirements** – Expand on:
   - Security controls with implementation details  
   - Audit logging mechanisms and what data is captured  
   - Regulatory compliance features  
   - Data integrity safeguards  

6. **Design Considerations** – Detail:
   - Security flags and their enforcement mechanisms  
   - Session state management and validation logic  
   - Performance optimizations and their rationale  
   - Safety mechanisms and fail-safes  

7. **Traceability Notes** – Include:
   - Specific risk control implementations  
   - Audit trail mechanisms (PaperTrail, analytics events, etc.)  
   - Logging infrastructure for regulatory compliance  
   - Known technical limitations with business impact  

**Additional Requirements:**
- **No abbreviations or “etc.”** – List all status values, rules, and conditions explicitly  
- **Implementation details** – Show actual code snippets for critical security or compliance logic  
- **Business rule explanations** – Explain the medical/regulatory rationale behind complex rules  
- **Security deep-dive** – Cover authentication, authorization, session management, and data protection  
- **Audit compliance** – Document all logging, tracking, and audit trail mechanisms required for regulatory review

Format the output in Markdown with headers for each section. Be comprehensive, technical, and audit-ready.

Example usage:

The system we are documenting is located within `packs/authentication`.

I want documentation covering the login and session management flow for users.  

How does the system validate credentials?
How is the session established and managed?
What audit trails are recorded for access attempts?

Like anything generated with AI it requires a careful review, but this prompt can help you quickly generate a starting point for a comprehensive SDS entry that meets FDA requirements. It ensures that you cover all necessary aspects of the system’s design and behavior, making it easier to compile your Design History File.

Let me know if this is helpful — and if you want me to share the next prompt in this series.

If you’re building a SaMD or working in a regulated domain and want help tuning your dev workflows with AI, let’s talk.

You’re excited about building a new application which allows users to sign up and host their own blog. You decide that each blog will have their own space by providing a subdomain.

Let’s start off with a feature spec.

require "rails_helper"

feature "user views a blog" do
  scenario "homepage" do
     blog = create(
      :blog,
      subdomain: "bobloblaw",
      title: "Bob Loblaw's Law Blog",
      description: "Welcome to my new blog.",
    )

    visit root_path

    expect(page).to have_content blog.title
    expect(page).to have_content blog.description
  end
end

In our app we render the blog homepage using the following:

# config/routes.rb
Rails.application.routes.draw do
  root to: "blogs#show"
end

# app/controllers/blogs_controller.rb
class BlogsController < ApplicationController
  def show
    @blog = current_blog
  end

  private

  def current_blog
    @_current_blog ||= Blog.find_by(subdomain: request.subdomains.first)
  end
end

<!-- app/views/blogs/show.html.erb -->
<h1><%= @blog.title %></h1>
<p><%= @blog.description %></p>

In order to visit the homepage via a subdomain in our test we need to set the app_host property for Capybara. We could try to use myblog.localhost but Rails will think that localhost is the top level domain and therefore won’t see myblog as a subdomain. Instead we’ll use a fake host name example.com. We can set it by adding the following to our spec before calling visit.

Capybara.app_host = "http://myblog.example.com"

If we run the test with the default Capybara driver, rack-test it should be green. rack-test interacts directly with Rack which means it never uses the external URL. If we need to use a JavaScript driver however we will need to use an actual accessible URL. Add the :js metadata to the scenario and you should see a failure occur.

In order to accommodate a driver like Selenium or capybara-webkit we’ll need to do some more work. To start, we will not be able to use our fake host example.com. Instead we need a host name which will point to 127.0.0.1. There is one readily available to us for use through lvh.me. Its DNS records are set up so that lvh.me and all of its subdomains resolve to your local machine at 127.0.0.1.

So update app_host from http://myblog.example.com to http://myblog.lvh.me. We’re still not done yet though.

Next, we need to instruct Capybara to include the port number for the Capybara server in all requests to work correctly. We can do that by adding the following to spec/rails_helper.rb:

Capybara.configure do |config|
  config.always_include_port = true
end

If you’re using the capybara-webkit driver and configuring it to block all unknown URLs as we do in Suspenders then you’ll need to do one more thing. In the configuration for capybara-webkit you’ll need to add the lvh.me host to the URL whitelist. If you’re using a Suspenders based app then open up spec/support/capybara_webkit.rb or whichever file you have configured capybara-webkit in. Update the configuration to look like:

Capybara::Webkit.configure do |config|
  config.block_unknown_urls
  config.allow_url("myblog.lvh.me")
end

This will allow Capybara to access our blog through lvh.me and not block it. With this in place we can run our tests and things should be green again.

Allowing more subdomains

Things are working great with the above but we realize that we are coupled to the myblog subdomain within all of our tests. We will finish things off by making this more flexible.

Let’s start by updating our capybara-webkit configuration to allow all subdomains on lvh.me and not just limiting it to myblog. We can do this by changing myblog to *.

Capybara::Webkit.configure do |config|¬
  config.block_unknown_urls¬
  config.allow_url("*.lvh.me")¬
end

Next, let’s extract a helper method to make testing subdomains easier.

We’ll add the following method to our feature spec:

def visit_blog(blog, path = '/')
  app_host = URI.join("http://#{blog.subdomain}.lvh.me", path).to_s
  using_app_host(app_host) do
    visit path
  end
end

def using_app_host(host)
  original_host = Capybara.app_host
  Capybara.app_host = host
  yield
ensure
  Capybara.app_host = original_host
end

using_app_host allows us to pass a host for Capybara to use and temporarily overrides the app_host rather then permanently setting it. Our use of ensure makes sure that the app_host is always set back to its original value regardless of exceptions being raised while yielding the block. visit_blog allows us to pass an instance of a blog as well as a path to visit. By default, this path is the root of the blog.

So we can update our spec to look as follows:

require "rails_helper"

feature "user views a blog" do
  scenario "homepage", :js do
     blog = create(
      :blog,
      subdomain: "bobloblaw",
      title: "Bob Loblaw's Law Blog",
      description: "Welcome to my new blog.",
    )

    visit_blog blog

    expect(page).to have_content blog.title
    expect(page).to have_content blog.description
  end

  def visit_blog(blog, path = '/')
    app_host = URI.join("http://#{blog.subdomain}.lvh.me", path).to_s
    using_app_host(app_host) do
      visit path
    end
  end

  def using_app_host(host)
    original_host = Capybara.app_host
    Capybara.app_host = host
    yield
  ensure
    Capybara.app_host = original_host
  end
end

When I first got started, I decided to search for some online iOS with Swift courses. I was hoping to find some courses on Code School or something similar. Unfortunately, I couldn’t find a course like this. The courses that I found were focused around learning iOS with Objective-C. I decided if I was going to start learning iOS, I wanted to learn it with the modern Swift language rather than Objective-C.

That’s not to say that having Objective-C knowledge isn’t valuable. It’s especially important to have Objective-C experience to work with older applications. For example, Tropos is built with Objective-C. In order to contribute to this project, I needed to think back to my Objective-C learnings. The specifics around iOS development tend to be the frameworks that Apple provides to its developers. These frameworks have great documentation and are in both, Swift and Objective-C. If you’re learning how to use these libraries and the concepts behind iOS, it shouldn’t be difficult to apply them to Objective-C.

Initially I was disappointed to see that there weren’t many resources to get started in iOS from a beginner’s perspective with Swift. Then after spending some more time searching I found that Stanford University has a free course on iTunes university titled Developing iOS 8 Apps with Swift. This has been an immensely useful resource. Paul Hegarty is a great instructor. He is really good at explaining the concepts and demonstrating things in real time. I highly recommend checking it out.

These are a few of the important things I’ve learned so far:

• Read the iOS Human Interface Guidelines This may go without saying but Apple has done a ton of research behind the user experience of iOS and its native applications. A fun fact that I learned was the loading screen for an iOS application shouldn’t be used as a splash page. Instead, it should be a representation of the inactive application to give the illusion of a quick load.

• Have a project idea While reading and following the Stanford online course was immensely useful to start learning, I always feel that I need a concrete project to work on to apply my learnings. Without this, I usually can’t keep sustained interest. I recommend coming up with a simple app idea to work on while learning.

• Read through Apple’s developer documentation While developing for iOS you have a large number of frameworks provided by Apple at your disposal. Apple also provides excellent documentation around all of these frameworks; XCode even has a documentation browser built in. I recommend taking the time to familiarize yourself with these documents and learn to navigate them, it will be a huge benefit in the long run.

• Use the Apple developer forums I wasn’t immediately aware of how useful the Apple developer forums would be. I was running into trouble getting an archive of my app sent to iTunes connect and spent hours if not days searching for support. Most of my searching turned up various StackOverflow posts which were not helpful, but after checking in the thoughtbot XCode slack channel I found that someone else had recently run into a similar issue. They lead me to the specific post on the Apple developer forums regarding the issue and I was able to work around it in a few minutes. Lesson I learned; search the Apple developer forums.

Recommended Reading

• Swift Programming Series (iBooks Store) This is the offical Swift book by Apple. It is a great resource on getting started with learning swift.
• Beginning iPhone Development with Swift This is a great book to get started with learning iOS and Swift.
• RayWenderlich Great blog with tutorials on iOS and Swift
• AppCoda Great resource for various tutorials
• Giant Robots Smashing into Other Giant Robots There may not be as many beginner content here but there has been very useful blog posts on Swift and functional programming in Swift.

Design Resources

• The iOS Design Guidelines Guide to design apps around the Apple HIG.
• iOS 8 GUI PSD (iPhone 6) Template of the GUI elements found in iOS 8
• Using Vector Images in XCode 6 Turns out you no longer have to generate several versions of an image asset to use within your app. XCode supports vector images which it will scale appropriately for various sizes. IE: @1x, @2x, @3x

The Rails fresh_when method is a powerful tool for conditionally caching resources via HTTP. However there are some pitfalls. For one, fresh_when only supports the default render flow in a controller; if a client’s cache is not fresh, it will just render the related view. We cannot utilize things like render json:.

Fortunately, Rails provides us with more tools to work with HTTP conditional caching. Some of the basics behind HTTP conditional caching are assumed in this post. If you haven’t already, or you just need a refresher take a look at Introduction to Conditional HTTP Caching with Rails.

Let’s work with a simple Rails blog application which renders posts as JSON.

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  def show
    @post = Post.find(params[:id])
    render json: @post
  end
end

# config/routes
Rails.application.routes.draw do
  resources :posts, only: [:show]
end

What if we wanted to allow the client to conditionally cache this content? Rails provides us with the stale? method which takes a resource and checks to see if the ETag provided by the client via the If-None-Matches header matches the one which represents the resource’s current state. The stale? method can also work with the If-Modified-Since header. It will check if the resource’s updated_at has been modified since the timestamp provided by If-Modified-Since. To utilize this, we just need to wrap our render call with a call to stale?.

class PostsController < ApplicationController
  def show
    @post = Post.find(params[:id])

    if stale?(@post)
      render json: @post
    end
  end
end

If we make a request to the individual post we’ll get something similar to the following response:

curl -i http://localhost:3000/posts/1

HTTP/1.1 200 OK
Etag: "f049a9825a0239db3d01ead65a5ea522"
Last-Modified: Tue, 02 Dec 2014 18:31:42 GMT
Content-Type: application/json; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
Server: WEBrick/1.3.1 (Ruby/2.1.5/2014-11-13)
Date: Tue, 02 Dec 2014 18:31:48 GMT
Content-Length: 93
Connection: Keep-Alive

{"post":{"title":"Blog post 0","body":"lorem ipsum","created_at":"2014-11-12T15:30:34.025Z"}}%

If we take a look at our Rails server log, we’ll see something similar to the following:

Started GET "/posts/1" for 127.0.0.1 at 2014-12-02 13:38:18 -0500
Processing by PostsController#show as */*
  Parameters: {"id"=>"1"}
  Post Load (0.2ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."id" = $1
LIMIT 1  [["id", 1]]
Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.2ms)

Now if we make a request with the If-None-Match header set to the ETag that the server provided us with earlier:

curl -i -H 'If-None-Match: "f049a9825a0239db3d01ead65a5ea522"' http://localhost:3000/posts/1

We’ll now receive a 304 Not Modified status for our request.

HTTP/1.1 304 Not Modified
Etag: "f049a9825a0239db3d01ead65a5ea522"
Last-Modified: Tue, 02 Dec 2014 18:31:42 GMT
Cache-Control: max-age=0, private, must-revalidate
Server: WEBrick/1.3.1 (Ruby/2.1.5/2014-11-13)
Date: Tue, 02 Dec 2014 18:32:53 GMT
Connection: Keep-Alive

If we take a look at our Rails log we’ll notice that the server skipped over the process of rendering the JSON all together:

Started GET "/posts/1" for 127.0.0.1 at 2014-12-02 13:38:55 -0500
Processing by PostsController#show as */*
  Parameters: {"id"=>"1"}
  Post Load (0.2ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."id" = $1
LIMIT 1  [["id", 1]]
Completed 304 Not Modified in 1ms (ActiveRecord: 0.2ms)

In this trivial example our gain is not huge. However with more complicated JSON objects this can be a very useful. A common example is a larger JSON response built via ActiveModel::Serializers. Combined with key-based caching in ActiveModel::Serializers this can be a powerful tool. We can cache individual items with russian doll caching, and then cache the entire response via HTTP caching.

Further Reading

• Fast JSON APIs in Rails with Key-Based Caches and ActiveModel::Serializers
• How to Evaluate Your Rails JSON API for Performance Improvements

HTTP provides developers with a powerful set of tools to cache responses. Often times we don’t want a client to blindly cache content that it has been given. We may not be able to rely on setting specific expiration headers either. Instead we need a way for the client to ask the server whether or not a resource has been updated.

HTTP provides us with the ability to do this with conditional caching. A client can make a request to the server and find out of the server has a new version of the resource available.

Rails provides us with tools to take advantage of this. Before we jump into Rails, let’s discuss some of the common ways to check if a client’s cache is fresh or stale.

ETags

ETags, short for entity tags, are a common way to conditionally verify an HTTP cache. An ETag is a digest which represents the contents of a given resource.

When a response is returned by the server it will include an ETag to represent the resource’s state as part of the HTTP response headers. Subsequent HTTP requests which want to know whether or not the resource has changed since the last request can send along the stored ETag via the If-None-Match header.

The server will then compare the current ETag for the resource to the one provided by the client. If the two ETags match, the client’s cache is considered fresh and the server can respond with a “304 Not Modified” status.

If the resource has changed since the last time the client has requested the resource the server will respond with a new ETag and the updated response.

Last-Modified Timestamp

Another header that a server can supply is the Last-Modified header.

As the name implies, this will return a timestamp of when the resource was last modified. The client can then include an If-Modified-Since header with the Last-Modified timestamp value to ask the server if the resource has been modified since the previous request.

Again, if the resource has not been modified since the last request, the server can respond with a “304 Not Modified” status. Otherwise, it will return an updated representation of the resource and a new Last-Modified timestamp for next time.

Caching with fresh_when

Now that we’ve discussed the ETags and Last-Modified HTTP headers, let’s take a look at how we might use them within our Rails application.

ActionController provides us with a powerful method called fresh_when. It will use either an ETag or a Last-Modified timestamp to determine whether or not a resource is fresh or stale.

Imagine a simple Rails blog application with the following posts route / controller:

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  def show
    @post = Post.find(params[:id])
  end
end

# config/routes.rb
Rails.application.routes.draw do
  resources :posts
end

If we made a curl request to an individual post we might see something like:

curl -i http://localhost:3000/posts/1

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Etag: "4af7e17fc369c6d1af99f8994d8fd387"
Server: WEBrick/1.3.1 (Ruby/2.1.2/2014-05-08)

<!DOCTYPE html>
<html>
  <body>
    <h1>Blog post 1</h1>
    <p>lorem ipsum</p>
  </body>
</html>

Notice that we have not done anything particularly noteworthy, yet if we take a look at the headers returned as a part of the response we’ll see that Rails has sent along an ETag to represent the resource being returned.

If we keep making the same request via curl we’ll see a different ETag value even though the resource has not been modified since our previous request. This goes against the notion of what an ETag would be used for.

We’ll need to modify our controller to instruct it that a request is fresh when a post has not been modified since the last request.

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  def show
    @post = Post.find(params[:id])

    fresh_when @post
  end
end

Now if we make our previous request over again we’ll see that our ETag no longer changes on repeated requests:

curl -i http://localhost:3000/posts/1

Content-Length: 667
Content-Type: text/html; charset=utf-8
Etag: "534279bfc931d4236713095ffd3efb28"
Last-Modified: Wed, 12 Nov 2014 15:44:46 GMT
Server: WEBrick/1.3.1 (Ruby/2.1.3/2014-09-19)

Upon closer inspection of the updated HTTP headers returned in the response, we’ll notice that the server has also started returning a Last-Modified timestamp for the post resource.

Our work is not complete.

In order to use our cache we need to verify it against the server with the ETag or last modified timestamp with either the If-None-Match or If-Modified-Since headers respectively.

curl -i -H 'If-None-Match: "534279bfc931d4236713095ffd3efb28"' http://localhost:3000/posts/1

HTTP/1.1 304 Not Modified
Etag: "534279bfc931d4236713095ffd3efb28"
Last-Modified: Wed, 12 Nov 2014 15:44:46 GMT
Server: WEBrick/1.3.1 (Ruby/2.1.3/2014-09-19)

Notice we now receive a 304 Not Modified response, instructing our client that it can use its cached version of the content. Browsers will automatically send a long a stored ETag and Last-Modified timestamp with conditional caching headers for us so we do not actually have to do anything to use this in the browser.

Let’s take a look at the similar request with the If-Modified-Since header:

curl -i -H 'If-Modified-Since: Wed, 12 Nov 2014 15:44:46 GMT' http://localhost:3000/posts/1

HTTP/1.1 304 Not Modified
Etag: "534279bfc931d4236713095ffd3efb28"
Last-Modified: Wed, 12 Nov 2014 15:44:46 GMT
Server: WEBrick/1.3.1 (Ruby/2.1.3/2014-09-19)

You’ll notice that we get the same results, the post has not been modified since the last time and we receive a 304 Not Modified response.

Conclusion

What have we gained through all of this? We are still making requests to our server to see if a resource has been modified before rendering.

To really understand what we’ve achieved we can take a look at our Rails log before we had caching:

Started GET "/posts/1" for 127.0.0.1 at 2014-11-12 11:33:57 -0500
Processing by PostsController#show as */*
  Parameters: {"id"=>"1"}
  Post Load (0.3ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."id" = $1
LIMIT 1  [["id", 1]]
  Rendered posts/show.html.erb within layouts/application (0.1ms)
  Rendered application/_flashes.html.erb (0.0ms)
  Rendered application/_analytics.html.erb (0.1ms)
  Rendered application/_javascript.html.erb (16.1ms)
Completed 200 OK in 35ms (Views: 33.4ms | ActiveRecord: 0.3ms)

We can see that our request takes about 33ms and has to render our posts/show template within the layout along with some other partials. We can also see that most of our time is spent within that rendering process. ActiveRecord took less than a millisecond to run.

Next, let’s take a look at our log with conditional caching in place, and put it to use by sending over a If-None-Match header:

Started GET "/posts/1" for 127.0.0.1 at 2014-11-12 11:39:52 -0500
Processing by PostsController#show as */*
  Parameters: {"id"=>"1"}
  Post Load (0.3ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."id" = $1
LIMIT 1  [["id", 1]]
Completed 304 Not Modified in 2ms (ActiveRecord: 0.3ms)

Notice we’ve cut off our entire rendering process from the request. Since the client’s cached response is fresh there is no reason for Rails to go through the rendering process. It can load in the post via ActiveRecord and verify if it has changed since the previous request.

Our requests went from taking about 33ms to complete to 2ms. That’s about a 94% improvement in performance. Not only is Rails skipping the rendering process, but our responses are now much lighter as they have no body to return.

This may not be much in our trivial example but for applications which have a lot of complicated logic behind rendering this can save lots of time.

Furthermore, these requests can be put in front of a public cache such as Rack::Cache to share a common cache for resources as opposed to a private cache which is only utilized by the user’s browser.

Getting more

Our simple fresh_when solution is not without its limitations.

It cannot handle user specific content. If we wanted to display anything different on a per-user basis (even if that’s the name in the header), we won’t be able to cache the content.

Another issue is that the cache is not being shared among users. Our performance gain is only realized when a single user looks at the same post over and over again.

Lastly, we’re limited to a simple render call in our controller. What if we wanted something more customized like render json: or render xml:? We’ll build on top of what we’ve learned next time to overcome these limitations and improve our caching even further.

Organizing our API

Building our API we set out to achieve a few goals:

• Make the API flexibile and decoupled from the front-end implementation
• Leverage HTTP caching whereever possible
• Ensure it could handle high scale

Making the API decoupled from it’s front-end implmentation ensured that any design or data usage changes made would not require a change in 2 areas of our code base, the JavaScript application and the Rails application / API.

Leveraging HTTP caching gave us 2 gains. The first is we wanted to leverage an individual user’s caching in their browser, reducing the amount of data we needed to send back to the user. The 2nd gain, which in my opinion was the bigger gain was the ability to add a caching layer in front of the entire response body through Rack::Cache.

ActiveModel::Serializers

(ActiveModel::Serializers)[https://github.com/rails-api/active_model_serializers] is a great library for encapsulating object serialization for JSON APIs in Rails. It provides a great Object Oriented approach to serialization. If you haven’t seen it before I highly recommend checking it out. Furthermore, it implements caching through Rails.cache to provide the ability to cache serialization of objects using (key based expiration)[http://37signals.com/svn/posts/3113-how-key-based-cache-expiration-works]. This provided us with some very nice, quick and easy caching of our serialized data. Thoughtbot has a great (article on ActiveModel::Serailziers and caching)[http://robots.thoughtbot.com/post/50091183897/fast-json-apis-in-rails-with-key-based-caches-and] so I’m not going to get into the details of it.

With this caching we were able to get a great method of caching our serialization. However we wanted more. Our models have a lot of associations which can be re-used through Russian doll caching. We implemented (this pull request)[https://github.com/rails-api/active_model_serializers/pull/307] in or serializers in order to benefit from side-effect free russian doll caching. With this in mind we could reuse caches just as easily across requests when possbile.

Rack::Cache

Leveraging Rack::Cache in front of the API was one of the biggest wins for us. Rails has Rack::Cache active in production mode by default. However if you aren’t using HTTP caching properly you won’t see the benfits. There are many articles on the details of the various HTTP caching mechanisms that can be used. From cache-control headers, to etags and last-modified time stamps. Before we jump into the details let me explain why Rack::Cache was so useful.

When leveraging russian doll caching for our serialization we were able to get very fast serialization for our objects. However this typically lead to lots of memcached requests for fetcihng the individual pieces of the cache. This was not optimal. Our architecture benefited from the fact that one parent object could be used to determine the freshess of a cache. Think of the classic todo list example. Where individual todos in a list are cached as part of the entire todo list. If any individual todo-item in the list expires, then the list itself is expired. So, if we were using ActiveModel::Serialziers with caching, it would hit a cache for each item in the todo list.

Outline

• Weeks JSON public to all users
• Picks public because it’s route is built from user ID.
• ActiveModel::Serializers
• Rack::Cache

• Smoother Experience CSS transitions offer a smoother experience when compared to JavaScript animations. Since everything is being handled natively by the browser engine, we can get much smoother transitions.
• Mobile Performance This subcategory to the first note. While CSS transitions will most always provide a smoother experience in regular browsers, mobile devices will benefit from this the most. Since mobile devices lack the power that our desktops have, taking advantage of the browser engine for transitions is a must.

Gracefully Degrading

In most cases front end developers will err on the side of progressive enhancement. That is, to provide users who support newer technologies an experience with these technologies and user’s who do not support it will be left behind, in a usable state just without the bells and whistles. I consider this following method to be graceful degradation instead. The reason behind this is that we will not be removing the transition effects all together from the page, but rather the method that we are using to apply the transition will be different, degrading to JavaScript animations when CSS transitions are not available (More on this later).

Detecting our features

Moving forward with our graceful degradation, we must determine how to apply the transition effect for the user. This should almost never be done via browser agent detection but rather through feature detection. Instead of only serving CSS transitions to certain versions of a browser we should instead detect whether or not the user supports our feature, in this case, CSS transitions. This can be done through JavaScript early on before trying to transition. One option is to use the open source JavaScript library Modernizr. Modernizr will detect the HTML5 and CSS3 features that the browser has and you have the option to populate your HTML tag with classes which describe support. Along with these CSS classes, a Modernizr object is created which you can check with JavaScript to see features that are supported. Modernizr even supports the ability for you to customize the features it detects. If you plan on using a lot of feature detection, then I highly recommend using Modernizr.

However, in some cases you may not need all of the features that Modernizr provides and it would be wasteful to add the bloat it provides or we simply may not be able to add Modernizr to our page. In this case we can detect features ourselves. For our transitions example we will need to detect whether or not a CSS3 property is available in the user’s browser. Since as of this writing the transition property is prefixed by the vendor we will need to run through and check each vendor prefix version of the transition property to see if the users’ browser supports it or not.

// Check for CSS3 Transition Support
var doc = document;

var css_transitions = function() {
    var el = doc.createElement('div');
    var vendors = ['', 'Khtml', 'Ms', 'Moz',' Webkit','O'];

    for (var i = 0, len = vendors.length; i < len; i++) {
        var prop = vendors[i] + 'Transition';
        if (prop in el.style) return true;
    }

    return false;
}();

First, we create a test element in memory to test against. Then we create an array of vendor prefixes ['', 'Khtml', 'Ms', 'Moz', 'Webkit', 'O']. Note that our first entry is a blank string. This is so that when browsers have fully implemented the transition spec and vendor prefixes are no longer used, we detect the feature correctly. Now we loop through the array. For each iteration we create a property to test which is a combination of the vendor prefix and our base property, in this case Transition. So for example. we will test MozTransition. We test to see if the property exists as part of the test elements style property. If this comes back true, then our browser supports CSS transitions. In the case of our example we execute our function immediately and store the result in a variable called css_transitions.

Putting our css_transitions variable to use

Now that we’ve determined whether or not the user’s browser supports CSS transitions we need to go put it to work. In this case, let’s transition the opacity of an element:

<style type="text/css">
    .transition-me {
        -o-transition(opacity 0.25s linear 0s);
        -moz-transition(opacity 0.25s linear 0s);
        -webkit-transition(opacity 0.25s linear 0s);
        -ms-transition(opacity 0.25s linear 0s);
        transition(opacity 0.25s linear 0s);
    }
</style>

In this example we will use jQuery which provides us with a rather handy syntax for changing CSS properties on an element or calling an animation on an element. The syntax for each actually matches exactly when used which allows us to use a pretty simple idiom to transition with.

// Fade out .transition-me elements
var method = (css_transitions) ? 'css' : 'method';
$('.transition-me')[method]({
    opacity: 0
});

If you visit the jQuery documentation both the css and animate support a map syntax which allows us to send an object to the method call with a property we want to transition as the key and the value to transition to as the value. With this in mind, we can simply call the appropriate method (css or transition) on the element we want to transition.

A note on Progressive Enhancement

While this particular post discusses how to apply graceful degradation to CSS transitions there is still room for some progressive enhancement. What I mean by this is there are certain times where a transition or animation is key to the user’s experience. Transitions help guide the user through changes on the page. There are times however where transitions are just applied to add a certain level of eye candy. It is this class of transition that I would simply not provide in browsers which do not support it since they are not key to the user’s experience with the page.

Case Study

So why the image map? Take a look at the following image:

So our image above is triangular, and is intended to be clickable. The expectation is that only the triangular area should be clickable. So in order to accomplish this, I’ve decided to move forward with using an image map.

Things get a bit more complicated, though. The image above is actually repeated many times on a page. It acts as a flag to being a favorite of sorts. Furthermore, we are not simply sending the user to a page link, we are looking to use JavaScript events on the image map (and unobtrusively at that). This unobtrusive JavaScript must send an ajax request to the server depending on which instance of the image is actually being clicked. So we have our work cut out for us.

Setting up the image map

Getting the image map set up is pretty simple. Let’s start with defining the map:

<map name="my-map">
    <area id="fav-area" shape=poly, coords="0,0, 46,0, 46,46" href="#fav"> 
</map>

We’ve defined a map named my-map with some coordinates. Note our shape is set to poly and the coordinates provided. In order to determine these coordinates I simply opened up Photoshop and grabbed the x and y coordinates of the 3 corners of the triangle. Along with that, since we have no intent on passing the user to an actual link we’ve provided a dummy hash on the href.

Setting up the image

So now the the map is defined we’ll create an instance of the image and have it use the map we made:

<img src="/images/map-case-study.gif" usemap="#my-map">

Adding an event handler

Adding an event handler is pretty straight forward as you might expect. I’ll be using jQuery for the demonstration.

$(function() {
	$('map[name=my-map] #fave-area').click(function (e) {
		e.preventDefault();
		alert('Clicked!');
	});
});

So we can still target and add an event as you normally would to the area.

Multiple instances of the same image and image map

Now that we have our image map set up, let’s move forward with the more complicated setup. Take the HTML structure below:

<div id="node-1" class="my-node">
	<img id="study-1" src="/images/map-case-study.gif" usemap="#my-map">
</div>

<div id="node-2" class="my-node">
	<img id="study-2" src="/images/map-case-study.gif" usemap="#my-map">
</div>

<div id="node-3" class="my-node">
	<img id="study-3" src="/images/map-case-study.gif" usemap="#my-map">
</div>

<map name="my-map">
	<area id="fav-area" shape=poly, coords="0,0, 46,0, 46,46" href="#fav">
</map>

We have multiple nodes which aim to make use of the same case-study image and they all use the same image map to define the clickable area. Now the problem is, how do we determine which image was actually clicked?

Well first, we can make use of a DOM method called elementFromPoint. This method allows you to pass in x and y coordinates and retrieve the element that is found at the given coordinates. I never knew that this method existed until today and it’s a pretty interesting find.

Putting elementFromPoint to use

$(function() {
	$('map[name=my-map] #fave-area').click(function (e) {
		e.preventDefault();
		var trigger = document.elementFromPoint(e.clientX, e.clientY);
		alert(trigger.id);
	});
});

So we take the coordinates of the actual click event and pass it to the elementFromPoint function to grab the actual element the user was trying to click. One problem still exists though: This works fine in Firefox and Safari but Chrome (and IE as some have reported) will still return the map area when using elementFromPoint. In order to counter this, I’ve found that disabling the image map before running elementFromPoint will then yield our expected element. Of course, once we’re done finding the element we must re-enable the image map.

So how do we do this? Take a look at the final JavaScript:

$(function() {
	$('map[name=my-map] #fave-area').click(function (e) {
		e.preventDefault();
		
		// Cache all instances of the image
		var $all_instances = $('.my-node img');
		
		// Store the usemap in order to re-enable later
		var usemap = $all_instances.attr('usemap');
		
		// Reset the use of an image map on all instances of it
		$all_instances.attr('usemap', '');
		
		// Grap the actual trigger we were looking for
		var trigger = document.elementFromPoint(e.clientX, e.clientY);
		
		// Re-enable the image map on all the instances we disabled it on
		$all_instances.attr('usemap', usemap);
		
		// Do something useful with the information
		alert(trigger.id);
	});
});

The sample has moved to using the background-image property instead of the background property alone. This change has been made to resolve issues that occur when defining a background with the CSS based gradient but leaving out the other properties common to a background such as position or repeat. When these attributes are left out, browsers define default values which can yield unexpected results with the CSS gradients.

So, the following:

background: -webkit-gradient(
    linear,
    left bottom,
    left top,
    color-stop(0.33, rgb(99,168,125)),
    color-stop(0.67, rgb(129,202,163)),
    color-stop(0.84, rgb(155,243,196))
);

background: -moz-linear-gradient(
    center bottom,
    rgb(99,168,125) 33%,
    rgb(129,202,163) 67%,
    rgb(155,243,196) 84%
);

Becomes,

background-image: -webkit-gradient(
    linear,
    left bottom,
    left top,
    color-stop(0.33, rgb(99,168,125)),
    color-stop(0.67, rgb(129,202,163)),
    color-stop(0.84, rgb(155,243,196))
);

background-image: -moz-linear-gradient(
    center bottom,
    rgb(99,168,125) 33%,
    rgb(129,202,163) 67%,
    rgb(155,243,196) 84%
);

The live sample on the page also makes use of background-image as well, so you can view that as another reference to this update as well.

Thanks to Avi Mahbubani for pointing out the issue and helping come to a resolution.

This sample is going to be using the paperclip gem by thoughtbot to handle our upload.

The Controller

So we’ve gotten our file uploaded successfully, authentication is passing thanks to our previous work but what now? One of the first issues that you might run into is the content type for the file you uploaded. Our uploadify upload will actually have the content_type set to ‘application/octet-stream’ which is not totally accurate representation of what we’re uploading.

As stated earlier, this example is using paperclip, which will need the true content_type to be set to accurately continue uploading the file. To resolve the ‘application/octet-stream’ issue we can use the mime-type gem and set the content_type of the file afterwards in the controller before trying to save the upload. Add the following to your gem file and run a bundle install:

gem 'mime-types', :require => 'mime/types'

With this in place we can force our controller to set the content type of the upload to the proper value.

We can do this with the following code:

params[:Filedata].content_type = MIME::Types.type_for(params[:Filedata].original_filename).to_s

Below is a full code sample of our Paperclip based model, and the full create controller method for handling our upload. I hope you’ve found this post useful and please feel free to comment below with any questions, as I’ll try to get to them as soon as possible.

GalleryImage Model

class GalleryImage < ActiveRecord::Base

	belongs_to :gallery
	delegate :venue, :to => :gallery

	# Storing photos in /galleries/:venue_name/:gallery_id/...
	has_attached_file :photo,
		:styles => { :thumb => "70x70", :iphone_thumb => "30x30#", :iphone_full => "320x480" },
		:url => '/galleries/:venue/:gallery/:id/:attachment/:style/:basename.:extension',
		:path => ':rails_root/public/galleries/:venue/:gallery/:id/:attachment/:style/:basename.:extension'
end

GalleryImagesController

def create

	@gallery = Gallery.find(params[:gallery_id])
	@gallery_image = GalleryImage.new
	@gallery_image.gallery = @gallery

	# Associate the correct MIME type for the file since Flash will change it
	params[:Filedata].content_type = MIME::Types.type_for(params[:Filedata].original_filename).to_s

	@gallery_image.photo = params[:Filedata]
    if @gallery_image.save
		render :json => { 'status' => 'success'  }
	else
		render :json => { 'status' => 'error' }
	end
end

Well that is until you’re trying to work with a Rails application. There are lots of resources around that go into the details of how to get this to work but the real issue that I had was that they were mostly for Rails 2.3.8 and my application is now using Rails 3 and the methods described in all of these articles don’t completely apply to Rails 3.

Why is this extra work needed?

Before moving forward on how to get everything working let’s take a look at why it’s not working in the first place to better understand what we’re dealing with and how we’re fixing the problem.

Requests made via POST, PUT, DELETE all require a valid authentication token and session information in order to complete the request successfully in order to protect against Cross-site request forgery. When building out forms using Rails form helpers, a hidden input is created which contains the authenticity token to prove the authenticity of the request along with your session cookie data.

This poses 2 problems for us using Flash to upload files.

First, Uploadify is not going to automatically send the authenticity token with the POST request. Second, Flash based requests do not send our session information along with the request. The first situation is pretty easy to fix while the session information requires a little more work on our part.

Sending the Authentication_Token

Let’s start by sending the authenticity token along with the request. In previous versions of Rails we would have had to found a way to get the authenticity token to send but as part of Rails 3 usage of Unobtrusive JavaScript, it now includes a new helper method called csrf_meta_tag. This helper method will provide us with 2 meta tags, csrf-param and csrf-token

<meta name="csrf-param" content="authenticity_token"/>
<meta name="csrf-token" content="s2yuw7u24N9fni1m+Wynr595kTphEeMrNSWPAnMroLM="/>

Once we have the CSRF meta tags in place, we can move forward and add the authenticity to the parameters which are sent with Uploadify’s request. The JavaScript snippet below is an example of pulling the CSRF token information and passing it along with our Uploadify requests using the scriptData configuration parameter.

<script type="text/javascript">
$(function () {
  // Create an empty object to store our custom script data
  var uploadify_script_data = {};

  // Fetch the CSRF meta tag data
  var csrf_token = $('meta[name=csrf-token]').attr('content');
  var csrf_param = $('meta[name=csrf-param]').attr('content');

  // Now associate the data in the config, encoding the data safely
  uploadify_script_data[csrf_token] = encodeURI(csrf_param);

   // Configure Uploadify
  $('#photo-upload').uploadify({
    'uploader' : '/assets/uploadify.swf',
    'script' : '/photos/',
    'scriptData' : uploadify_script_data
  });
});
</script>

Uploadify allows you to pass extra data along with it’s request using the scriptData parameter which accepts an object of properties to send with the request. In our case we are fetching the CSRF and creating a new property in the scriptData object with the name of the CSRF token and setting it’s value to the CSRF param. Using this, we can pass along the authenticity token with the request. It’s important to note that we are encoding the authentication token value using encodeURI to escape any special characters so the data gets sent along to the server correctly.

I actually ran into an issue where even using encodeURI once was not enough and I needed to double encode because + signs were being converted into spaces. So if you are continuing to run into InvalidAuthenticityToken error, check your log to see that the authenticity token is being correctly sent along. While trying to debug the problem a quick look into the log showed that I was receiving

authenticity_token=>"3WB Cj0h7y4St5ZyvlYU xIEVT7yIIr6IRAhOOLCb5w=

The spaces in the above authenticity token should have been + signs instead. In order to resolve this issue I ended up having to double encode the authenticity token before sending it along like so:

uploadify_script_data[csrf_token] = encodeURI(encodeURI(csrf_param));

Now that we have the authenticity token being sent by Uploadify, let’s move forward and get our session information passed along as well.

Passing the Session Cookie

One of the first steps to passing the session cookie along with our Uploadify requests is to add it to the scriptData like we did with the authentication_token. Unfortunately there is no quick way to handle this through JavaScript alone, we need to include it using some ERB to apply it.

In earlier versions of Rails the session cookie key could be accessed via: ActionController::Base.session_options[:key]

As of Rails 3 though we can access the session cookie key using: Rails.application.config.session_options[:key]

Below is a simple example of how to send the session cookie information in the scriptData.

<script type="text/javascript">
<%- session_key = Rails.application.config.session_options[:key] -%>
$(function () {

  var uploadify_script_data = {};

  // Fetch the CSRF meta tag data
  var csrf_token = $('meta[name=csrf-token]').attr('content');
  var csrf_param = $('meta[name=csrf-param]').attr('content');

  // Now associate the data in the config, encoding the data safely
  uploadify_script_data[csrf_token] = encodeURI(csrf_param)

  // Associate the session information
  uploadify_script_data['<%= session_key %>'] = '<%= cookies[session_key] %>';

  $('#upload-photo').uploadify({
    script : '/photos/',
    scriptData : uploadify_script_data
  });
});
</script>

Now that we have the session cookie data being passed with the request we’ll need to move forward and have that information set with the headers of the Flash request, enter Middleware.

Rack Middleware

In order to get the session information to be associated with the request correctly we’ll need to create some custom middleware which will intercept the requests from Flash and inject the cookie information into the header.

Some middleware has been around for accomplishing this but most are not compatible with Rails 3. I’ve posted a Gist on Github which has a working middleware for Rails 3 which I found on metautonomo.us which we can use to accomplish our task.

Create a file in app/middleware/flash_session_cookie_middleware.rb creating the middleware directory if it doesn’t already exist and add the following code in it:

require 'rack/utils'
 
class FlashSessionCookieMiddleware
  def initialize(app, session_key = '_session_id')
    @app = app
    @session_key = session_key
  end
 
  def call(env)
    if env['HTTP_USER_AGENT'] =~ /^(Adobe|Shockwave) Flash/
      req = Rack::Request.new(env)
      env['HTTP_COOKIE'] = [ @session_key,
                             req.params[@session_key] ].join('=').freeze unless req.params[@session_key].nil?
      env['HTTP_ACCEPT'] = "#{req.params['_http_accept']}".freeze unless req.params['_http_accept'].nil?
    end
 
    @app.call(env)
  end
end

Once you’ve created that file we’ll need to add the middleware directory to our autoload’s path if it isn’t already. You can do this by modifying you’re config/application.rb file and add the following in the autoload section:

# Custom directories with classes and modules you want to be autoloadable.
# config.autoload_paths += %W(#{config.root}/
config.autoload_paths += %W(#{config.root}/app/middleware/)

Once that’s done we have one more step to do, inserting the middleware so that it will go to work when we need it.

Place the following code in config/initializers/session_store.rb

Rails.application.config.middleware.insert_before(
  ActionDispatch::Session::CookieStore,
  FlashSessionCookieMiddleware,
  Rails.application.config.session_options[:key]
)

Update

If you are using ActiveRecord based session storage instead of cookie based session storage you will need the following intializer instead:

SampleApp::Application.config.session_store :active_record_store, :key => '_uploader_session'

Rails.application.config.middleware.insert_before(
  Rails.application.config.session_store,
  FlashSessionCookieMiddleware,
  Rails.application.config.session_options[:key]
)

Thanks to reader @fromthought2web for sharing his findings!

With all this in place you should now be ready to successfully upload files via Flash with your Rails project.

Good luck!

Updates

2012-04-03

Included sample middleware when using ActiveRecord for session store, thanks to @fromthought2web for sharing his finds.

<dt>2010-08-26</dt>
<dd>Resolved issue in code sample where session_key_name was being reference instead of session_key. Original sample updated</dd>
<dd>`uploadify_script_data['<%= session_key %>'] = '<%= cookies[session_key] %>';`</dd>

While researching on the topic over the last few weeks, I discovered what I believe is a misconception of an iPhone web application, and my goal is to hopefully, clear things up.

Let’s start by defining some of the possible ways of targeting an iPhone user, which actually diverges into two separate categories of its own. The first and probably the most popular way of targeting an iPhone user is through an iPhone app, sold through the iTunes App store. Coding and compiling Objective C/Cocoa based native iPhone applications.

Yet there are still other ways of creating a custom user experience for a user on the iPhone.

Mobile Web Sites and Web Applications

With Smart Phones becoming so widespread companies are working harder towards providing users with access to their websites with mobile phone designed websites.

WAP (Wireless Application Protocol) sites have been around for several years; the WAP was established in 1998 and was created for less powerful, smaller phones that had limited web browsers. Today’s smart phones, however, are becoming more and more powerful and now provide users with a fully fledged web browser such as the iPhone’s mobile Safari.

So with this in mind, websites have two options. Leaving things alone and letting the phone’s browser do the work, which may work well in some cases; or create a smart phone targeted version of their website. One of the biggest issues though with leaving a website as is for mobile device users is the bandwidth that a user has. Sure, the page may render correctly and work just fine, but that doesn't mean it may not take a while to load all the resources used on the page. Many of those same resources just aren't needed on the mobile device.

Through the power of HTML 5, CSS 3, Javascript and CSS Webkit extensions a developer has incredible tools at their disposal to work with in order to make a mobile friendly page.

Webkit

One of the things I am most impressed with so far is the power that CSS3 and webkit extensions provide. More specifically, CSS based gradients, shadows, and animations. These are really amazing pieces of technology allowing to cut down on bandwidth issues created from using images for certain elements just because of a gradient or other effect, or eliminating the need of Flash or special javascript libraries for animations.

Let's take a closer look at the benefits of using CSS gradients to cut down on the number of images that are required to download. Perhaps you have a consistent gradient used on several buttons on a site. If you were to use images, you would have to re-cut each variation of button, which leads to longer page loads which is a major concern when working on a mobile device. Now with a CSS gradient, we can define it as a single class and apply it to any buttons that use the gradient, making much better use of our resources.

Finally, with these tools in hand we can diverge into two sections, the general purpose mobile website, or an iPhone web application. Now by no means do I mean to downplay the power of the mobile website, I simply want to talk about something that’s less known, and that’s the iPhone Web Application.

So what is an iPhone Web Application

An iPhone Web Application in its simplest form is actually a website which is created with the same technologies web developers are already familiar with, but are created with the goal of emulating a native application on the iPhone.

HTML5, CSS3, Javascript, and CSS Webkit extensions provide a developer with rich tools to create power applications for an iPhone. Users can bookmark a web page to their home screen and right there, they can see one of the first signs of a web application, a custom home screen icon. Just like the application icon created for the home screen for a native iPhone application, through the use of Meta tags a developer can define a custom icon to be shown on the home screen for their web application. Furthermore, the developer can define another Meta tag which can hide the whole Safari UI including the address bar, and the navigation bar at the bottom of the screen. These are only some of the simpler things a developer can do.

Through javascript alone a developer has access to many of the events that an iPhone application developer would have, such as touch events, gesture events orientation changes, viewport settings and more.

Furthermore, the local storage engine, powered by SQLite, allows local databases to be created from a web application, which can be accessed through javascript. A popular example of local database storage is Gmail's offline mode, which uses the local storage to keep emails stored locally.

Not only can we create a local database on the user’s phone, we can work with more powerful caching functionality through the use of Manifest files which we can use to define a set of files to cache on the phone which can then be accessed later if the user tries to access the web application without an internet connection. So a developer can specify to cache CSS files and images to make sure the user enjoys the same UI experience while offline just like a native application. Manifest caches and the SQLite local databases are what I consider some of the most important tools at a developer's belt when writing an iPhone web application because it really blurs the line between an iPhone application and an iPhone web application.

Stay tuned as I plan to get into some more specifics relating to all aspects of iPhone web application development.