Design an AI agent that discovers, evaluates, and curates content at scale
Content agents power recommendation systems, search platforms, and knowledge bases at scale. They combine retrieval, evaluation, and synthesis to find and present relevant information.
In this lab, you'll create:
You're building a content agent for a news aggregation platform. The agent ingests thousands of articles daily, evaluates them against user interests and credibility standards, and ranks the top stories for each user.
The challenge: Your agent is flooding users with feel-good stories because they're easy to find and highly shareableâbut missing critical hard news that users actually need. Meanwhile, some credible but niche sources are getting filtered out entirely.
The question: How do you design an agent that balances engagement metrics with editorial integrity?
Retrieval Stage: The agent must search a content corpus using user interests as queries. But broad searches return too much; narrow searches miss important context.
Evaluation Stage: Scoring content for relevance, credibility, freshness, and bias is complex. Each dimension conflicts with the others sometimes.
Ranking & Filtering: The agent decides which content makes the final cut. Too strict a filter? Users miss important stories. Too loose? Content quality plummets.
Feedback Loop: User engagement signals (clicks, time spent, shares) shape what the agent learns. But they reward clickbait and outrageânot truth.
The agent consumes articles, blog posts, videos, or structured data from multiple sources. It extracts metadata: author, publication date, source credibility score, topic tags, and raw content.
Key question: How do you know if a source is trustworthy? (By reputation, fact-checking records, editorial standards, peer review? All are incomplete.)
The agent builds a model of what each user cares about. This comes from explicit signals (preferences, searches, saved items) and implicit signals (reading time, engagement patterns, topic overlap).
Key question: When should the agent expose users to content outside their preferences? (Serendipity vs. echo chambers)
Given a piece of content and a user, the agent assigns a relevance score. This combines textual similarity, topic alignment, semantic matching, and learned preferences.
Key question: How do you prevent the algorithm from reinforcing what users already think?
The agent filters content by credibility (fact-check status, source reputation), freshness (publish date, topic recency), and quality (writing clarity, evidence density, expert authorship).
Key question: Who decides what counts as "high quality"? And can those standards change per user?
The agent ranks surviving content by relevance, but also ensures diversity: multiple perspectives on the same topic, mix of news types (breaking vs. analysis vs. opinion), and sources of varying credibility levels.
Key question: Should the agent show users content that challenges them?
The agent optimizes for user clicks and time-on-site, which rewards sensationalism, outrage, and conspiracy content over factual news. Engagement metrics and truth diverge.
The agent learns that certain sources (e.g., major newspapers) are "safer" to recommend, while niche outlets (specialist publications, independent journalists) get downranked despite high quality. Distribution becomes centralized.
By personalizing content to each user's history, the agent creates information silos. Users see more of what they already like and less of what they don't, narrowing their worldview.
Don't just optimize for relevance. Explicitly require diversity of sources, perspectives, and content types. Make it a hard constraint, not a soft bonus.
Train content evaluation on explicit quality metrics (fact-checks, expert ratings, citations) separate from user engagement. Don't let clicks define quality.
Source credibility should inform ranking, not exclusion. Low-credibility sources can appear, but lower. Users can still choose to engage.
Transparency: show users why content was ranked this way. Autonomy: let users adjust filters and preferences. Trust follows.
When building a content agent, ask:
Design a content agent for one of these scenarios. The AI coach will guide your design and challenge your assumptions.
Your task: Design a content agent for ONE of these scenarios. You'll specify the architecture, evaluation criteria, and failure mitigation strategy. The AI coach will ask clarifying questions and pressure-test your design.
Pick one above, or describe your own scenario. Then talk through your agent design with the coach.