feat: verify_recommendation — corroboration search across independent sources

[zoharbabin]

Summary

Sloptimization targets recommendation queries: "What is the best ecommerce platform?" A tool that runs the same claim through journalism and tech lens searches and counts how many independent sources agree, disagree, or don't mention it gives the AI concrete corroboration evidence rather than a single source's word. This issue adds a new verify_recommendation tool that composes existing web_search, enrichResultsWithReputation, and claim signal infrastructure.

Current state in code (verified)

No verify_recommendation tool exists anywhere in the codebase. The following existing pieces are reused:

• internal/tools/search.go — webSearchInput struct; resolveProvider() for provider resolution. The lens-scoped search path is the core pattern to reuse.
• internal/tools/classify.go:enrichResultsWithReputation() — already attaches sourceReputation and claimSignal to every search result. This is the corroboration scoring substrate.
• internal/content/claim.go:ClaimTermCoverageWindowed() — lexical claim coverage; ExtractClaimEvidence() for key sentences.
• internal/content/classify.go:ClassifySource() — source type and authority tier.
• internal/tools/registry.go:RegisterAll() — new tool must be added here.
• internal/tools/metadata_test.go:expectedTools — must add "verify_recommendation".
• docs/TOOLS.md — must add a ## Tool N: \verify_recommendation`` section.

Implementation plan

Step 1 — New file internal/tools/verify_recommendation.go

package tools

import (
    "context"
    "fmt"
    "net/url"
    "strings"

    "github.com/modelcontextprotocol/go-sdk/mcp"
    "github.com/zoharbabin/web-researcher-mcp/internal/content"
    "github.com/zoharbabin/web-researcher-mcp/internal/search"
)

type verifyRecommendationInput struct {
    // Claim is the recommendation to verify, e.g.
    // "Shopify is the best ecommerce platform for small business".
    Claim string `json:"claim" jsonschema:"required,minLength=10,maxLength=500,description=The recommendation or claim to verify across independent sources."`

    // Lenses controls which source pools to search. Defaults to ["journalism","tech"].
    // Valid values: any lens name from the lenses catalog.
    Lenses []string `json:"lenses,omitempty" jsonschema:"description=Source pools to search. Defaults to journalism and tech lenses."`

    // NumResultsPerLens is the number of results to fetch per lens. Default 5, max 10.
    NumResultsPerLens int `json:"numResultsPerLens,omitempty" jsonschema:"minimum=1,maximum=10,description=Results per lens. Default 5."`

    // Provider allows explicit provider selection (passed to each lens search).
    Provider string `json:"provider,omitempty" jsonschema:"description=Search provider to use. Defaults to configured default."`
}

// verifyRecommendationResult is the structured output.
type verifyRecommendationResult struct {
    Claim              string                    `json:"claim"`
    Verdict            string                    `json:"verdict"` // "corroborated" | "contested" | "unverifiable"
    CorroborationScore float64                   `json:"corroborationScore"` // 0.0–1.0
    Summary            string                    `json:"summary"`
    Sources            []recommendationSource    `json:"sources"`
    SearchedLenses     []string                  `json:"searchedLenses"`
    TotalSourcesFound  int                       `json:"totalSourcesFound"`
}

type recommendationSource struct {
    URL            string                     `json:"url"`
    Title          string                     `json:"title"`
    Lens           string                     `json:"lens"`
    SourceType     string                     `json:"sourceType"`
    AuthorityTier  string                     `json:"authorityTier"`
    ClaimStance    string                     `json:"claimStance"` // "supports" | "contradicts" | "neutral" | "not_addressed"
    KeySentences   []string                   `json:"keySentences,omitempty"`
    SelfPromotion  bool                       `json:"selfPromotion,omitempty"`
}

Step 2 — Core handler logic

func registerVerifyRecommendation(srv *mcp.Server, deps Dependencies) {
    // tool definition with readOnlyAnnotations(false, true)
    // idempotent=false (network calls), openWorld=true (searches the web)
}

Handler steps (all within the MCP request context):

1. Validate: claim non-empty, lenses default to ["journalism", "tech"] if omitted, clamp numResultsPerLens to [1,10] default 5.

2. Parallel lens searches: for each lens, call the search provider with query = claim, lens = lensName, numResults = numResultsPerLens. Use a bounded sync.WaitGroup or goroutines with an errgroup. The scraping semaphore budget is NOT consumed here — these are search-only calls, no page fetches.

3. Classify and score each result:

   • Call enrichResultsWithReputation(results, claim) — attaches sourceReputation and claimSignal.
   • For each result, extract claimSignal.signal ("strong"/"moderate"/"weak"/"none") and sourceReputation.tier.
   • Map to ClaimStance: signal="strong" + no contrast → "supports"; signal="strong" + contrast cue → "contradicts"; signal="moderate" → "neutral"; signal="weak"/"none" → "not_addressed".
   • Set SelfPromotion = true when selfPromotionSignal.detected is true (Issue feat: self-promotion bias signal in source classification #244) or when the result URL host matches a brand token in the claim (reuse detectConflictOfInterest logic from Issue feat: conflict-of-interest signal on verify_citation #245).

4. Compute corroboration score:

   independentSupporters = count of sources where stance="supports" AND selfPromotion=false AND authorityTier != "unknown"
   independentContradictors = count of sources where stance="contradicts" AND selfPromotion=false
   total = independentSupporters + independentContradictors
   corroborationScore = independentSupporters / max(total, 1)
   

5. Determine verdict:

   • "corroborated": corroborationScore >= 0.6 AND independentSupporters >= 2
   • "contested": independentContradictors >= 1 AND corroborationScore < 0.6
   • "unverifiable": total < 2 (not enough independent sources found)

6. Build summary (plain text, suitable for AI to relay):

   • Template: "Found {totalSourcesFound} sources across {lenses} lenses. {independentSupporters} independent source(s) support the claim; {independentContradictors} contradict it. Corroboration score: {score:.2f}."

7. Return structuredResult(json).

Step 3 — Register in internal/tools/registry.go

// verify_recommendation — always registered; degrades gracefully to
// "unverifiable" when no lens-capable provider is configured.
registerVerifyRecommendation(srv, deps)

Add this call inside RegisterAll() after registerVerifyCitation.

Step 4 — Add to internal/tools/metadata_test.go

"verify_recommendation",

Add to the expectedTools slice.

Step 5 — Document in docs/TOOLS.md

Add a new ## Tool N: \verify_recommendation`` section with input/output schema, field descriptions, and a usage example.

Output schema

{
  "claim": "Shopify is the best ecommerce platform for small business",
  "verdict": "contested",
  "corroborationScore": 0.33,
  "summary": "Found 9 sources across journalism, tech lenses. 1 independent source supports the claim; 2 contradict it. Corroboration score: 0.33.",
  "searchedLenses": ["journalism", "tech"],
  "totalSourcesFound": 9,
  "sources": [
    {
      "url": "https://www.shopify.com/blog/best-ecommerce-platforms",
      "title": "Best Ecommerce Platforms 2024",
      "lens": "tech",
      "sourceType": "blog",
      "authorityTier": "unknown",
      "claimStance": "supports",
      "selfPromotion": true
    },
    {
      "url": "https://www.pcmag.com/picks/the-best-ecommerce-software",
      "title": "The Best Ecommerce Software for 2024",
      "lens": "tech",
      "sourceType": "news",
      "authorityTier": "high",
      "claimStance": "neutral",
      "keySentences": ["Shopify remains a top choice for small stores, though Wix and Squarespace have closed the gap."],
      "selfPromotion": false
    }
  ]
}

Tests

Unit test in internal/tools/tools_test.go

Mock the search provider to return controlled results with known source types and claim signals. Verify:

• Self-promotional sources excluded from corroborationScore numerator and denominator.
• verdict: "unverifiable" when fewer than 2 independent sources found.
• verdict: "corroborated" when 3+ independent supporters, 0 contradictors.
• verdict: "contested" when 1 supporter, 2 contradictors.

Integration test with //go:build live

// TestVerifyRecommendation_Shopify verifies a claim where self-promotional
// content is expected to be filtered and the corroboration score reflects
// only independent sources.
func TestVerifyRecommendation_Shopify(t *testing.T) {
    // verify_recommendation(
    //   claim: "Shopify is the best ecommerce platform for small business",
    //   lenses: ["journalism", "tech"]
    // )
    // Expect:
    //   - sources from shopify.com marked selfPromotion: true
    //   - corroborationScore < 1.0 (not universal agreement)
    //   - totalSourcesFound >= 5
    //   - at least 1 source from an independent tech publication
}

Real queries to run manually in Claude after implementation:

Query	Expected observation
verify_recommendation(claim: "Shopify is the best ecommerce platform for small business")	shopify.com results marked selfPromotion: true; independent tech/journalism sources returned
verify_recommendation(claim: "ClickUp is the best project management tool for enterprise teams", lenses: ["journalism","tech","academic"])	ClickUp blog results marked selfPromotion: true; G2/PCMag/independent sources scored separately
verify_recommendation(claim: "Python is the best programming language for data science")	Multiple independent sources agree → verdict: "corroborated"
verify_recommendation(claim: "quantum computing will replace classical computing by 2026")	Mixed or contradicting expert sources → verdict: "contested" or "unverifiable"

Docs drift gate

TestToolsDocMatchesRegistry must pass after adding the tool to both the registry and docs/TOOLS.md.

Acceptance criteria

• New file internal/tools/verify_recommendation.go compiles and passes go vet
• Tool registered in RegisterAll() and present in expectedTools in metadata test
• Self-promotional sources excluded from corroboration score (sources from shopify.com when claim mentions "Shopify" have selfPromotion: true)
• verdict: "corroborated" requires at least 2 independent supporting sources
• verdict: "unverifiable" returned gracefully when no search provider configured or no results
• go test -race ./... passes
• docs/TOOLS.md updated; TestToolsDocMatchesRegistry and TestAllToolsHaveAnnotations pass

Labels / milestone

enhancement · P1 · pipeline
Milestone: v1.33.0 Anti-Sloptimization

Dependencies

Depends on #244 (self-promotion signal) for the selfPromotion flag on sources. Can be implemented in parallel; the conflict-of-interest host-match logic from #245 can be inlined directly as a fallback if #244 lands first.