A first-hand infrastructure teardown of usnews.com and nerdwallet.com — measured directly over the wire on 25 August 2026, not scraped from a stack-guessing service. Every technical claim below has a reproducible probe behind it.
Both sites are large American consumer-advice publishers. They have almost nothing in common architecturally, and the divergence traces to a single decision made years ago: U.S. News monetises attention; NerdWallet monetises intent.
You do not need insider access to work out a publisher's business model. You just read ads.txt — the public file declaring who is authorised to sell their ad inventory.
/ads.txtLines in the file · measured 25 Aug 2026
U.S. News runs a full header-bidding stack — Magnite/Rubicon (35 entries), PubMatic (29), OpenX (22), Xandr/AppNexus (21), Index Exchange (18), Sharethrough, Smart AdServer, Amazon APS, TripleLift, Yieldmo, Media.net and roughly fifty more. That is a CPM business: revenue is a function of impressions served, so every lost pageview is lost money, linearly.
NerdWallet declares four lines, all Google. They barely sell display. Revenue comes from CPA and lead-generation — a bank pays when a user actually opens an account. A user who converts is worth hundreds of times a user who merely reads, so traffic can fall while revenue climbs.
Every row below was observed directly via DNS queries, TLS handshakes and HTTP response headers on 25 Aug 2026.
Architecture is not abstract. Here is what each homepage actually pushes down the wire, measured with the same client, same network, same minute.
Kilobytes over the wire, compressed
Megabytes of markup the browser must parse
Count of <script src> tags
NerdWallet does not run one application. Each vertical is a separately deployed micro-frontend, and they are visibly mid-migration from Next.js to Astro — a migration they can perform one route at a time without a big-bang replatform.
| Route | Framework | Transfer | TTFB |
|---|---|---|---|
/ | Astro | 138 KB | 0.72 s |
/banking | Astro + headless WP | 167 KB | 0.59 s |
/insurance/auto | Astro + headless WP | 169 KB | 0.85 s |
/mortgages | Astro + headless WP | 170 KB | 0.68 s |
/investing | Astro + headless WP | 165 KB | 0.80 s |
/personal-loans | Next.js + headless WP | 273 KB | 0.99 s |
/small-business | Astro | 130 KB | 0.36 s |
/taxes | Astro + headless WP | 161 KB | 0.66 s |
Note the un-migrated Next.js route is the heaviest and slowest on the estate — the migration is buying real performance, not just novelty.
U.S. News by contrast serves money.usnews.com, health.usnews.com and www.usnews.com/banking from the same build pipeline (static-atlas, builds 14028–14059), the same Brightspot CMS, the same six googletag ad calls and the same two Akamai IPs. One monolith, one deploy, one blast radius.
Both companies have made a deliberate, documented bet about large language models. They bet in opposite directions.
| Crawler | U.S. News | NerdWallet |
|---|---|---|
| GPTBot (OpenAI training) | ◐ 5 sponsored URLs only | ✓ Allowed |
| ClaudeBot (Anthropic) | ◐ 5 sponsored URLs only | ✓ Allowed |
| PerplexityBot | ◐ 5 sponsored URLs only | ✓ Allowed |
| OAI-SearchBot (ChatGPT search) | ◐ 5 sponsored URLs only | ✓ Allowed |
| Google-Extended (Gemini training) | ◐ 5 sponsored URLs only | ✕ Blocked |
| Applebot-Extended | ✕ Blocked | ✓ Allowed |
| meta-externalagent | ✕ Blocked | ✓ Allowed |
| Amazonbot / bedrockbot | ✕ Blocked | ✓ Allowed |
| CCBot, Bytespider, DeepSeek, Mistral, Kimi, +40 more | ✕ Blocked | ✓ Allowed |
/brandfuse/articles/lemonade/: paid sponsored content for an insurance advertiser. They are withholding their editorial crown jewels from LLMs while deliberately feeding those same LLMs a sponsor's marketing copy. Whatever the intent, that is the observable configuration, and it is a defensible legal position only for as long as nobody frames it as "we'll sell you training data, just not give it away."
The blocking posture is consistent with litigation, not negotiation: U.S. News & World Report, L.P. v. OpenAI, Inc., No. 1:25-cv-09912 (S.D.N.Y., filed December 2025), now consolidated into In re OpenAI Copyright Infringement Litigation, No. 1:25-md-03143, before Judge Sidney H. Stein. Reporting notes the complaint leans specifically on the rankings as the distinguishing asset — the argument being that U.S. News owns something no other plaintiff has.
NerdWallet took the opposite view. Their CEO Tim Chen told analysts in the Q2 2026 call that "intent is extremely high when someone is coming through an LLM." They let the models read almost everything, treat LLMs as a new top-of-funnel distribution channel, and are spending to convert that traffic — guiding to a fivefold increase in marketing investment ($15–20M annually) aimed at owned audiences with multi-year payback.
Stripped of marketing language, NerdWallet runs a product-matching engine, not a chatbot. The mechanics, in order:
/cc-prequal-service/ for card pre-qualification, /insurance/*/match/ for carrier matching, /prequalify/ — with /ab-taproom-api/ running the experimentation layer over the top.Ordered by leverage, with the honest constraint noted on each.
U.S. News already has what NerdWallet spent a decade assembling: authoritative, structured, comparative data on colleges, hospitals, cars, funds and insurers. Today it is rendered as Brightspot pages and monetised at CPM. The same data as a queryable catalogue behind a matching engine is worth an order of magnitude more per user.
Constraint: the rankings' credibility is the asset. Any matching layer must be provably separate from editorial ranking, or the brand — the only real moat — erodes. NerdWallet has a far easier time here because nobody thinks their "best savings account" list is a public good.
Every vertical U.S. News runs is a high-stakes, high-consideration decision — choosing a college, a hospital, a car. Those are exactly the moments users will trade an email for personalisation. Right now that intent evaporates into an ad impression. NerdWallet's entire AI advantage rests on knowing who the user is across sessions; without an identity layer, no amount of model sophistication helps.
Constraint: requires an identity service U.S. News does not appear to run — there is no identity.usnews.com or auth.usnews.com in DNS, while NerdWallet has both.
Education and health are lead-generation businesses that U.S. News is currently running as display businesses. The infrastructure for compliant lead capture is already partly in place — their DNS carries an activeprospect-domain-verification TXT record, meaning ActiveProspect/TrustedForm, which exists to document consent on generated leads. Something in the building is already doing this; it is not yet the centre of the model.
Constraint: a 479-line programmatic stack represents real, booked, current revenue. This is a multi-year mix shift, not a switch to flip, and it will look like a revenue decline before it looks like a win.
Every U.S. News vertical ships from one static-atlas pipeline and one Brightspot instance. That means the money vertical cannot adopt a transactional, app-like architecture without dragging the newsroom along. NerdWallet's per-route micro-frontends let them run Astro on content pages and heavier JS on marketplace pages, and migrate one route at a time.
Constraint: Brightspot is genuinely good at what U.S. News bought it for — 400+ contributors, 180K articles, editorial workflow. The answer is not to rip it out; it is to make it headless for the commercial verticals while it keeps serving the newsroom.
No Brotli (NerdWallet has it), no HTTP/3 (NerdWallet has it), 48 external scripts, 3.98 MB of decoded markup, JPEGs where WebP would do, and a CSP containing 'unsafe-inline' 'unsafe-eval' https: — which is close to no CSP at all. Enabling Brotli and HTTP/3 at the Akamai layer is a configuration change, not a project.
Constraint: the script count is mostly ad tech, so it falls out of recommendation 3 rather than being independently fixable. The transport and CSP items are not — do those now.
Blocking fifty crawlers protects training-data leverage, and with active litigation that posture is coherent. But it also means U.S. News is absent from the answer layer that is steadily replacing the search results page. NerdWallet is discovering that LLM referrals convert unusually well because intent is high. A rankings brand that is never cited in AI answers loses the thing it has spent ninety years building: being the default reference.
Constraint: I am second-guessing a live legal strategy from the outside, which is worth exactly what you paid for it. The narrower, defensible version: separate training access (keep blocking, keep litigating) from retrieval and citation access (allow it, instrument it, measure conversion). The current robots.txt does not make that distinction — it blocks OAI-SearchBot and Perplexity-User, which are retrieval agents, not training crawlers.
Reproducible probes. All measurements taken 25 August 2026 from a single client.
| Dimension | U.S. News | NerdWallet | How observed |
|---|---|---|---|
| Nameservers | Akamai | AWS Route 53 | dig NS |
| Edge IPs | 23.46.207.138/.145 | 172.64.154.223 (CF) | dig A + whois |
| Edge vendor | Akamai Technologies | Cloudflare, Inc. | whois netname |
| Static origin | AmazonS3 via CloudFront | Vercel | server:, x-amz-cf-pop, x-vercel-id |
| Origin header | x-origin: Static-Atlas | — | HTTP HEAD |
| Build identifier | USN-CICD-STAG 14059 | dpl_G9AFDPzobR6espxMA3W5PHBA6DNd | HTML comment / asset query string |
| CMS | Brightspot | WordPress (headless) | component path / wp-sitemap |
| Frontend | React (SSR) | Astro + Next.js | data-astro-cid, markup analysis |
| HTTP/3 | Not advertised | h3=":443" | alt-svc header |
| Compression | gzip | Brotli | content-encoding |
| TLS issuer | Let's Encrypt YR1 | Google Trust Services WE1 | openssl s_client |
| Homepage transfer | 720 KB | 138 KB | curl -w %{size_download} |
| Homepage decoded | 3.98 MB | 1.08 MB | wc -c |
| <script> tags | 59 | 11 | markup count |
| External scripts | 48 | 3 | markup count |
| ads.txt lines | 479 | 4 | /ads.txt |
| Bot mitigation | Akamai Bot Manager (tarpits HTML) | Cloudflare (identity endpoint returns 429) | direct request behaviour |
| Identity service | None found in DNS | identity. + auth. | subdomain enumeration |
| AI crawlers blocked | ~50, root-level | 1 (Google-Extended) | /robots.txt |
| Vendor TXT records | Anthropic, Cursor, Adobe, New Relic, ActiveProspect, Sophos, KnowBe4, Amazon SES | none at apex | dig TXT |