fix: news_search 파라미터 보정을 코드 레벨에서 강제 (프롬프트 반복 실패 후)

같은 문제("태풍 돌핀" 쿼리 → 0건)를 프롬프트/스키마 설명으로 4번
연속 고치려 했으나, 실사용 재현 테스트마다 모델이 매번 예전 방식
그대로 호출했다(country 안 넣거나, 카테고리어+고유명사를 한 쿼리에
계속 붙임). prompt-gates.ts 파일 자체의 설계 철학대로 — "프롬프트
지시만으론 안 믿을 때 결정적으로 강제하는 백스톱" — 이번엔 코드가
직접 보정하도록 바꿨다.

execute()에 두 가지 결정적 로직 추가:

1. 쿼리에 한글이 있는데 country/language 둘 다 없으면 자동으로
   country="kr" 적용. 모델이 계속 빠뜨리던 파라미터를 기본값으로
   메운다.
2. 검색 결과가 0건이고 쿼리에 재해 카테고리어(태풍/지진/산불 등)가
   섞여 있으면, 그 단어만 제거하고 자동 재시도. 라이브 API로 직접
   검증: q="태풍 돌핀"은 0건, q="돌핀"은 5건 — 실제 기사 제목이
   "태풍 '돌핀'"(따옴표 있음)이라 연속 문자열 매칭이 실패했던 것.

두 로직을 resolveNewsParams/stripCategoryWordForRetry로 분리해
네트워크 없이 단위 테스트 가능하게 했다(link-validator.ts의
rewriteDeadLinks와 같은 패턴) — 실제 사고 수치를 그대로 회귀
테스트에 고정, 10개 추가.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kim
2026-08-10 16:17:15 +09:00
co-authored by Claude Opus 5
parent cea2a49efb
commit 6c02050db6
2 changed files with 134 additions and 27 deletions
+69 -27
View File
@@ -1,6 +1,7 @@
import { ToolResult } from '../types.js'; import { ToolResult } from '../types.js';
import { getConfig } from '../config/config.js'; import { getConfig } from '../config/config.js';
import { getVault } from '../security/vault.js'; import { getVault } from '../security/vault.js';
import { DISASTER_PATTERN } from '../gateway/guards/prompt-gates.js';
const NEWSDATA_BASE = 'https://newsdata.io/api/1/latest'; const NEWSDATA_BASE = 'https://newsdata.io/api/1/latest';
@@ -33,6 +34,52 @@ function formatArticles(results: any[]): string {
}).join('\n\n'); }).join('\n\n');
} }
// Split out from execute() (2026-08-10) so the parameter-resolution logic — the part that was
// actually buggy — can be unit-tested without a live NewsData.io call. Pure: no I/O.
export function resolveNewsParams(args: any): { country: string; language: string; category: string } {
const rawQuery = String(args?.query || '').trim();
// NewsData.io hard-errors ("Number of country cannot exceeded 5") instead of just using
// the first 5 — silently truncating here saves the model a wasted round-trip on requests
// that naturally name 6+ countries (e.g. "소아시아" mapped to 6 nearby country codes).
let country = String(args?.country || '').trim().toLowerCase().split(',').map((c: string) => c.trim()).filter(Boolean).slice(0, 5).join(',');
// 2026-08-10: schema-description guidance alone (telling the model to always pass country
// for a non-English query) did not reliably change behavior across repeated live tests — the
// model kept omitting it. Defaulting it here instead of only asking for it: a query containing
// Hangul with no country/language given almost certainly wants Korean sources, so default to
// "kr" rather than falling through to NewsData.io's English-leaning pool.
const hasHangul = /[가-힣]/.test(rawQuery);
if (!country && !args?.language && hasHangul) country = 'kr';
// NewsData.io silently drops the country filter (falling back to a generic English-language
// world-news pool instead of erroring) in two independently-confirmed cases — verified
// 2026-07-16 against the live API:
// 1. language doesn't match the country's dominant language (country=de+language=en
// returned US Congress/FDA-recall articles, not German news; country=de alone or
// country=de+language=de both returned real German news)
// 2. category is "world" (country=de+category=world broke the same way; country=de with
// any other category like "politics"/"top" returned real German news). "world" scoped
// to a single country is a contradictory combination anyway.
// Dropping both whenever country is set is a strict improvement: a no-op when they'd have
// been compatible, and prevents the silent-wrong-country fallback when they aren't.
const language = country ? '' : String(args?.language || '').trim().toLowerCase();
let category = String(args?.category || '').trim().toLowerCase();
if (country && category === 'world') category = '';
return { country, language, category };
}
// 2026-08-10, verified against the live API: a query combining a generic disaster/category word
// with a specific proper noun ("태풍 돌핀") returns ZERO results even with country correctly
// set, because NewsData.io matches q as a literal substring and real headlines write it as
// "태풍 '돌핀'" (quote marks break the exact match) — while "돌핀" alone returned 5 relevant
// articles and "태풍" alone returned 12. Schema guidance to avoid this combination didn't change
// model behavior across repeated live tests either, so this strips the category word
// deterministically for a retry. Returns '' when there's nothing left to retry with (query was
// only the category word, or had no category word at all).
export function stripCategoryWordForRetry(query: string): string {
if (!DISASTER_PATTERN.test(query)) return '';
const stripped = query.replace(new RegExp(DISASTER_PATTERN.source, 'g'), ' ').replace(/\s+/g, ' ').trim();
return stripped && stripped !== query.trim() ? stripped : '';
}
// NewsData.io's /latest endpoint only covers the past 48 hours (free tier) — exactly the // NewsData.io's /latest endpoint only covers the past 48 hours (free tier) — exactly the
// "today's news" window this tool exists for. Unlike web_search's generic scrape-and-rank, // "today's news" window this tool exists for. Unlike web_search's generic scrape-and-rank,
// every result here carries a real pubDate and source from the provider's own metadata, so // every result here carries a real pubDate and source from the provider's own metadata, so
@@ -61,40 +108,35 @@ export const newsSearchTool = {
const apiKey = getApiKey(); const apiKey = getApiKey();
if (!apiKey) return { success: false, error: 'NewsData.io API key not configured (set news.newsdata_api_key in .smallclaw/config.json)' }; if (!apiKey) return { success: false, error: 'NewsData.io API key not configured (set news.newsdata_api_key in .smallclaw/config.json)' };
const query = String(args?.query || '').trim(); const rawQuery = String(args?.query || '').trim();
// NewsData.io hard-errors ("Number of country cannot exceeded 5") instead of just using const { country, language, category } = resolveNewsParams(args);
// the first 5 — silently truncating here saves the model a wasted round-trip on requests
// that naturally name 6+ countries (e.g. "소아시아" mapped to 6 nearby country codes).
const country = String(args?.country || '').trim().toLowerCase().split(',').map(c => c.trim()).filter(Boolean).slice(0, 5).join(',');
// NewsData.io silently drops the country filter (falling back to a generic English-language
// world-news pool instead of erroring) in two independently-confirmed cases — verified
// 2026-07-16 against the live API:
// 1. language doesn't match the country's dominant language (country=de+language=en
// returned US Congress/FDA-recall articles, not German news; country=de alone or
// country=de+language=de both returned real German news)
// 2. category is "world" (country=de+category=world broke the same way; country=de with
// any other category like "politics"/"top" returned real German news). "world" scoped
// to a single country is a contradictory combination anyway.
// Dropping both whenever country is set is a strict improvement: a no-op when they'd have
// been compatible, and prevents the silent-wrong-country fallback when they aren't.
const language = country ? '' : String(args?.language || '').trim().toLowerCase();
let category = String(args?.category || '').trim().toLowerCase();
if (country && category === 'world') category = '';
const params = new URLSearchParams({ apikey: apiKey }); const fetchOnce = async (q: string) => {
if (query) params.set('q', query); const params = new URLSearchParams({ apikey: apiKey });
if (country) params.set('country', country); if (q) params.set('q', q);
if (language) params.set('language', language); if (country) params.set('country', country);
if (category) params.set('category', category); if (language) params.set('language', language);
if (category) params.set('category', category);
try {
const res = await fetch(`${NEWSDATA_BASE}?${params}`, { signal: AbortSignal.timeout(15_000) }); const res = await fetch(`${NEWSDATA_BASE}?${params}`, { signal: AbortSignal.timeout(15_000) });
const data: any = await res.json(); const data: any = await res.json();
if (!res.ok || data.status !== 'success') { if (!res.ok || data.status !== 'success') {
const msg = data?.results?.message || data?.message || `HTTP ${res.status}`; const msg = data?.results?.message || data?.message || `HTTP ${res.status}`;
return { success: false, error: `NewsData.io error: ${msg}` }; throw new Error(`NewsData.io error: ${msg}`);
} }
const results: any[] = Array.isArray(data.results) ? data.results : []; const results: any[] = Array.isArray(data.results) ? data.results : [];
return { data, results };
};
try {
let query = rawQuery;
let { data, results } = await fetchOnce(query);
if (results.length === 0) {
const stripped = stripCategoryWordForRetry(query);
if (stripped) {
query = stripped;
({ data, results } = await fetchOnce(query));
}
}
return { return {
success: true, success: true,
stdout: formatArticles(results), stdout: formatArticles(results),
+65
View File
@@ -0,0 +1,65 @@
/**
* news.test.ts
*
* resolveNewsParams / stripCategoryWordForRetry were extracted from news_search's execute()
* (2026-08-10) after schema-description guidance alone repeatedly failed to change model
* behavior in live tests — 4 consecutive real chat turns still called news_search without a
* country param, or with a category+name query that returns zero results, despite the tool's
* own description explaining both pitfalls with the exact verified numbers. Deterministic
* code fixes the failure regardless of what the model writes; these tests pin the fix down.
*/
import { test, describe } from 'node:test';
import assert from 'node:assert/strict';
import { resolveNewsParams, stripCategoryWordForRetry } from '../src/tools/news';
describe('resolveNewsParams — country/language/category 정규화', () => {
test('한글 쿼리 + country/language 둘 다 없으면 kr로 기본값 설정', () => {
// 실제 사고: "태풍 돌핀" 쿼리에 country를 4번 연속 안 넣었다.
assert.equal(resolveNewsParams({ query: '태풍 돌핀' }).country, 'kr');
});
test('영어 쿼리엔 kr을 강제하지 않는다', () => {
assert.equal(resolveNewsParams({ query: 'typhoon dolphin' }).country, '');
});
test('country나 language를 명시하면 한글 쿼리여도 그대로 존중한다', () => {
assert.equal(resolveNewsParams({ query: '태풍', country: 'jp' }).country, 'jp');
assert.equal(resolveNewsParams({ query: '태풍', language: 'en' }).country, '');
});
test('country가 6개 이상이면 5개로 자른다', () => {
assert.equal(resolveNewsParams({ country: 'de,fr,gb,it,es,pl' }).country, 'de,fr,gb,it,es');
});
test('country가 있으면 language는 무시된다', () => {
assert.equal(resolveNewsParams({ country: 'de', language: 'en' }).language, '');
});
test('country + category=world 조합은 category를 지운다', () => {
assert.equal(resolveNewsParams({ country: 'de', category: 'world' }).category, '');
});
});
describe('stripCategoryWordForRetry — 카테고리어+고유명사 조합 실패 대응', () => {
// 실제 사고(2026-08-10, 라이브 API로 검증): NewsData.io에 직접 호출해본 결과
// q="태풍 돌핀"+country=kr → 0건
// q="태풍"+country=kr → 12건 (돌핀 기사 포함)
// q="돌핀"+country=kr → 5건 (전부 관련 기사)
// 실제 기사 제목이 "태풍 '돌핀'"(따옴표 있음)이라 "태풍 돌핀"과 리터럴 매칭이 안 됐다.
test('재해 카테고리어 + 고유명사 조합에서 카테고리어를 제거한다', () => {
assert.equal(stripCategoryWordForRetry('태풍 돌핀'), '돌핀');
});
test('카테고리어만 있으면 재시도할 게 없다', () => {
assert.equal(stripCategoryWordForRetry('태풍'), '');
});
test('재해 키워드가 아예 없으면 재시도할 게 없다', () => {
assert.equal(stripCategoryWordForRetry('삼성전자 실적'), '');
});
test('카테고리어가 중간에 껴 있어도 나머지를 살린다', () => {
assert.equal(stripCategoryWordForRetry('간토 지진 피해'), '간토 피해');
});
});