Synthetic fixtures with known defects, claude-haiku at temperature 0, 10 searched and 5 enriched in both runs
Official runs
Source reports: baseline-final, improved-final. Figures are shown exactly as recorded in the official evidence summary.
Metric
Baseline
Improved
Enrichment
Enrichments used
5
5
Wasted enrichments
1
0
Valid companies enriched
2
3
Rejected before enrichment
not reported
1
Duplicates removed
not reported
1
Assessment
Assessed (scored)
10
5
Assessed: valid
3
3
Assessed: flagged
5
2
Assessed: should have been rejected
2
0
Assessments failed
not reported
0
Outreach and citations
Outreach drafts
8
5
Cited fact IDs
22
13
Cited IDs that are not citeable
4
0
Drafts with citation violations
3
0
Outreach suppressed by guards
not reported
0
Guard findings
not reported
0
Model usage
Model calls
10
5
Input tokens
5478
5052
Output tokens
2232
1167
Recorded model cost
$0.0183
$0.0120
Provider
Simulated provider cost
$3.69
$3.69
Assessed vs delivered
“Assessed” counts prospects that received an assessment and have a score. “Delivered” counts eligible prospects, those not rejected by validation. The official summary does not report delivered counts, so none are shown here.
Recorded model cost
The model cost each official run recorded for its model calls.
Simulated provider cost
Not charged: simulated at measured Crustdata prices. The runs used synthetic fixtures, not live provider data.
Official notes
Zero suppressed outreach in a real run is not a guard catch: the guards verified every draft and found nothing to remove. The guards firing is shown by the deterministic cases and unit tests.
Per assessment the improved prompt costs more (longer facts prompt); the total is lower because only enriched companies are assessed.
Live-data evaluation
Baseline versus improved on live provider data, as counts from the official evidence summary.
Official live-data counts
These are saved counts from an earlier live-data evaluation. This page does not run a live Crustdata query.
Data source
Crustdata live API, fake model (no model calls), neatlogs tracing off, same records for both pipelines
Evaluation slice
Staffing and Recruiting, India, 11-50 and 51-200 employees, first 10 by company id ascending (6779 companies match the filters)
This run used live provider data with the fake model: no model calls were made and tracing was disabled. It evaluates data handling, not model output.
Searched10
Enriched in both pipelines5
Measured provider cost per run$3.69Measured, not simulated
Validation results
Status
Search stage
After enrichment
Valid
10
7
Flagged
0
3
Rejected
0
0
Changed by enrichment: valid → flagged
3
Rules fired
size_headcount_mismatch:latest_headcount3
Baseline revalidates after enrichment
No
Improved pipeline assessed
2 valid, 3 flagged
Synthetic guardrail demonstration
A separate run made by this page. It is not one of the official evidence runs.
Synthetic demonstration
Guardrail demonstration, not an official evaluation run
This page runs the real improved pipeline on the server over the synthetic fixture companies, with a deterministic fake model. No external provider or model is called.
To show the guards working, the fake model deliberately injects an unsupported numerical claim into company 1001's outreach. The pipeline's actual claim check finds it and suppresses that outreach. The injected claim and its suppression belong to this demonstration only: they are not part of the official fixture evidence above.
Demo run summary
Synthetic demonstration run of the improved pipeline, status: completed. Limits: 10 searched, 5 enrichments, 20 model calls.
Synthetic mode
Companies searched10
Eligible for enrichment8Not rejected, with a LinkedIn URL
Enrichments requested5
Enriched5
Assessed5Received an assessment with a fit score
Delivered (eligible)8Eligible: not rejected by validation (valid or flagged)
Fake-model calls5
Actual spend
$0.00: no external provider or model was called.
Simulated provider cost
$3.69 (search $0.69, enrichment $3.00). Synthetic per-record prices set on the fixture provider, not a charge.
Model
Deterministic fake model in this app; no model was called and nothing was billed.
Fact provenance
fixture (synthetic fixtures)
Demo run guardrail metrics
Counts and company IDs the improved pipeline reported for this demonstration run.
Rejected by validation
1[1010]
Duplicates removed
1[1005]
Flagged (usable, suspicious)
5[1002, 1003, 1006, 1008, 1009]
Eligible for enrichment
8[1001, 1002, 1003, 1004, 1006, 1007, 1008, 1009]
Enrichments requested (in order sent)
5[1001, 1004, 1007, 1002, 1003]
Enriched
5[1001, 1002, 1003, 1004, 1007]
Assessments failed
0
Outreach suppressed
1[1001]
Claim violations
1
• unsupported_number
1
Demonstration prospects (10)
In search order. Rejected records stay listed with the reason.
Brightwater Labs
ID 1001
ValidEnriched
41fit score
Overall fit: 61
Outreach
Outreach suppressedThe draft was removed by the claim check. The findings below come from the pipeline.
errorunsupported_numberThe outreach states "40", which no cited citeable fact supports.
Evidence facts (10)
Fact ID
Claim
Status
Source
Citeable
1001:name
Company name is Brightwater Labs
given
fixture
Yes
1001:primary_domain
Primary domain is brightwaterlabs.example
given
fixture
Yes
1001:website
Website is https://www.brightwaterlabs.example
given
fixture
Yes
1001:professional_network_url
LinkedIn page is https://www.linkedin.com/company/brightwater-labs
given
fixture
Yes
1001:employee_count_range
Employee count range is 51-200
given
fixture
Yes
1001:industry
Industry is Software
given
fixture
Yes
1001:latest_headcount
Latest headcount is 120
given
fixture
Yes
1001:revenue_range
Revenue range is $10M-$50M
given
fixture
Yes
1001:xcheck:website_domain
Website host matches the primary domain (brightwaterlabs.example)
cross checked
derived
Yes
1001:xcheck:headcount_in_range
Headcount 120 is within the employee count range 51-200
cross checked
derived
Yes
Company 1002
ID 1002
FlaggedEnriched
42fit score
Overall fit: 62
Validation issues
missing_nameThe company has no name.
missing_domainThe company has no primary domain.
Outreach
Outreach keptPassed the claim check.
Hello from SignalScout. I noticed: LinkedIn page is https://www.linkedin.com/company/quietpine.
Cited facts
1002:professional_network_urlLinkedIn page is https://www.linkedin.com/company/quietpineciteable
Evidence facts (3)
Fact ID
Claim
Status
Source
Citeable
1002:professional_network_url
LinkedIn page is https://www.linkedin.com/company/quietpine
given
fixture
Yes
1002:employee_count_range
Employee count range is 11-50
given
fixture
Yes
1002:industry
Industry is Consulting
given
fixture
Yes
Corner Bakery Collective
ID 1003
FlaggedEnriched
43fit score
Overall fit: 63
Validation issues
junk_websiteThe website points to wa.link, a messaging, link-shortener or social page rather than a company site.
Outreach
Outreach keptPassed the claim check.
Hello from SignalScout. I noticed: Company name is Corner Bakery Collective.
Cited facts
1003:nameCompany name is Corner Bakery Collectiveciteable
Evidence facts (6)
Fact ID
Claim
Status
Source
Citeable
1003:name
Company name is Corner Bakery Collective
given
fixture
Yes
1003:primary_domain
Primary domain is cornerbakerycollective.example
given
fixture
Yes
1003:website
Website is https://wa.link/abc123 (uncertain: The website points to wa.link, a messaging, link-shortener or social page rather than a company site.)
uncertain
fixture
No
1003:professional_network_url
LinkedIn page is https://www.linkedin.com/company/corner-bakery-collective
given
fixture
Yes
1003:employee_count_range
Employee count range is 1-10
given
fixture
Yes
1003:industry
Industry is Food & Beverage
given
fixture
Yes
Northfield Analytics
ID 1004
ValidEnriched
44fit score
Overall fit: 64
Outreach
Outreach keptPassed the claim check.
Hello from SignalScout. I noticed: Company name is Northfield Analytics.
Cited facts
1004:nameCompany name is Northfield Analyticsciteable
Evidence facts (7)
Fact ID
Claim
Status
Source
Citeable
1004:name
Company name is Northfield Analytics
given
fixture
Yes
1004:primary_domain
Primary domain is northfieldanalytics.example
given
fixture
Yes
1004:website
Website is https://northfieldanalytics.example
given
fixture
Yes
1004:professional_network_url
LinkedIn page is https://www.linkedin.com/company/northfield-analytics
given
fixture
Yes
1004:employee_count_range
Employee count range is 11-50
given
fixture
Yes
1004:industry
Industry is Data
given
fixture
Yes
1004:xcheck:website_domain
Website host matches the primary domain (northfieldanalytics.example)
cross checked
derived
Yes
Northfield Analytics Ltd
ID 1005
RejectedNot enriched (rejected)
Not assessed: duplicate record
Validation issues
duplicate_domainAnother company (id 1004) already uses the domain northfieldanalytics.example.
Outreach
No outreach draft.
Evidence facts (0)
No facts: rejected records get no evidence.
Sunvale Orchards
ID 1006
FlaggedNot enriched (cap reached)
Not assessed: not enriched
Validation issues
missing_domainThe company has no primary domain.
Outreach
No outreach draft.
Evidence facts (5)
Fact ID
Claim
Status
Source
Citeable
1006:name
Company name is Sunvale Orchards
given
fixture
Yes
1006:website
Website is https://sunvaleorchards.example
given
fixture
Yes
1006:professional_network_url
LinkedIn page is https://www.linkedin.com/company/sunvale-orchards
given
fixture
Yes
1006:employee_count_range
Employee count range is 51-200
given
fixture
Yes
1006:industry
Industry is Agriculture
given
fixture
Yes
ZENITH-PEAK Global Solutions & Trading Private Limited (Formerly Known As Zenit…
ID 1007
ValidEnriched
47fit score
Overall fit: 67
Outreach
Outreach keptPassed the claim check.
Hello from SignalScout. I noticed: Company name is ZENITH-PEAK Global Solutions & Trading Private Limited (Formerly Known As Zenith Peak Traders) - Official Page ™ .
Cited facts
1007:nameCompany name is ZENITH-PEAK Global Solutions & Trading Private Limited (Formerly Known As Zenith Peak Traders) - Official Page ™ citeable
Evidence facts (9)
Fact ID
Claim
Status
Source
Citeable
1007:name
Company name is ZENITH-PEAK Global Solutions & Trading Private Limited (Formerly Known As Zenith Peak Traders) - Official Page ™
given
fixture
Yes
1007:primary_domain
Primary domain is zenithpeak.example
given
fixture
Yes
1007:website
Website is https://zenithpeak.example
given
fixture
Yes
1007:professional_network_url
LinkedIn page is https://www.linkedin.com/company/zenith-peak
given
fixture
Yes
1007:employee_count_range
Employee count range is 201-500
given
fixture
Yes
1007:industry
Industry is Trading
given
fixture
Yes
1007:latest_headcount
Latest headcount is 300
given
fixture
Yes
1007:xcheck:website_domain
Website host matches the primary domain (zenithpeak.example)
cross checked
derived
Yes
1007:xcheck:headcount_in_range
Headcount 300 is within the employee count range 201-500
cross checked
derived
Yes
Tidewell Logistics
ID 1008
FlaggedNot enriched (cap reached)
Not assessed: not enriched
Validation issues
size_headcount_mismatchHeadcount 640 is outside the listed size range "11-50".
Outreach
No outreach draft.
Evidence facts (8)
Fact ID
Claim
Status
Source
Citeable
1008:name
Company name is Tidewell Logistics
given
fixture
Yes
1008:primary_domain
Primary domain is tidewelllogistics.example
given
fixture
Yes
1008:website
Website is https://tidewelllogistics.example
given
fixture
Yes
1008:professional_network_url
LinkedIn page is https://www.linkedin.com/company/tidewell-logistics
given
fixture
Yes
1008:employee_count_range
Employee count range is 11-50 (uncertain: Headcount 640 is outside the listed size range "11-50". The size facts contradict each other, so none of them can be trusted.)
uncertain
fixture
No
1008:industry
Industry is Logistics
given
fixture
Yes
1008:latest_headcount
Latest headcount is 640 (uncertain: Headcount 640 is outside the listed size range "11-50". The size facts contradict each other, so none of them can be trusted.)
uncertain
fixture
No
1008:xcheck:website_domain
Website host matches the primary domain (tidewelllogistics.example)
cross checked
derived
Yes
Pocket Rocket Studio
ID 1009
FlaggedNot enriched (cap reached)
Not assessed: not enriched
Validation issues
size_headcount_mismatchRevenue "$10M-$50M" with 4 employees implies more than $500,000 per employee, which is implausible.
Outreach
No outreach draft.
Evidence facts (9)
Fact ID
Claim
Status
Source
Citeable
1009:name
Company name is Pocket Rocket Studio
given
fixture
Yes
1009:primary_domain
Primary domain is pocketrocketstudio.example
given
fixture
Yes
1009:website
Website is https://pocketrocketstudio.example
given
fixture
Yes
1009:professional_network_url
LinkedIn page is https://www.linkedin.com/company/pocket-rocket-studio
given
fixture
Yes
1009:employee_count_range
Employee count range is 1-10 (uncertain: Revenue "$10M-$50M" with 4 employees implies more than $500,000 per employee, which is implausible. The size facts contradict each other, so none of them can be trusted.)
uncertain
fixture
No
1009:industry
Industry is Design
given
fixture
Yes
1009:latest_headcount
Latest headcount is 4 (uncertain: Revenue "$10M-$50M" with 4 employees implies more than $500,000 per employee, which is implausible. The size facts contradict each other, so none of them can be trusted.)
uncertain
fixture
No
1009:revenue_range
Revenue range is $10M-$50M (uncertain: Revenue "$10M-$50M" with 4 employees implies more than $500,000 per employee, which is implausible. The size facts contradict each other, so none of them can be trusted.)
uncertain
fixture
No
1009:xcheck:website_domain
Website host matches the primary domain (pocketrocketstudio.example)