Journal Trust Rating — methodology
Version 1.0 · September 2026 · public, reproducible, evidence-linked
Principles
- Only public evidence. Every check reads a public registry (ISSN Portal, Crossref, DOAJ, OpenAlex, Keepers) or the journal's own site, and records the source URL.
- Same rule for every journal. Weights are fixed and published; no manual scoring.
- Live, not listed. Ratings expire after 7 days and are recomputed; watched journals are re-rated weekly. A journal that stops maintaining itself loses its grade.
- Rating is separate from remediation. Journals cannot buy a rating; they can fix items, and the next check reflects it. Smart Scholars' tools that fix items are sold separately and disclosed.
- Not a quality judgement of articles. JTR measures registration, maintenance and visibility — the conditions for research to be found, counted and preserved.
Pillars and weights
| Pillar | Points | Checks |
|---|---|---|
| Identity & governance | 25 | ISSN registered (8) · key-title match (3) · HTTPS site (3) · board (4) · ISSN on site (3) · contact (2) · publisher (2) |
| Editorial integrity | 25 | peer review (7) · ethics (6) · plagiarism screening (3) · red flags (9, −4 each) |
| Technical standing | 20 | Crossref DOIs (6) · metadata coverage (8) · DOI links (3) · OAI-PMH + GS tags (3) |
| Preservation & access | 15 | Keepers / archives (6) · CC licence (4) · fee transparency (3) · OA statement (2) |
| Reach & citation | 15 | DOAJ (6) · OpenAlex works + citations (5) · h-index (2) · other listings (2) |
Grades: A ≥ 85 · B 70–84 · C 55–69 · D 40–54 · E < 40. "U" when fewer than 40 points' worth of checks could be performed (site unreachable and no registry data). Two or more red flags cap the grade at D regardless of score.
Mapping to the UGC suggestive parameters (post-CARE)
The UGC framework groups ~36 parameters under eight heads. The table shows which JTR check evidences each parameter family; items UGC leaves to institutional judgement (scope fit, reviewer quality) are not scored.
| UGC parameter family | JTR check(s) | Source |
|---|---|---|
| Journal identity: ISSN registered with the ISSN International Centre | ISSN Portal record (identity.issn), key-title match | ISSN Portal |
| Journal has a functioning, secure website with stable URL | HTTPS live check (identity.site) | journal site |
| Editorial board with names and affiliations, editor-in-chief named | Board detection (identity.board); audit EB01–EB09 where available | journal site / audit |
| Contact details and publisher clearly identified | identity.contact, identity.publisher | site, Crossref |
| Peer-review policy published and followed | integrity.peer; audit PR01–PR10 | site / audit |
| Publication ethics and malpractice statement (COPE-aligned) | integrity.ethics; audit ET01–ET08 | site / audit |
| Plagiarism screening declared | integrity.plag | site / audit |
| No misleading metrics or fast-publication promises | integrity.redflags (fake impact factors, speed promises, unverifiable "ISI" claims); audit ET09 | site / audit |
| DOIs assigned to articles and registered | technical.crossref (Crossref journal record); technical.coverage — articles on the site with no registered DOI, from the DOI Doctor census | Crossref / DOI Doctor |
| Registered DOIs resolve to a live article page | technical.resolve — dead-link count from the last DOI check | doi.org / DOI Doctor |
| Complete article metadata (abstracts, licences, ORCID, references) | technical.metadata (Crossref coverage) | Crossref |
| Machine-readable access (OAI-PMH) and Google Scholar indexing tags | technical.harvest | DOI Doctor monitor / site |
| Digital preservation arrangement | access.keepers (Keepers Registry; Internet Archive / Zenodo mirrors) | ISSN Portal / GetIndexed |
| Licensing and copyright clearly stated | access.licence | site / DOAJ |
| Transparent fees (APC) or no-fee statement | access.apc | site |
| Open access status declared | access.oa | site / OpenAlex |
| Listed in recognised directories/indexes | reach.doaj, reach.indexes | DOAJ / GetIndexed |
| Evidence of scholarly use (citations) | reach.openalex, reach.hindex | OpenAlex |
Journal Risk Radar — the fifteen rules
A second, separate score: not "is the journal maintained to indexer standard" (that is the JTR) but "does the public record show the behaviour that has preceded delisting, hijacking and retraction waves". Each signal is a rule with a threshold; points add to a score out of 100 — Low 0–19 · Guarded 20–39 · Elevated 40–64 · High 65+. Fewer than five measurable signals give "not enough data", never a low score. A journal on the Retraction Watch hijacked-journal checker is at least Elevated; one on an admin-imported removed-journal list is High. Sources: OpenAlex (CC0), Crossref, DOAJ, the ISSN Portal, Retraction Watch. Re-scored weekly for watched journals; the evidence page shows every rule, the number it was applied to and the source link.
| Signal | Max | Rule |
|---|---|---|
| Publication volume surge | 15 | ≥3× the prior three-year average (and ≥100 articles) in the last complete year: 15 · ≥2× (and ≥60): 8 |
| Output has stopped | 10 | no articles in the last two complete years although the journal has published before: 10 |
| Citations mostly from itself | 15 | share of citations to its most-cited recent articles that come from the journal itself — ≥40%: 15 · ≥25%: 8 (needs ≥20 citations) |
| Citations concentrated in one other journal | 10 | one other journal supplies ≥40% of those citations (≥30): 10 · ≥25% (≥20): 5 |
| Authorship shifted to a new country | 10 | the top author country of the last two years holds ≥50% and rose ≥30 points against the three years before: 10 |
| Retracted articles | 15 | ≥3 retractions and ≥1% of all articles: 15 · ≥3: 8 · 1–2: 3 (Retraction Watch via OpenAlex; Crossref retraction notices) |
| AI-writing signals in recent abstracts | 12 | of the abstracts published since last year (20 or more read): ≥40% read as assistant-written by the excess-vocabulary rule, or any abstract carries an assistant fingerprint sentence: 12 · ≥25%: 6 |
| Tortured phrases in recent abstracts | 10 | recent abstracts carrying a paper-mill tortured phrase — 3 or more abstracts: 10 · 1–2: 4 |
| Scope spread thin across fields | 8 | recent articles spread over ≥6 top-level fields each holding ≥8% (with ≥50 articles): 8 · 4–5 fields: 3 |
| Open-access journal not in DOAJ | 5 | OpenAlex says open access, DOAJ has no record: 5 |
| No digital preservation on record | 8 | no archive in the Keepers Registry (ISSN Portal) and none declared to DOAJ: 8 |
| Articles deposited without reference lists or abstracts | 8 | Crossref coverage for current articles — references <20%: 5 · abstracts <20%: 3 (needs ≥20 DOIs) |
| Title or website differ from the ISSN record | 12 | ISSN Portal key-title differs from the indexed title: 6 · the website on the ISSN record is a different host from the one OpenAlex indexes: 6 |
| Title shared with another journal | 6 | another OpenAlex source carries the same title under a different ISSN: 6 |
| On a hijacked- or removed-journal list | 40 | Retraction Watch hijacked-journal checker (a clone of this journal exists): 25 · an admin-imported removed-journal list: 40 |
What it is not: a blacklist, a verdict on any article, a measure of quality, or an impact factor. A score is evidence a committee reads. Not in this version: an editor/author graph, a learned model, Scopus or Web of Science status (not public data).
Manuscript Screen — the signals and their sources
The screen shows what an integrity desk can show, and nothing it cannot: every flagged phrase, the rule it broke and where the rule comes from. It is calibrated for English; for other languages only the structural, reference and picture checks apply. A level is a signal for a person, never a verdict; a clean reading proves nothing either way about authorship.
| Signal | What is read | Source |
|---|---|---|
| Assistant fingerprints | Sentences an assistant writes about itself, left in the text: "As an AI language model…", "Regenerate response", "As of my last knowledge update…" and 17 more. Definite when present. | Found in published papers — Cabanac (Nature news, 8 Sep 2023) and others |
| Excess vocabulary | 148 "style" words whose frequency rose sharply with LLM-assisted writing, counted per 1,000 words; the words are shown. Bars: ≥12 strong · ≥6 notable · ≥3 low (300+ words); a short text of 80–299 words needs ≥20 / ≥12. | Kobak, González-Márquez, Horváth & Lause, Science Advances 11(27), 2025; data github.com/berenslab/llm-excess-vocab (MIT) |
| Tortured phrases | 34 phrase patterns with the standard term they stand for ("counterfeit consciousness" → artificial intelligence). 3 or more strong · 1–2 notable. | Cabanac, Labbé & Magazinov 2021 (arXiv:2107.06751, Table 1); the Problematic Paper Screener's best-known fingerprints |
| References | The reference list through ScholaRef: entries no registry holds, and DOIs that belong to a different paper. ≥30% not found (10+ refs) with a wrong-paper DOI → strong; ≥30% not found, or 2+ wrong-paper DOIs → notable; one → low. | Crossref, doi.org |
| Pictures | Every picture against every other in eight orientations (64-bit difference hash; ≤2 bits apart = the same picture, ≤8 = near-duplicate) and byte-identical files; what each file declares — IPTC DigitalSourceType trainedAlgorithmicMedia, C2PA content credentials and their claim generator, PNG generation chunks, generator names in the metadata. Declared AI-generated or the same picture for different figures → strong; near-duplicate or a generator name → notable. No claim about pictures without metadata. | The files themselves; IPTC and C2PA specifications |
| Structure | Sentence-length spread (coefficient of variation), connective openers, stock framings. Informational; only very flat prose with many openers adds a low signal. | Our own reading |
| Second opinion | Optionally a commercial classifier (Pangram, by its documented API), shown beside the signals as what it is. It never moves the level. | docs.pangram.com |
The same reading is applied to a journal's recent abstracts (Risk Radar signals "AI-writing signals in recent abstracts" and "Tortured phrases in recent abstracts") and to the abstracts of the publications in an institution's vault (policy rule "an abstract with strong signals"). Reports keep counts and short snippets, never the manuscript.
Governance
Methodology changes are versioned and announced 30 days ahead; ratings show the version used. A Rating Committee of independent librarians, editors and research-integrity practitioners reviews the weights annually and hears appeals. Editors may appeal any check with evidence at mou@smartscholars.in; corrections are logged and visible in the journal's history.
Limitations
- Homepage-based checks can miss policies published only on inner pages; the full Journal Audit (121 criteria, whole-site crawl) closes that gap and is used when available.
- Registry outages produce "unchecked" items, never penalties.
- Name matching in publication verification is tolerant of initials; confirm common names manually.
