Journal Trust Rating — methodology
Version 2026.10 · from 9 Oct 2026 · public, reproducible, evidence-linked · every version
Principles
- Public evidence, linked. The checks read the ISSN Portal (which also carries the Keepers Registry's archive records), Crossref, DOAJ, OpenAlex and the journal's own homepage, and every check records the address it read. Where a journal uses our own tools, some checks also read their results — the Journal Audit, the DOI Doctor census, DOI check and monitor, GetIndexed's verified index listings and archive copies — and those links open only for the journal's owner, which the rating says beside them.
- What was read is kept. From 9 Oct 2026, every check decided by reading the homepage or the ISSN Portal record keeps the page's SHA-256 fingerprint, the time it was read and the words that decided it.
- Same rule for every journal. Weights are fixed and published; no manual scoring.
- Live, not listed. A rating older than 7 days is made again when anyone opens it, and every journal on an institution's watch-list, in an institution's vault, on a Risk Radar watch or on an author's rating watch is re-rated at least every 7 days. A journal that stops maintaining itself loses its grade.
- Rating is separate from remediation. Journals cannot buy a rating; they can fix items, and the next check reflects it. Smart Scholars' tools that fix items are sold separately and disclosed.
- Not a quality judgement of articles. JTR measures registration, maintenance and visibility — the conditions for research to be found, counted and preserved.
Pillars and weights
| Pillar | Points | Checks |
|---|---|---|
| Identity & governance | 25 | ISSN registered (8) · key-title match (3) · HTTPS site (3) · board (4) · ISSN on site (3) · contact (2) · publisher (2) |
| Editorial integrity | 25 | peer review (7) · ethics (6) · plagiarism screening (3) · red flags (9, −4 each) |
| Technical standing | 20 | Crossref DOIs (5) · metadata coverage (6) · DOI links on the site (2) · OAI-PMH + GS tags (3) · registered DOIs resolve (2) · articles on the site carry a DOI (2) |
| Preservation & access | 15 | Keepers / archives (6) · CC licence (4) · fee transparency (3) · OA statement (2) |
| Reach & citation | 15 | DOAJ (6) · OpenAlex works + citations (5) · h-index (2) · other listings (2) |
Grades: A ≥ 85 · B 70–84 · C 55–69 · D 40–54 · E < 40. "U" when fewer than 40 points' worth of checks could be performed (site unreachable and no registry data). Two or more red flags cap the grade at D regardless of score. Registered DOIs that do not resolve (more than 5%) or articles on the site without a registered DOI (more than a quarter) cap it at B. A check that could not be run scores nothing and is marked unchecked — with one exception kept from version 1.0: when neither the site nor a Journal Audit could be read, "No predatory-style red flags" keeps its 9 points, because no flag was seen. From 9 Oct 2026, a reading that comes back "U" does not replace a journal's last complete grade: that grade stays, dated, with the day the reading failed.
Versions
Every rating names the version it was made under — on the journal's page, in its history and in its JSON. Ratings made before 9 Oct 2026 carry no version and say "before 9 Oct 2026".
- Every rating records the version it was made under; ratings made before 9 Oct 2026 show "before 9 Oct 2026".
- For every check decided by reading the journal's homepage or its ISSN Portal record, the rating keeps what was read: the page's SHA-256 fingerprint, the time, and the words (at most 200 characters) that decided the check — or the words looked for, when none was found.
- A reading that cannot reach enough sources to rate the journal (grade U) no longer replaces its last complete rating: the last grade stays, dated, with "could not be read on <date> — will try again", and the next run tries again.
- No check, weight, threshold or cap changed: the same answers from the registries and the site give the same score and grade as before.
- A source that gives no answer is asked once more, alone, with twice the time.
- When the ISSN Portal does not answer but DOAJ lists the journal under the ISSN, the ISSN counts as registered on DOAJ's word (DOAJ confirms every ISSN at issn.org before listing); the key-title stays unchecked.
- When no address is given, the journal's own site is taken from OpenAlex's homepage or the address DOAJ holds, and read.
- The method as first published (the earliest copy in our history is of 29 Sep 2026): five pillars, 100 points, grades A–E and U, the red-flag cap at D and the maintenance cap at B.
Mapping to the UGC suggestive parameters (post-CARE)
The UGC framework groups ~36 parameters under eight heads. The table shows which JTR check evidences each parameter family; items UGC leaves to institutional judgement (scope fit, reviewer quality) are not scored.
| UGC parameter family | JTR check(s) | Source |
|---|---|---|
| Journal identity: ISSN registered with the ISSN International Centre | ISSN Portal record (identity.issn), key-title match | ISSN Portal |
| Journal has a functioning, secure website with stable URL | HTTPS live check (identity.site) | journal site |
| Editorial board with names and affiliations, editor-in-chief named | Board detection (identity.board); audit EB01–EB09 where available | journal site / audit |
| Contact details and publisher clearly identified | identity.contact, identity.publisher | site, Crossref |
| Peer-review policy published and followed | integrity.peer; audit PR01–PR10 | site / audit |
| Publication ethics and malpractice statement (COPE-aligned) | integrity.ethics; audit ET01–ET08 | site / audit |
| Plagiarism screening declared | integrity.plag | site / audit |
| No misleading metrics or fast-publication promises | integrity.redflags (fake impact factors, speed promises, unverifiable "ISI" claims); audit ET09 | site / audit |
| DOIs assigned to articles and registered | technical.crossref (Crossref journal record); technical.coverage — articles on the site with no registered DOI, from the DOI Doctor census | Crossref / DOI Doctor |
| Registered DOIs resolve to a live article page | technical.resolve — dead-link count from the last DOI check | doi.org / DOI Doctor |
| Complete article metadata (abstracts, licences, ORCID, references) | technical.metadata (Crossref coverage) | Crossref |
| Machine-readable access (OAI-PMH) and Google Scholar indexing tags | technical.harvest | DOI Doctor monitor / site |
| Digital preservation arrangement | access.keepers (Keepers Registry; Internet Archive / Zenodo mirrors) | ISSN Portal / GetIndexed |
| Licensing and copyright clearly stated | access.licence | site / DOAJ |
| Transparent fees (APC) or no-fee statement | access.apc | site |
| Open access status declared | access.oa | site / OpenAlex |
| Listed in recognised directories/indexes | reach.doaj, reach.indexes | DOAJ / GetIndexed |
| Evidence of scholarly use (citations) | reach.openalex, reach.hindex | OpenAlex |
Journal Risk Radar — the fifteen rules
A second, separate score: not "is the journal maintained to indexer standard" (that is the JTR) but "does the public record show the behaviour that has preceded delisting, hijacking and retraction waves". Each signal is a rule with a threshold; points add to a score out of 100 — Low 0–19 · Guarded 20–39 · Elevated 40–64 · High 65+. Fewer than five measurable signals give "not enough data", never a low score. A journal on the Retraction Watch hijacked-journal checker is at least Elevated; one on an admin-imported removed-journal list is High. Sources: OpenAlex (CC0), Crossref, DOAJ, the ISSN Portal, Retraction Watch. Every watched journal, and every journal in an institution's vault, is re-scored at least every 7 days; a watcher hears the week a band rises, and once — separately — the first time the identity, title-sharing or hijacked-list signal appears. The evidence page shows every rule, the number it was applied to and the source link.
| Signal | Max | Rule |
|---|---|---|
| Publication volume surge | 15 | ≥3× the prior three-year average (and ≥100 articles) in the last complete year: 15 · ≥2× (and ≥60): 8 |
| Output has stopped | 10 | no articles in the last two complete years although the journal has published before: 10 |
| Citations mostly from itself | 15 | share of citations to its most-cited recent articles that come from the journal itself — ≥40%: 15 · ≥25%: 8 (needs ≥20 citations) |
| Citations concentrated in one other journal | 10 | one other journal supplies ≥40% of those citations (≥30): 10 · ≥25% (≥20): 5 |
| Authorship shifted to a new country | 10 | the top author country of the last two years holds ≥50% and rose ≥30 points against the three years before: 10 |
| Retracted articles | 15 | ≥3 retractions and ≥1% of all articles: 15 · ≥3: 8 · 1–2: 3 (Retraction Watch via OpenAlex; Crossref retraction notices) |
| AI-writing signals in recent abstracts | 12 | of the abstracts published since last year (20 or more read): ≥40% read as assistant-written by the excess-vocabulary rule, or any abstract carries an assistant fingerprint sentence: 12 · ≥25%: 6 |
| Tortured phrases in recent abstracts | 10 | recent abstracts carrying a paper-mill tortured phrase — 3 or more abstracts: 10 · 1–2: 4 |
| Scope spread thin across fields | 8 | recent articles spread over ≥6 top-level fields each holding ≥8% (with ≥50 articles): 8 · 4–5 fields: 3 |
| Open-access journal not in DOAJ | 5 | OpenAlex says open access, DOAJ has no record: 5 |
| No digital preservation on record | 8 | no archive in the Keepers Registry (ISSN Portal) and none declared to DOAJ: 8 |
| Articles deposited without reference lists or abstracts | 8 | Crossref coverage for current articles — references <20%: 5 · abstracts <20%: 3 (needs ≥20 DOIs) |
| Title or website differ from the ISSN record | 12 | ISSN Portal key-title differs from the indexed title: 6 · the website on the ISSN record is a different host from the one OpenAlex indexes: 6 |
| Title shared with another journal | 6 | another OpenAlex source carries the same title under a different ISSN: 6 |
| On a hijacked- or removed-journal list | 40 | Retraction Watch hijacked-journal checker (a clone of this journal exists): 25 · an admin-imported removed-journal list: 40 |
What it is not: a blacklist, a verdict on any article, a measure of quality, or an impact factor. A score is evidence a committee reads. Not in this version: an editor/author graph, a learned model, Scopus or Web of Science status (not public data).
Manuscript Screen — the signals and their sources
The screen shows what an integrity desk can show, and nothing it cannot: every flagged phrase, the rule it broke and where the rule comes from. It is calibrated for English; for other languages only the structural, reference and picture checks apply. A level is a signal for a person, never a verdict; a clean reading proves nothing either way about authorship.
| Signal | What is read | Source |
|---|---|---|
| Assistant fingerprints | Sentences an assistant writes about itself, left in the text: "As an AI language model…", "Regenerate response", "As of my last knowledge update…" and 17 more. Definite when present. one sentence or more: its strong bar. | Found in published papers — Cabanac (Nature news, 8 Sep 2023) and others |
| Vocabulary and sentence pattern | 76 "style" words whose frequency rose sharply with LLM-assisted writing, counted per 1,000 words, the words shown — since 9 Oct 2026 without the 72 everyday ones ("crucial", "notably", "innovative" …); read with the sentence-length spread and the connective openers. 12 or more per 1,000 (20 or more for a text of 80–299 words): its strong bar · 6 or more per 1,000 (12 or more for a text of 80–299 words): notable · 3 or more excess words per 1,000 (texts of 300+ words); very uniform sentence lengths with many connective openers: low. Shown but not counted when the authors declare language editing, translation, or that English is an additional language for them. | Kobak, González-Márquez, Horváth & Lause, Science Advances 11(27), 2025; data github.com/berenslab/llm-excess-vocab (MIT); the sentence reading is our own |
| Tortured phrases | 34 phrase patterns with the standard term they stand for ("counterfeit consciousness" → artificial intelligence). 3 or more: its strong bar · 1–2: notable. | Cabanac, Labbé & Magazinov 2021 (arXiv:2107.06751, Table 1); the Problematic Paper Screener's best-known fingerprints |
| DOIs that belong to a different paper | The reference list through ScholaRef: a DOI whose Crossref record carries another title. 2 or more: notable · one: low. A reference that could not be matched at Crossref is shown and never counted — books, theses and reports often have no record, so that is not evidence that a reference is made up. | Crossref, doi.org |
| Retracted or withdrawn works cited | Cited works Crossref holds a retraction, partial retraction, withdrawal or removal notice for (its update notices, which carry the Retraction Watch data), each with the link to the notice. 2 or more: notable · one: low. An expression of concern is shown, never counted. | Crossref (update notices) |
| Pictures | Every picture against every other in eight orientations (64-bit difference hash; ≤2 bits apart = the same picture, ≤8 = near-duplicate) and byte-identical files; what each file declares — IPTC DigitalSourceType trainedAlgorithmicMedia, C2PA content credentials and their claim generator, PNG generation chunks, generator names in the metadata. declared AI-generated by the file itself; the same picture for different figures: its strong bar · a flipped, rotated or rescaled reuse; a generator name in the metadata: notable. No claim about pictures without metadata. | The files themselves; IPTC and C2PA specifications |
| The level | Strong needs two independent families: one at its strong bar and another at notable or above. One family alone tops out at Notable. A journal's recent output reads Strong only when at least 3 abstracts were screened and at least 10% of them are Strong on their own; otherwise the abstracts with signals are listed. (From 9 Oct 2026; reports written before keep their level and say so.) | Our own rule |
| Evidence beside the level | Moving no level: the data availability statement and its dataset DOIs (asked of doi.org); the authors' identity signals when given (the address's domain against the institution's ROR record, the ORCID iD's check digit, the institution on OpenAlex's author profile); the duplicate-submission check among journals taking part (a fingerprint of numbers, no text, kept 12 months). | ROR, ORCID, OpenAlex, doi.org |
| Published work | Up to 6 distinctive passages of 9–12 plain words (letters a–z; no headings, figures, citations or common phrasing; the reference list left out) searched as phrases in titles and abstracts, and the registered records holding their rarest words; when the author names the target journal, its recent abstracts compared with the whole text. The figure is the share of the manuscript's five-word phrases found in a source, counted only in runs of 8 or more identical words. Its own level: strong — 25% or more of the manuscript's five-word phrases found in the published sources together; or 50% or more of ONE source's title and abstract reproduced (a source of 40 or more five-word phrases) · notable — 10% or more together; or 25% or more of ONE source's title and abstract reproduced (a source of 40 or more five-word phrases); or an identical passage of 40 words or more · low — 3% or more together; or an identical passage of 20 words or more · none — nothing above; a run of fewer than 8 identical words is never counted — that is the common phrasing of a field. Titles and abstracts only, never full texts: not plagiarism detection and not Crossref's Similarity Check (iThenticate), which compares full texts and is available to Crossref members. | OpenAlex (CC0, search.title_and_abstract); the Smart Scholars Registry |
The same reading is applied to a journal's recent abstracts (Risk Radar signals "AI-writing signals in recent abstracts" and "Tortured phrases in recent abstracts") and to the abstracts of the publications in an institution's vault (policy rule "an abstract with strong signals"). Reports keep counts and short snippets, never the manuscript.
Integrity-weighted output — a Smart Scholars index — not a NAAC or NIRF figure
An institution's publications counted so that publishing more in weak journals does not raise the figure, a retraction takes a paper out of it, and correcting the record never costs anyone anything. It is a Smart Scholars index, introduced on 9 October 2026: it is not a NAAC metric, not an NIRF parameter and is not submitted to either.
| Part | The rule | Source |
|---|---|---|
| What is counted | Every DOI in the institution's vault that Crossref holds (verified and registered). Each DOI is counted once at each level: once for the institution, once for each department it appears in, once for each person — a paper of two departments counts in both, so department figures can add up to more than the institution's. A DOI not verified yet, one Crossref did not answer for, or one Crossref does not hold, has no facts to weigh: it is listed as not counted, never weighed as "not rated". | The vault; Crossref |
| The weight of a publication | The institution's weight for its journal's Journal Trust Rating grade × 0 when Crossref records the publication as retracted (a retraction or partial-retraction notice, or a title that says so). Defaults: A 1 · B 0.8 · C 0.5 · D 0.2 · E 0 · not rated 0.5. An institution may set its own (Settings); every report prints the weights it used and the day they were saved, and a change of weights is said on the page with the figure before and after. | This page's rating; Crossref update notices |
| Open access (beside the number) | The share of counted articles that OpenAlex holds as open access (its open_access.is_oa for the article), over the articles it gave an answer for. Never folded into the weighted figure. | OpenAlex (CC0) |
| Shared data (beside the number) | The share of counted articles whose Crossref record links a dataset — the relation is-supplemented-by or based-on-data — or carries a data-availability statement in its Crossmark assertions. A paper without either may still share its data; Crossref simply holds no link. Never folded in. | Crossref |
| Corrections published (beside the number) | Articles with a correction, erratum, corrigendum, addendum or clarification notice. Shown as openness — correcting the record is what a careful author does — and never a penalty. | Crossref update notices |
| Expressions of concern, withdrawals, retractions | Listed beside the number. A retraction is also the reason a paper weighs 0; an expression of concern, a withdrawal or a removal is shown on the paper's row, not weighed. | Crossref (including the Retraction Watch data Crossref made public) |
What the weight does not look at: the quality of the paper itself (a grade describes how the journal is kept, not the article), the institution's policy verdict (the output page filters by verdict, box by box), the author-list check and the Risk Radar score. When the monthly re-check finds a new retraction, withdrawal, expression of concern or correction on a faculty paper, the institution is told once; nothing is sent when nothing changed.
Governance
Methodology changes are versioned: every rating shows the version it was made under, and the versions are listed with their dates and what changed. A Rating Committee of independent librarians, editors and research-integrity practitioners reviews the weights annually and hears appeals. Editors may appeal any check with evidence at mou@smartscholars.in; corrections are logged and visible in the journal's history.
Limitations
- Homepage-based checks can miss policies published only on inner pages; the full Journal Audit (121 criteria, whole-site crawl) closes that gap and is used when available.
- Registry outages produce "unchecked" items, never penalties.
- Name matching in publication verification is tolerant of initials; confirm common names manually.
