Jaemin's Research Notes

Paper Review Workflow

4 min read

Paper Review Workflow

A paper review workflow turns a public paper into a verified note, reusable concept updates, and optional synthesis writing. The goal is not to summarize everything; it is to extract claims that can be connected and checked later.

Flow

  1. Triage the paper with Paper triage.
  2. Verify metadata and source links.
  3. Route the paper through Claim routing.
  4. Decompose reusable updates with Paper to wiki extraction.
  5. Use AI paper intake, Computational Biology paper intake, or Formula intake as needed.
  6. Match important equations to Formula pattern catalog before deciding how much derivation belongs in the note.
  7. Run AI Computational Biology Math contract and AI Computational Biology Math readiness gate for multi-axis AI, computational biology, or Math candidates.
  8. Check missing support notes with Coverage matrix.
  9. Choose the writing form: compact wiki note with Paper note format, multi-axis technical note with AI Computational Biology Math paper template, or beginner-friendly blog-style review with Longform paper review guide.
  10. Extract claims using Claim extraction.
  11. Map claims to evidence using Evidence table.
  12. Create a Benchmark card for the benchmark when the evaluation protocol matters.
  13. Map component claims with Ablation map when the paper argues why a method works.
  14. Record limits with Limitation taxonomy.
  15. Record public artifacts using Artifact availability.
  16. Check reproducibility using Reproducibility checklist.
  17. Decide Implementation readiness before spending compute.
  18. Write a Reproduction plan only if the paper is worth rerunning or reimplementing.
  19. Record any rerun, reimplementation, or diagnostic as Reproduction result.
  20. Compare related papers with Paper comparison matrix when useful.
  21. Extract reusable concepts with Concept update contract into Concepts.
  22. Link research relevance into Research.
  23. Synthesize related work with Literature synthesis when a topic has enough papers.
  24. Promote mature themes through the Posts workflow only after the paper notes are stable.

Evidence Levels

  • Metadata verified: title, authors, venue or preprint source, and link are checked.
  • Method understood: objective, architecture, data, and evaluation are identified.
  • Claims checked: results are tied to metric, split, benchmark, and uncertainty.
  • Evidence table complete: main claims have supporting experiments and limits.
  • Benchmark card complete: benchmark scope, split, metric, and leakage risks are explicit.
  • Ablation map complete: component claims are tied to isolated experiments.
  • Baseline checked: comparisons and ablations support the claimed contribution.
  • Limits recorded: failure modes, assumptions, and missing comparisons are explicit.
  • Artifact availability checked: public code, data, splits, configs, weights, logs, predictions, and environment are marked present or missing.
  • Reproducibility checked: code, data, config, compute, and evaluation details are marked present or missing.
  • Implementation readiness checked: the target claim, minimum viable experiment, compute estimate, and failure interpretation are explicit.
  • Reproduction result recorded: the rerun or diagnostic outcome is tied to artifacts, metrics, limitations, and next decision.
  • Synthesis ready: the paper can support a public blog post or research map.

Checks

  • Is the paper worth a curated note, or should it only update an existing concept?
  • Does the paper add a reusable definition, formula, contract, or evidence boundary that belongs in a concept note?
  • Are formulas rewritten with symbol definitions rather than copied blindly?
  • If the note is a public longform review, does it follow Longform paper review guide and fit a 20-30 minute reading budget?
  • Has the paper been routed through Claim routing and the relevant intake page: AI, Computational Biology, or Math?
  • Has the paper been decomposed with Paper to wiki extraction before updating several support notes?
  • Has a multi-axis candidate passed AI Computational Biology Math readiness gate before being promoted?
  • Are metrics connected to Metric and split protocol?
  • Are reported gains larger than confidence intervals or run-to-run variance?
  • Does the paper require statistical significance checks?
  • Are baselines and ablation studies sufficient?
  • Are domain risks such as leakage, scaffold split, protein family split, or invalid geometry noted?
  • Are uncertain claims marked as unresolved?

Graph View

Backlinks