Triage Paper
Triage an academic paper into a structured reference summary for the research repo. Use when given an arxiv ID, arxiv URL, DOI, or paper PDF. Creates references/{slug}.md; adds REVIEWED.md entry and REFERENCE_INDEX.md row. Triggers: triage paper, add paper, analyse paper, review paper, literature review, survey paper, benchmark paper, deep dive paper, arxiv, DOI.
npx skills add github:pantheon-org/tekhne --skill documentation--research--triage-paperAdd a new academic paper to the research repo as a structured reference summary.
When to Use
- User provides an arxiv ID (e.g.
2310.08560), arxiv URL, or paper PDF path - User says “triage this paper”, “add this paper”, or “analyse this paper”
- Evaluating whether a paper belongs in the repo
When Not to Use
- The paper has already been triaged (check
REVIEWED.mdfirst) - The paper is clearly out of scope (not related to the research domain)
- User wants a full deep-dive analysis — use
triage-paperfirst, then promote toANALYSIS-*.md
Recommended MCP Servers
When available, prefer the semantic-scholar MCP (primary) and then the google-scholar MCP over WebFetch for paper discovery and metadata resolution. Fall back to WebFetch on the arxiv abstract page only when neither is configured or returns results. The server configuration and the order of preference are in mcp-servers.md.
Philosophy
Triage is a quality gate, not a data-entry task. The goal is a scannable, honest record.
- Evidence first: quote what the paper reports; never infer or embellish claims.
- Triage-then-promote: every paper enters via
REVIEWED.md; promotion toANALYSIS-*.mdrequires a deliberate user decision, never automatic. - Scope over completeness: a well-reasoned rejection is as valuable as a full summary. If the paper is tangentially related, triage it and flag it; do not silently skip it.
Workflow
1. Resolve the source
- If given an arxiv ID or URL: use the
semantic-scholarMCP to resolve metadata (title, authors, date, abstract, DOI). If not configured, fall back toWebFetchon the arxiv abstract page. - If given a DOI: prefer
semantic-scholarorgoogle-scholarMCP over a raw HTTP fetch. - If given a PDF path, read it to extract the same fields.
- Derive a stable slug:
<firstauthor-surname>-<2-3-word-topic>(e.g.jiang-llmlingua,press-longchat).
2. Check for duplicates
- Search
REVIEWED.mdandreferences/REFERENCE_INDEX.mdfor the slug or arxiv ID. - If already present, report it and stop.
3. Classify the paper
Assign one or more tags appropriate to the research domain: survey, benchmark, empirical, theoretical or system. Their meanings are in reviewed-and-index-formats.md.
4. Fill in the reference summary
Read assets/templates/REFERENCE-paper.yaml to get the required frontmatter fields and section structure. Create references/<slug>.md with a YAML frontmatter block (all required_fields from the template) followed by the required sections.
Populate every section:
- TL;DR: 3–8 bullets capturing what the paper does and why it matters.
- What’s novel: one paragraph — what does this do that adjacent work does not?
- Mechanism overview: describe the core approach and methodology. Skip sections that are not applicable (mark
N/A). - Evaluation: quote benchmark name, baselines, key metric, and result — always mark
(as reported). - Implementation details worth stealing: concrete takeaways only.
- Open questions: gaps, risks, unverified claims.
Keep language precise. Do not pad. Quote all numbers with their source.
5. Update REVIEWED.md
Add a row to the summary table at the top (reverse-chronological), then a detailed section below the table with the arxiv ID, authors, date, tags, a 2–3 sentence summary and the disposition pending. The exact formats are in reviewed-and-index-formats.md.
6. Update REFERENCE_INDEX.md
Add a row under the most relevant category table. If no category fits, add it to the closest one and note it for the user.
7. Report and offer next step
Summarise what was created. Then ask:
This paper is now in
REVIEWED.mdas pending. Would you like to:
- Promote it to a standalone
ANALYSIS-arxiv-<id>-<slug>.mddeep dive?- Keep it in REVIEWED.md for now?
- Skip it (mark as not promoted with reasoning)?
8. If user confirms promotion
Read assets/templates/ANALYSIS-paper.yaml to get the required frontmatter fields and section structure. Create analysis/ANALYSIS-arxiv-<id>-<slug>.md with a YAML frontmatter block (all required_fields from the template) followed by the required stage sections. Update the disposition in REVIEWED.md from pending to analysis.
Quick Commands
# Check for duplicate before starting
grep -i "<arxiv-id-or-slug>" REVIEWED.md references/REFERENCE_INDEX.md
# Fetch arxiv abstract (title, authors, date)
curl -s "https://arxiv.org/abs/<id>"
# Create reference file from template
cp templates/REFERENCE-paper.md references/<slug>.md
# Validate the completed file
./scripts/validate-reference-paper.sh references/<slug>.md
./scripts/validate-analysis-paper.sh ANALYSIS-arxiv-<id>-<slug>.md
# | YYYY-MM-DD | <slug> | paper | pending | <one-line description> |
Anti-Patterns
NEVER omit YAML frontmatter
WHY: Files without frontmatter fail schema validation and break indexing tools that rely on structured metadata.
BAD: Start the file with # ANALYSIS: <slug> followed by bold-text fields.
GOOD: Open with a --- YAML frontmatter block containing all required fields before any prose.
NEVER invent benchmark numbers
WHY: Fabricated metrics corrupt the research record.
BAD: "Achieves 87% recall on LongMemEval."
GOOD: "Reports 87% recall on LongMemEval (as reported, Table 3)."
NEVER skip the duplicate check
WHY: Re-triaging the same paper wastes effort and creates conflicting entries.
BAD: Create a new file without checking REVIEWED.md.
GOOD: Run grep -i "<slug>" REVIEWED.md references/REFERENCE_INDEX.md first.
NEVER promote without user confirmation
WHY: Promotion to ANALYSIS-*.md is a quality gate, not automatic.
BAD: Create ANALYSIS-*.md as part of triage. GOOD: Triage to REVIEWED.md, then ask the user.
NEVER infer or embellish claims the paper does not make
WHY: The record must show what the paper reports, so later readers can trust each statement.
BAD: "The method generalises to all model sizes." when only two sizes were tested.
GOOD: "Tested on two model sizes (as reported)."
NEVER silently skip a tangential paper
WHY: A well-reasoned rejection is as valuable as a full summary, and a silent skip leaves no record.
BAD: Drop an adjacent paper without telling the user. GOOD: Triage it, flag the scope question, and let the user decide.
NEVER leave a template section blank
WHY: A blank section reads as an oversight; an explicit marker shows the section was considered.
BAD: An empty “Mechanism overview” for a survey.
GOOD: N/A with a one-line reason for sections that do not apply.
NEVER state an evaluation number without marking it as reported
WHY: Numbers come from the authors and have not been reproduced; the marker keeps that visible.
BAD: "Recall is 87%."
GOOD: "Recall is 87% (as reported)."
NEVER fall back to WebFetch while a configured MCP can answer
WHY: The MCP servers return structured metadata, whereas scraping an HTML page is brittle.
BAD: WebFetch on the arxiv abstract page with semantic-scholar configured.
GOOD: Resolve through semantic-scholar, then google-scholar, and use WebFetch last.
References
- Reference artifacts: YAML template · schema · validator
- Analysis artifacts: YAML template · schema · validator
Reference Material
| Topic | Reference | When to Use |
|---|---|---|
| MCP server setup and source order | mcp-servers.md | Resolving arxiv IDs, DOIs and metadata |
| REVIEWED.md and index formats, tags, promotion | reviewed-and-index-formats.md | Writing steps 3, 5, 6 and 8 |
Explicit NEVER/WHY/consequence triplets with side-by-side bad/good examples. Generic warnings score low — concrete failure modes with real impact (security, correctness, performance) score high.
Runtime validation via eval scenarios proving the skill actually changes agent behaviour. Requires evals/ directory, instructions.json, ≥3 scenarios, and ≥80% instruction coverage in summary.json.
Appropriate rigidity for the skill type: strict rules for critical/security skills, balanced steps for process skills, flexible options for tool skills. Mismatched calibration (too rigid or too loose) reduces score.
Expert-only knowledge the AI couldn't derive on its own — domain-specific patterns, production gotchas, decision frameworks, and non-obvious tool behaviour. High scores mean <5% redundancy with what the model already knows.
A clear philosophy plus numbered step-by-step workflows with explicit entry/exit points. Should answer: what is the core principle, how to execute it, and when (and when not) to apply it.
Rich activation keywords and trigger scenarios in the description so the skill fires when needed. Best descriptions read like an exhaustive list of the tasks and concepts that should invoke this skill.
Copy-paste-ready code, complete runnable examples, real file paths and commands. Pseudocode and abstract descriptions score low — concrete, executable content scores high.
SKILL.md as a <100-line navigation hub pointing to focused references/ files. Content should be lazy-loaded on demand, not front-loaded. Includes actionable 'When to Use' conditions per reference.
Proper frontmatter, single-task focus, activation keywords, and cross-harness portability. Skills must work across 40+ agentic harnesses without hardcoded paths or agent-specific references.
Expert-only knowledge the AI couldn't derive on its own — domain-specific patterns, production gotchas, decision frameworks, and non-obvious tool behaviour. High scores mean <5% redundancy with what the model already knows.
A clear philosophy plus numbered step-by-step workflows with explicit entry/exit points. Should answer: what is the core principle, how to execute it, and when (and when not) to apply it.
Explicit NEVER/WHY/consequence triplets with side-by-side bad/good examples. Generic warnings score low — concrete failure modes with real impact (security, correctness, performance) score high.
Proper frontmatter, single-task focus, activation keywords, and cross-harness portability. Skills must work across 40+ agentic harnesses without hardcoded paths or agent-specific references.
SKILL.md as a <100-line navigation hub pointing to focused references/ files. Content should be lazy-loaded on demand, not front-loaded. Includes actionable 'When to Use' conditions per reference.
Appropriate rigidity for the skill type: strict rules for critical/security skills, balanced steps for process skills, flexible options for tool skills. Mismatched calibration (too rigid or too loose) reduces score.
Rich activation keywords and trigger scenarios in the description so the skill fires when needed. Best descriptions read like an exhaustive list of the tasks and concepts that should invoke this skill.
Copy-paste-ready code, complete runnable examples, real file paths and commands. Pseudocode and abstract descriptions score low — concrete, executable content scores high.
Runtime validation via eval scenarios proving the skill actually changes agent behaviour. Requires evals/ directory, instructions.json, ≥3 scenarios, and ≥80% instruction coverage in summary.json.
No eval scenarios available.
Recommended MCP Servers
Prefer these MCP servers over WebFetch for paper discovery and metadata resolution. They return structured data and avoid HTML scraping.
{
"mcpServers": {
"semantic-scholar": {
"type": "stdio",
"command": "uvx",
"args": ["semantic-scholar-fastmcp"]
},
"google-scholar": {
"type": "stdio",
"command": "uvx",
"args": ["google_scholar_mcp_server"]
}
}
}
Order of preference
semantic-scholaris the primary source: open, structured, and it covers most CS and ML papers.google-scholaris the fallback for papers not indexed there.WebFetchon the arxiv abstract page is the last resort, used only when neither MCP is configured or returns results.
Which source for which input
| Input | Source |
|---|---|
| arxiv ID or URL | semantic-scholar MCP for title, authors, date, abstract and DOI |
| DOI | semantic-scholar or google-scholar MCP, not a raw HTTP fetch |
| PDF path | Read the file and extract the same fields |
REVIEWED.md and REFERENCE_INDEX.md Formats
REVIEWED.md summary row
Add a row to the summary table at the top, in reverse-chronological order:
| <today's date> | <slug> | paper | pending | <one-line description> |
REVIEWED.md detailed section
Add a detailed section below the table:
## <slug> — <Full paper title>
- **arxiv**: <ID>
- **Authors**: <list>
- **Date**: <YYYY-MM-DD>
- **Tags**: <tags>
- **Summary**: <2–3 sentences>
- **Disposition**: pending — awaiting user decision on promotion
REFERENCE_INDEX.md
Add a row under the most relevant category table. If no category fits, add it to the closest one and note it for the user.
Universal tags
| Tag | Meaning |
|---|---|
survey |
Overview paper covering the topic broadly |
benchmark |
Evaluation dataset or framework |
empirical |
Study with experimental evaluation |
theoretical |
Formal or theoretical contribution |
system |
System or implementation paper |
Promotion
When the user confirms promotion, create analysis/ANALYSIS-arxiv-<id>-<slug>.md from assets/templates/ANALYSIS-paper.yaml and change the disposition in REVIEWED.md from pending to analysis.