Search and read arXiv papers
Search and retrieve arXiv academic papers by keyword, author, category, or ID through the free REST API using only curl — no API key or dependencies — with a Python helper script for clean output and the Semantic Scholar API for citation data.
Requirements
- Runs on Linux, macOS, and Windows (skill version 1.0.0 by Hermes Agent, MIT license). Needs only curl and the free arXiv REST API — no API key and no dependencies; the helper script uses only the Python standard library. Semantic Scholar is also free without a key for basic use.
How to use
Searching and retrieving papers works through arXiv's free REST API: no API key and no dependencies, just curl.
The helper script has no dependencies — it uses only the Python standard library.
arXiv does not provide citation data or recommendations; use the Semantic Scholar API for that — it is free, needs no key for basic use (1 request per second), and returns JSON.
Rate limits: arXiv allows about 1 request per 3 seconds with no authentication; Semantic Scholar allows 1 request per second without a key, or 100 requests per second with an API key.
arXiv returns Atom XML — use the helper script or the parsing snippet for clean output.
Semantic Scholar returns JSON — pipe through python -m json.tool for readability.
arXiv IDs come in an old format (hep-th/0601001) and a new one (2402.03300).
Withdrawn papers show a withdrawal notice in the <summary> field (look for "withdrawn" or "retracted").
For withdrawn papers, metadata fields may be incomplete.
Main actions and their commands: search papers with curl "https://export.arxiv.org/api/query?search_query=all:QUERY&max_results=5", get a specific paper with curl "https://export.arxiv.org/api/query?id_list=2402.03300", read the abstract on the web with web_extract(urls=["https://arxiv.org/abs/2402.03300"]), and read the full paper PDF with web_extract(urls=["https://arxiv.org/pdf/2402.03300"]).
The API returns Atom XML. Parse it with grep/sed or pipe it through python for clean output.
Run a basic search by query.
curl -s "https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5"
Parse the XML response into a readable format.
curl -s "https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5&sortBy=submittedDate&sortOrder=descending" | python -c "
import sys, xml.etree.ElementTree as ET
ns = {'a': 'http://www.w3.org/2005/Atom'}
root = ET.parse(sys.stdin).getroot()
for i, entry in enumerate(root.findall('a:entry', ns)):
title = entry.find('a:title', ns).text.strip().replace('\n', ' ')
arxiv_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]
published = entry.find('a:published', ns).text[:10]
authors = ', '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))
summary = entry.find('a:summary', ns).text.strip()[:200]
cats = ', '.join(c.get('term') for c in entry.findall('a:category', ns))
print(f'{i+1}. [{arxiv_id}] {title}')
print(f' Authors: {authors}')
print(f' Published: {published} | Categories: {cats}')
print(f' Abstract: {summary}...')
print(f' PDF: https://arxiv.org/pdf/{arxiv_id}')
print()
"
Search prefixes: all: searches all fields (e.g. all:transformer+attention), ti: searches the title (ti:large+language+models), au: the author (au:vaswani), abs: the abstract (abs:reinforcement+learning), cat: the category (cat:cs.AI), and co: the comment (co:accepted+NeurIPS).
Combine terms with Boolean operators.
# AND (default when using +)
search_query=all:transformer+attention
# OR
search_query=all:GPT+OR+all:BERT
# AND NOT
search_query=all:language+model+ANDNOT+all:vision
# Exact phrase
search_query=ti:"chain+of+thought"
# Combined
search_query=au:hinton+AND+cat:cs.LG
Control sorting and pagination with parameters: sortBy takes relevance, lastUpdatedDate, or submittedDate; sortOrder takes ascending or descending; start is the 0-based result offset; max_results is the number of results (default 10, max 30000).
Sort and paginate result sets using these query parameters.
# Latest 10 papers in cs.AI
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10"
Fetch specific papers by arXiv ID, one or several at a time.
# By arXiv ID
curl -s "https://export.arxiv.org/api/query?id_list=2402.03300"
# Multiple papers
curl -s "https://export.arxiv.org/api/query?id_list=2402.03300,2401.12345,2403.00001"
Generate a BibTeX entry.
After fetching metadata for a paper, generate a BibTeX entry.
curl -s "https://export.arxiv.org/api/query?id_list=1706.03762" | python -c "
import sys, xml.etree.ElementTree as ET
ns = {'a': 'http://www.w3.org/2005/Atom', 'arxiv': 'http://arxiv.org/schemas/atom'}
root = ET.parse(sys.stdin).getroot()
entry = root.find('a:entry', ns)
if entry is None: sys.exit('Paper not found')
title = entry.find('a:title', ns).text.strip().replace('\n', ' ')
authors = ' and '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))
year = entry.find('a:published', ns).text[:4]
raw_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]
cat = entry.find('arxiv:primary_category', ns)
primary = cat.get('term') if cat is not None else 'cs.LG'
last_name = entry.find('a:author', ns).find('a:name', ns).text.split()[-1]
print(f'@article{{{last_name}{year}_{raw_id.replace(\".\", \"\")},')
print(f' title = {{{title}}},')
print(f' author = {{{authors}}},')
print(f' year = {{{year}}},')
print(f' eprint = {{{raw_id}}},')
print(f' archivePrefix = {{arXiv}},')
print(f' primaryClass = {{{primary}}},')
print(f' url = {{https://arxiv.org/abs/{raw_id}}}')
print('}')
"
Read paper content.
After finding a paper, read it: fetch the abstract page for metadata and abstract, or fetch the full paper PDF for conversion to markdown.
# Abstract page (fast, metadata + abstract)
web_extract(urls=["https://arxiv.org/abs/2402.03300"])
# Full paper (PDF → markdown via Firecrawl)
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])
Use the helper script.
The scripts/search_arxiv.py script handles XML parsing and provides clean output for searches by query, author, category, or ID, with options like --max and --sort.
python scripts/search_arxiv.py "GRPO reinforcement learning"
python scripts/search_arxiv.py "transformer attention" --max 10 --sort date
python scripts/search_arxiv.py --author "Yann LeCun" --max 5
python scripts/search_arxiv.py --category cs.AI --sort date
python scripts/search_arxiv.py --id 2402.03300
python scripts/search_arxiv.py --id 2402.03300,2401.12345
Get paper details plus citations from Semantic Scholar by arXiv ID, Semantic Scholar paper ID, or DOI.
# By arXiv ID
curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300?fields=title,authors,citationCount,referenceCount,influentialCitationCount,year,abstract" | python -m json.tool
# By Semantic Scholar paper ID or DOI
curl -s "https://api.semanticscholar.org/graph/v1/paper/DOI:10.1234/example?fields=title,citationCount"
Get citations OF a paper — who cited it.
curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/citations?fields=title,authors,year,citationCount&limit=10" | python -m json.tool
Get references FROM a paper — what it cites.
curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/references?fields=title,authors,year,citationCount&limit=10" | python -m json.tool
Search papers via Semantic Scholar as an alternative to arXiv search; it returns JSON.
curl -s "https://api.semanticscholar.org/graph/v1/paper/search?query=GRPO+reinforcement+learning&limit=5&fields=title,authors,year,citationCount,externalIds" | python -m json.tool
Get paper recommendations from Semantic Scholar.
curl -s -X POST "https://api.semanticscholar.org/recommendations/v1/papers/" \
-H "Content-Type: application/json" \
-d '{"positivePaperIds": ["arXiv:2402.03300"], "negativePaperIds": []}' | python -m json.tool
Look up an author profile on Semantic Scholar.
curl -s "https://api.semanticscholar.org/graph/v1/author/search?query=Yann+LeCun&fields=name,hIndex,citationCount,paperCount" | python -m json.tool
Discover: python scripts/search_arxiv.py "your topic" --sort date --max 10
Assess impact: curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID?fields=citationCount,influentialCitationCount"
Read abstract: web_extract(urls=["https://arxiv.org/abs/ID"])
Read full paper: web_extract(urls=["https://arxiv.org/pdf/ID"])
Find related work: curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID/references?fields=title,citationCount&limit=20"
To get recommendations, POST to the Semantic Scholar recommendations endpoint.
Track authors: curl -s "https://api.semanticscholar.org/graph/v1/author/search?query=NAME"
When generating citations, preserve the version suffix you actually read to prevent citation drift — a later version may substantially change content.
Papers can be withdrawn after submission; when this happens, the signs below apply.
Always check the summary before treating a result as a valid paper.
Source: nousresearch/hermes-agent and its contributors. License: MIT.
Copyright (c) 2025 Nous Research
Translation and editorial adaptation by ClawCases.
>Resources
>Related Use Cases
Benchmark LLMs with lm-evaluation-harness
Benchmark large language models with lm-evaluation-harness, which evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) using standardized prompts and metrics.
Competitor News Monitor
Set up a recurring competitor news monitor that tracks a declared company set and reports only material, new developments with primary-source evidence.
Function-Calling Patterns with Hermes-Function-Calling
Wire a Hermes-family model into your own tool-calling loop using the Hermes-Function-Calling companion repo, whose maintainer-shipped scripts show the exact template, decorator, and recursion flags for live financial data, structured JSON output, and multi-step planning.