When the authors of The Alien Space of Science asked a frontier model for a research idea 300 times, it kept coming back to the same few. They gave Claude Opus 4.7 a catalogue of 273 building blocks drawn from 16,068 papers on language models and asked it, in 300 separate calls, to pick three that would make a novel but feasible project. I counted the picks in their released data. Sparse autoencoders, an interpretability technique, appear in 128 of them. Only 190 of the 300 picks are distinct.
The paper, by Alejandro H. Artiles, Nasim Rahaman and seven co-authors from Tiptree Systems, two Max Planck institutes, Mila, the ELLIS Institute Tübingen and Montréal universities, is a NeurIPS 2026 main-track paper. Version 3 went up on 21 October. Its argument is that a research idea has two separate properties that LLM ideation tends to conflate: whether it holds together, and whether anyone currently working in the field is placed to think of it. The method scores both, then searches for ideas that score high on the first and low on the second. The authors call that region the “alien space of science.”
I read the paper and its appendices, the code repository, the released data and the hosted version built on it. It’s useful for anyone who wants a broad set of research directions instead of one good one. But the headline validation number comes from a setting the sampler doesn’t use, and at the setting it does use, the paper can’t yet show that the field would ever take those ideas up.
Two scores instead of one
The pipeline starts by turning papers into parts. Each of the 16,068 papers (NeurIPS, ICLR, ICML and the main NLP venues, 2017 to 2025) is compressed into a two-page summary of its methods, then broken into “conceptual units”: short statements of a technique and why it matters. Results are left out on purpose, because “experimental results are environment-dependent outcomes that cannot be meaningfully recombined.” The units are embedded and clustered into 273 “idea atoms,” and the average paper is about 3.38 of them. One atom, as written in the released file, begins: “Sparse Autoencoders (SAEs) resolve the problem of neural superposition.”
Two small models are then trained over that vocabulary. The coherence model is a 42.8-million-parameter transformer that reads a set of atoms like a sentence and scores how likely they are to appear together in one paper. The availability model is a dual encoder that compares a candidate set of atoms with the published repertoires of 39,791 authors. It takes the ten closest authors and uses their median similarity, so a single author who publishes on everything can’t make an idea count as available by themselves. High availability means a community of researchers is already close to the idea.
Ranking is one line: standardize both scores across the candidate pool and sort by (1 − β) × coherence + β × unavailability. At β = 0 you get the most paper-like combinations. At β = 1 you get the least available ones, often at the cost of plausibility. The authors swept β and picked 0.7. In that sweep a Gemini 3.1 Pro judge gave each written-up idea a coherence rating, and at 0.8 the mean fell to 3.22, below the 3.32 it gave random triples. A language model then turns each selected triple into a readable proposal that has to use all three atoms.
The idea behind the availability score is older than the paper. Tversky and Kahneman described the availability heuristic in 1973: people judge how likely something is by how easily examples come to mind. In 2023 Sourati and Evans applied it to whole fields. Modelling which experts were placed to reach which conclusions improved prediction of future discoveries “by up to 400%,” and tuning the model to “avoid the crowd” produced what they called “‘alien’ hypotheses unlikely to be imagined or pursued without intervention until the distant future.” The new paper keeps their concept and changes how it is measured. Sourati and Evans walked a graph of researchers and concepts. In machine learning, where people publish across subfields, “nearly every atom pair” is “reachable within two hops,” and the paper reports that “direct hypergraph approaches largely collapse” there. So the authors learn availability from what each author has published instead.
What the evidence shows
The first of the paper’s three claims is that the scores, trained on papers up to 2024, can pick out what got published in 2025. The authors rebuilt everything from pre-2025 papers only (191 atoms, 1,143,135 possible triples), then checked how many three-atom sets from real 2025 papers landed in the top 10,000, which is 0.875% of the space. With coherence alone, 762 of the 2,424 targets did, against 21.2 expected by chance. That is the abstract’s “35.9 times chance.” Adding unavailability lowers the total but raises the share that is new. At β = 0.4 the list caught 453 targets, 40 of them “strictly novel,” meaning their three atoms never appeared together in any earlier paper. That is the “3.24 times chance” in the abstract.
The “35.9 times” comes from the β = 0 setting, which ignores availability entirely. At β = 0.7, the setting used for every idea the paper evaluates, the top 10,000 contains 10 of the 2,424 targets, which is half of what chance would give. The authors report this in Appendix C.2, and they note that unavailable directions “may require a longer evaluation horizon.” It follows from the design: a direction nobody is placed to propose is unlikely to show up in next year’s papers. But it means the temporal test supports the coherence model and the moderate settings. It doesn’t show that the ideas you’d get at 0.7 are directions science will actually take. The paper’s own phrase for the one-year test is “an early proxy rather than a final test.”
The second claim is that the availability model works on researchers it has never seen. The authors took researchers who first appeared in the corpus in 2025 and so were never seen in training. For 2,779 of them they hid three atoms the author had used in different papers but never combined, then asked the model to match authors to these never-published combinations. It did so with an AUC of 0.656, or 0.673 matching in the other direction, where 0.5 is chance. Simple atom-overlap baselines scored below chance. That’s a modest number from a clean test, and it’s the best evidence that “availability” tracks what individual researchers can reach, beyond what the corpus happens to contain.
The third claim, that the ideas are more varied and just as good, matters most in practice. It’s also the one I could check myself.
A wider set of ideas, at the same quality per idea
Each method produced 300 three-atom combinations. The paper measures diversity with the Vendi score, which estimates the effective number of distinct ideas in a set, from 1 to 300. Claude’s 300 independent picks are worth 56.5 distinct ideas. Gemini 3.1 Pro’s are worth 97.4. Asking each model for all 300 in one context, so it could see and spread out its own output, helped a lot: 197.9 for Claude and 182.7 for Gemini. The alien sampler scored 225.2. Uniform random triples scored 250.4, the ceiling, but random triples aren’t selected for coherence.
The repository links the 300 picks from each method on Hugging Face, so I recounted them with my own script. The independent-run numbers match Table 1 exactly: 85 distinct atoms and a Vendi score of 56.5 for Claude, and 118 atoms and 97.4 for Gemini. The released alien set has 299 combinations, not 300, and gives 224.6, close to the paper’s 225.2. My counts also show where Claude’s picks bunched. Sparse autoencoders appear in 128 picks, the atom about concepts stored as linear directions in activation space in 102, and activation patching in 73. All three are interpretability methods. Gemini’s most common atom is also sparse autoencoders, in 77 of its 300. The paper’s embedding map shows the same thing: both models’ ideas cluster “around sparse-autoencoder and interpretability concepts.”
Quality held up, though the evidence is thin in places. Ten paid raters with graduate degrees in machine learning or computer science each read one idea from each method, so 40 ideas got one rating apiece. Alien and Claude tied at 4.03 out of 5, with Random at 3.75 and Gemini at 3.68. That sample is too small to rank anything, and the authors call it small. The larger tests use GPT-5.5 as the judge. In a blind tournament it picked whichever idea was “plausible enough to investigate, but less likely to be proposed by the research community,” which is the sampler’s own goal, so this measures how well each method hits that target. Overall, Alien won 55.6% of its matches, Gemini 54.4%, Claude 48.9% and Random 41.1%. In a second round with only each method’s top half (picked by first-round wins, 180 judgments), Alien’s win rate rose to 72.2% and Claude’s fell to 27.8%.
For me the most useful result comes after the idea stage. The authors ran ten ideas per method through an automated research loop built on Claude Code with Sonnet 4.6, which refined each idea, ran bounded experiments on A100s and wrote a report. GPT-5.5 then compared the reports, and the authors checked that judge against people: on 10 pairs of reports, each read by five of 50 screened experts, it agreed with individual experts 68% of the time, while two experts agreed with each other 58% of the time. Alien’s reports came out best, with an average rank percentile of 0.633 against 0.574 for Claude. The paper also tracked SAE- and feature-level interpretability themes. Two of Claude’s ten starting ideas had them, and eight of its ten final reports did. Gemini went from four to five. None of the alien ideas or reports did. So the drift toward the familiar topic happened during the research itself. With ten ideas per method this is a small sample, but anyone running an agent loop on open-ended research should check for the same drift.
The comparison has limits. The baselines weren’t free brainstorming. Claude and Gemini picked from the same 273-atom menu, and the same decoder (Claude Opus 4.7) wrote every proposal, so the test isolates which combinations get chosen. That is a fair test of the method’s claim, but a narrower one than the claim that the sampler has better ideas than the models. Also, the batch runs, the strongest baselines, were never rated for quality. They were given the quality scores of the same model’s independent runs. The authors argue this is conservative, and I agree, but it is an assumption.
How to use it today
The three ways in have very different costs.
The cheapest is to browse the hosted version. Lacuna is Tiptree Systems’ production version. When I opened it on 14 November its home page listed 2,250,251 papers, 215,519 authors, 28,588 research directions and 25,120 proposals. There is also a Lacuna MCP server that lets a coding agent search those proposals and the directions behind them, and its README says the hosted connection needs “no local Python installation or Lacuna account.” This is the right place to start if you work in machine learning and want to see what the output looks like. Keep in mind who makes it: the paper discloses that two authors, Martin Weiss and Nasim Rahaman, co-founded Tiptree, which “develops commercial research-discovery tools for machine learning researchers,” and that Tiptree funded the work.
The second way is to run the released models. The repository is MIT-licensed and the instructions are short. You create a Python 3.11 environment, install the requirements, download the two model folders and the data from Hugging Face, and run python alien.py sample --top-k 20. That step makes no LLM calls. By default it draws 1,000 candidate sequences, keeps the distinct triples and ranks them at β = 0.7. The README says inference “works on CPU or CUDA” and recommends a GPU for the paper’s exhaustive search over all 3.35 million triples. A second command, alien.py reconstruct, sends the chosen triples to a model through LiteLLM and writes Markdown proposals. That step costs money, so start with --top-k 3. The README also warns that “different candidate pools and numerical backends can change rankings,” so a default run won’t reproduce the paper’s list exactly.
The third is to rebuild it for another field. This is where the method could matter most, and it takes the most work. ADVANCED.md covers it: a TSV of paper IDs and PDF links, Gemini calls to compress and atomize each paper, clustering, then rebuilding both training sets and retraining. The availability model needs real author identities from a database of papers and authors, because “the PDF TSV alone does not supply author identities.” The released availability model was trained on four A100s from 43,077,647 records, a run of 13 hours 42 minutes. A small field is cheaper, but you still need a corpus close to complete. The paper’s argument depends on it: when the model calls a direction unavailable, that should reflect “a real absence from the author community rather than missing data.”
Settings I’d use
The paper fixes β at 0.7, but the code lets you change it. The temporal test suggests how. If you want the combinations your field is most likely to publish next, β near 0 found the most. If you want likely next directions that are also new combinations, 0.4 kept 59% of the β = 0 hits while the share of strictly novel ones rose from 3.8% to 8.8%. If you want directions your field is unlikely to reach without help, use the paper’s 0.7, and accept that the one-year test can’t vouch for those ideas yet. Above 0.7, in the authors’ single LLM-judged sweep, coherence dropped below random. One caution: those numbers come from scoring every triple in a frozen 191-atom vocabulary. The released sample command standardizes scores within 1,000 sampled candidates by default, so the same β won’t behave identically there.
If you don’t use the sampler at all, the cheapest lesson from Table 1 is to have the model generate many ideas in one context instead of many separate calls. That moved Claude from 56.5 effective ideas to 197.9 on the same menu.
The decoder matters too, because the sampler picks atoms and a model still has to write the proposal. Appendix A.13 held 50 alien triples fixed and tried eight decoders. Kimi K3, an open-weight model, came first, Claude Opus 4.7 second, and the 27-billion-parameter Qwen 3.6 third, ahead of GLM 5.2 and Gemini 3.1 Pro. The judge was Gemini 3.1 Pro, told not to reward length. Kimi’s proposals were long, 1,340 words on average, but Ministral 3 8B wrote longer ones and placed sixth, so length doesn’t explain the order. Kimi K3 has 2.8 trillion parameters, so the practical open option for most people is the third-placed 27B Qwen. It is one judge on 50 triples, but it does show you don’t need a proprietary decoder.
I’d add a feasibility check before running anything. The authors say feasibility is “assessed only implicitly and late”: a bad idea fails during automated research, “after compute has been spent on it.” They propose a critique step “between decoding and ranking, covering compute, data availability, and known hardness results.”
And I’d read availability as relative. A low score means no community in this corpus has published near the combination. Someone may still have thought of it. The authors call their estimate “a lower bound” and list what publications leave out: “private reading, tacit knowledge, unpublished projects, and failed attempts.”
Where this is heading
Recombination as a source of new science has good evidence behind it. Uzzi and colleagues analyzed 17.9 million papers and found that the highest-impact work is “primarily grounded in exceptionally conventional combinations of prior work yet simultaneously features an intrusion of unusual combinations.” Foster, Rzhetsky and Evans found in MEDLINE that risky, innovative strategies are rare and that their extra reward “does not compensate for the risk of failing to publish.” Shi and Evans linked surprising combinations to impact and to “scientific outsiders from distant disciplines.” The new paper uses that literature as a search objective: put AI effort where the incentives discourage people from going.
The first open question is scale. A 273-atom menu fits in one prompt, which, the authors note, makes the setup “unusually favorable to the LLM baselines.” Expanding to all of AI gives “roughly 30k atoms.” At that size no prompt can see the whole menu, and the trained scorers, which need no LLM calls during the search, should gain the most. That’s the regime Lacuna seems to target, and I’d want to see the comparison rerun there.
New building blocks are a harder problem. The system “can only recombine concepts already present in the literature.” A truly new technique, an atom no paper contains yet, is outside what it can reach. That’s a hard limit for now.
Then there is time. The real test is whether directions rated coherent and unavailable in 2025 become productive by 2028 or 2030. The authors say “such evidence takes years to accumulate.” Because the code and frozen scores are public, someone can run that test later without trusting the authors.
Last, the authors propose a new way to measure: “availability-adjusted discovery value,” meaning how much coherent search an AI system moves outside what the field would do anyway. I’d take this idea from the paper even if I took nothing else. The yardsticks it says AI-science systems are usually judged by are novelty, quality and downstream task performance, and none of them tells you whether the field would have got there on its own. In the discussion they write: “without an explicit pressure away from the community prior, more powerful systems may simply industrialize the production of familiar ideas.” That fits my notes from a 100-minute research sprint where most AI suggestions got no recorded decision and from Anthropic’s session data, where ideation was 2.1% of Claude Code use. Ideation is a small share of how researchers use these models, and in this paper’s controlled menu test, the models’ picks clustered on a few topics.
The paper also names a risk. Availability “cannot distinguish combinations that are unexplored because they are cognitively unavailable from those the community avoids for ethical or safety reasons.” In biology or chemistry, a search for what nobody is doing could surface what people decided not to do. The authors limit themselves to low-risk literatures and recommend screening outputs. If this moves to other fields, that screening has to be part of the pipeline.
What I’d take from it
With the same menu of 273 methods, the two-score sampler gives a set of ideas about four times as varied as one-at-a-time prompting of Claude, at the same rated quality per idea, and somewhat more varied than batch prompting, whose quality wasn’t rated. I’d quote “35.9 times chance” only with its setting, β = 0, which isn’t the one that produces alien ideas. If you run research agents, the cheapest test to borrow from this paper is to count how many of your final reports ended up on the same topic as each other.
References
- Artiles, A. H., Weiss, M., Brinkmann, L., Rahwan, I., Schölkopf, B., Pal, C., Larochelle, H., Goyal, A. and Rahaman, N. (2026). The Alien Space of Science: Sampling Coherent but Cognitively Unavailable Research Directions. arXiv:2603.01092, v3, 21 October 2026 (first version 1 March 2026). NeurIPS 2026 Main Track.
- Hernández Artiles, A. (2026). alien-space-of-science. GitHub repository (README, DATA.md, ADVANCED.md), MIT licence, last commit 2 November 2026, read 14 November 2026. Released models and data: alien-models and alien-data on Hugging Face.
- Tiptree Systems. Lacuna: A Research Map for Machine Learning and lacuna-research-mcp. Home page counts and README read 14 November 2026.
- Tversky, A. and Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology 5(2), 207-232, September 1973.
- Sourati, J. and Evans, J. A. (2023). Accelerating science with human-aware artificial intelligence. Nature Human Behaviour 7(10), 1682-1696, 13 July 2023.
- Uzzi, B., Mukherjee, S., Stringer, M. and Jones, B. (2013). Atypical Combinations and Scientific Impact. Science 342(6157), 468-472, 25 October 2013.
- Foster, J. G., Rzhetsky, A. and Evans, J. A. (2015). Tradition and Innovation in Scientists’ Research Strategies. American Sociological Review 80(5), 875-908, October 2015.
- Shi, F. and Evans, J. (2023). Surprising combinations of research contents and contexts are related to impact and emerge with scientific outsiders from distant disciplines. Nature Communications 14, 1641, 24 March 2023.