Statistics
Repos
822
PRs collected
8,818
Tasks
926
Verified
270
PR retention
5.1%
qualifying / scanned
Task creation
29.2%
verified / attempted
Mean difficulty
7.32
Median difficulty
7.4
Tagged
926
Pipeline funnel repos → PRs → tasks, and success by language
From repos to verified tasks each stage against the one above
| Stage | Count | Kept |
|---|---|---|
| Repos searched | 33,783 | — |
| Repos with qualifying PRs | 803 | 2.4% |
| PRs scanned | 159,534 | — |
| PRs merged | 124,916 | 78.3% |
| PRs qualifying | 8,075 | 5.1% |
| Tasks attempted | 926 | 11.5% |
| Tasks verified | 270 | 29.2% |
imported datasets are excluded from the task rows — they have no PR provenance
Task creation success by language — verified / attempted
Go40.6% (58/143)
Java36.8% (39/106)
C++33.3% (31/93)
TypeScript30.0% (36/120)
JavaScript25.8% (31/120)
C23.1% (25/108)
solidity23.1% (9/39)
Python21.5% (39/181)
ruby20.0% (2/10)
bash0.0% (0/3)
shell0.0% (0/1)
fortran0.0% (0/1)
r0.0% (0/1)
language is read from each task's task.toml, not its directory
Task composition difficulty, language, area, topic and bug-class distributions
Difficulty distribution All batches
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 770 / 156 |
0 | 770 | 156 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 258 | 3.7 | 6.1 | 7.4 | 7.32 | 8.7 | 10.0 |
Score bins
<=30 (0.0%)
3.1-520 (7.8%)
5.1-782 (31.8%)
7.1-857 (22.1%)
>899 (38.4%)
Languages
Python181 (19.5%)
Go143 (15.4%)
JavaScript120 (13.0%)
TypeScript120 (13.0%)
C108 (11.7%)
Java106 (11.4%)
C++93 (10.0%)
solidity39 (4.2%)
ruby10 (1.1%)
bash3 (0.3%)
shell1 (0.1%)
fortran1 (0.1%)
r1 (0.1%)
Area / tier
backend531 (57.3%)
library280 (30.2%)
cli65 (7.0%)
frontend44 (4.8%)
fullstack5 (0.5%)
embedded1 (0.1%)
Top topics
smart-contracts24 (2.6%)
autosubmit20 (2.2%)
testing17 (1.8%)
podio16 (1.7%)
terraform15 (1.6%)
vscode13 (1.4%)
react12 (1.3%)
ibc11 (1.2%)
logging10 (1.1%)
networking10 (1.1%)
ethereum10 (1.1%)
http9 (1.0%)
concurrency9 (1.0%)
api7 (0.8%)
rpc7 (0.8%)
robotics6 (0.6%)
postgresql6 (0.6%)
error-handling6 (0.6%)
telemetry6 (0.6%)
data-structures6 (0.6%)
lsp6 (0.6%)
graphql5 (0.5%)
blockchain5 (0.5%)
security5 (0.5%)
database5 (0.5%)
Bug classes
incomplete-validation51 (5.5%)
missing-feature43 (4.6%)
missing-implementation23 (2.5%)
wrong-default19 (2.1%)
missing-functionality19 (2.1%)
missing-metadata-propagation15 (1.6%)
unknown-defect15 (1.6%)
missing-fallback14 (1.5%)
incomplete-implementation11 (1.2%)
race-condition11 (1.2%)
feature-implementation9 (1.0%)
type-handling-inconsistency9 (1.0%)
feature-addition8 (0.9%)
missing-error-handling8 (0.9%)
missing-validation7 (0.8%)
resource-leak6 (0.6%)
state-management4 (0.4%)
backward-compatibility4 (0.4%)
missing-parameter3 (0.3%)
feature-enhancement3 (0.3%)
Repos searched
33,783
Repos with qualifying PRs
803
PRs scanned
159,534
PRs qualifying
8,075
Tasks produced
926
PR → task
10.5%
Collection funnel by language
| Language | Repos searched | Repos kept | PRs scanned | PRs qualifying | PRs collected | Repos collected | Tasks produced | PR → task |
|---|---|---|---|---|---|---|---|---|
| Go | 2,820 | 100 | 17,272 | 1,521 | 1,621 | 102 | 143 | 8.8% |
| C++ | 4,519 | 100 | 23,891 | 1,205 | 1,326 | 104 | 93 | 7.0% |
| TypeScript | 2,489 | 101 | 23,286 | 1,216 | 1,268 | 102 | 120 | 9.5% |
| Java | 4,321 | 100 | 20,042 | 984 | 1,165 | 104 | 106 | 9.1% |
| Python | 5,431 | 101 | 10,453 | 967 | 1,140 | 107 | 181 | 15.9% |
| JavaScript | 6,467 | 101 | 30,413 | 837 | 890 | 101 | 120 | 13.5% |
| Rust | 2,732 | 100 | 17,026 | 737 | 787 | 101 | 0 | 0.0% |
| C | 5,004 | 100 | 17,151 | 608 | 621 | 101 | 108 | 17.4% |
funnel from filtering_report.md · generated 2026-07-17T01:13:29.571815
source: /gpufs/haoli/code/SWE-Lego-Live/subblock/curator/artifacts/collected_prs
Repos dropped by reason
inactive14,853 (51.0%)
no_ci_config5,607 (19.2%)
low_pr_count4,943 (17.0%)
no_dep_files2,664 (9.1%)
excluded_name941 (3.2%)
low_language_pct129 (0.4%)
PRs dropped by reason
no_test_files88,827 (59.8%)
not_merged33,809 (22.8%)
excluded_title12,801 (8.6%)
too_many_lines6,798 (4.6%)
no_linked_issue5,077 (3.4%)
only_test_files730 (0.5%)
file_count_out_of_range495 (0.3%)
no_diff21 (0.0%)
empty_patch2 (0.0%)
Most-collected repositories top 12 by PR count
| Repository | PRs |
|---|---|
| operator-framework/api | 50 |
| Ptt-official-app/pttbbs-backend | 50 |
| linode/cluster-api-provider-linode | 50 |
| pingcap/tiunimanager | 50 |
| optimizely/agent | 50 |
| sirgwain/craig-stars | 50 |
| device-management-toolkit/console | 50 |
| smartcontractkit/chainlink-common | 50 |
| gardener/gardener-extension-provider-openstack | 50 |
| schmitthub/clawker | 50 |
| localstack/lstk | 50 |
| descope/terraform-provider-descope | 50 |
Collection filters configuration, not measurements
| Setting | Value |
|---|---|
| min_stars | 30 |
| min_merged_prs | 5 |
| min_language_percentage | 0.4 |
| max_days_since_push | 1095 |
| min_issue_body_length | 10 |
| min_files_changed | 1 |
| max_files_changed | 25 |
| max_lines_changed | 1500 |
Total tasks
159
Mean difficulty
7.65
Median difficulty
8.1
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_py
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 138 / 21 |
0 | 138 | 21 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 30 | 4.4 | 6.3 | 8.1 | 7.65 | 9.0 | 9.7 |
Score bins
<=30 (0.0%)
3.1-51 (3.3%)
5.1-78 (26.7%)
7.1-85 (16.7%)
>816 (53.3%)
Languages
Python157 (98.7%)
JavaScript2 (1.3%)
Area / tier
backend73 (45.9%)
library53 (33.3%)
cli29 (18.2%)
frontend3 (1.9%)
fullstack1 (0.6%)
Top topics
autosubmit20 (12.6%)
robotics6 (3.8%)
graphql5 (3.1%)
chemistry4 (2.5%)
logging3 (1.9%)
django3 (1.9%)
cli3 (1.9%)
asapdiscovery3 (1.9%)
signal-processing3 (1.9%)
simulation3 (1.9%)
docking2 (1.3%)
dask2 (1.3%)
alchemiscale2 (1.3%)
machine-learning2 (1.3%)
alchemy2 (1.3%)
sqlalchemy2 (1.3%)
httpx2 (1.3%)
http2 (1.3%)
pygame2 (1.3%)
numpy2 (1.3%)
config-parsing2 (1.3%)
argparse2 (1.3%)
couchbase2 (1.3%)
owlready22 (1.3%)
acme2 (1.3%)
Bug classes
missing-feature9 (5.7%)
missing-implementation5 (3.1%)
missing-functionality5 (3.1%)
incomplete-validation4 (2.5%)
wrong-default3 (1.9%)
feature-addition3 (1.9%)
type-handling-inconsistency3 (1.9%)
missing-parameter2 (1.3%)
missing-fallback2 (1.3%)
missing-metadata-propagation2 (1.3%)
feature-enhancement2 (1.3%)
missing-error-handling2 (1.3%)
language-migration1 (0.6%)
fuzzy-search1 (0.6%)
missing-sensitive-data-redaction1 (0.6%)
incomplete-path-sanitization1 (0.6%)
validation-error1 (0.6%)
emcee1 (0.6%)
incorrect-output-format1 (0.6%)
data-parsing1 (0.6%)
Total tasks
154
Mean difficulty
7.50
Median difficulty
7.8
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_js
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 131 / 23 |
0 | 131 | 23 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 32 | 4.0 | 6.8 | 7.8 | 7.50 | 8.8 | 9.7 |
Score bins
<=30 (0.0%)
3.1-53 (9.4%)
5.1-76 (18.8%)
7.1-88 (25.0%)
>815 (46.9%)
Languages
JavaScript115 (74.7%)
solidity32 (20.8%)
TypeScript5 (3.2%)
Python2 (1.3%)
Area / tier
backend84 (54.5%)
library32 (20.8%)
frontend26 (16.9%)
cli9 (5.8%)
fullstack3 (1.9%)
Top topics
smart-contracts18 (11.7%)
vscode13 (8.4%)
react8 (5.2%)
postgresql5 (3.2%)
openstack5 (3.2%)
ethereum3 (1.9%)
pm23 (1.9%)
obsidian3 (1.9%)
testing2 (1.3%)
defi2 (1.3%)
aragon2 (1.3%)
web32 (1.3%)
postgres2 (1.3%)
networking2 (1.3%)
pivotaltracker2 (1.3%)
vscode-extension2 (1.3%)
mongoose2 (1.3%)
discord2 (1.3%)
sails.js2 (1.3%)
sails2 (1.3%)
bull2 (1.3%)
redis2 (1.3%)
error-handling2 (1.3%)
governance2 (1.3%)
egg-orm2 (1.3%)
Bug classes
missing-feature14 (9.1%)
incomplete-validation8 (5.2%)
missing-validation4 (2.6%)
missing-implementation4 (2.6%)
missing-functionality4 (2.6%)
incomplete-implementation3 (1.9%)
missing-error-handling3 (1.9%)
feature-addition2 (1.3%)
unknown-defect2 (1.3%)
optimization2 (1.3%)
gas-optimization2 (1.3%)
missing-state-update2 (1.3%)
missing-metadata-propagation2 (1.3%)
missing-retry-logic2 (1.3%)
missing-fallback2 (1.3%)
missing-method-implementation2 (1.3%)
feature-implementation2 (1.3%)
api-change1 (0.6%)
missing-field-extraction1 (0.6%)
gas-limit-vulnerability1 (0.6%)
Total tasks
124
Mean difficulty
7.42
Median difficulty
7.5
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_ts
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 99 / 25 |
0 | 99 | 25 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 41 | 5.0 | 6.2 | 7.5 | 7.42 | 8.5 | 10.0 |
Score bins
<=30 (0.0%)
3.1-51 (2.4%)
5.1-715 (36.6%)
7.1-813 (31.7%)
>812 (29.3%)
Languages
TypeScript109 (87.9%)
solidity7 (5.6%)
bash3 (2.4%)
Python2 (1.6%)
JavaScript2 (1.6%)
shell1 (0.8%)
Area / tier
backend79 (63.7%)
library17 (13.7%)
cli15 (12.1%)
frontend12 (9.7%)
fullstack1 (0.8%)
Top topics
ethereum7 (5.6%)
smart-contracts6 (4.8%)
api5 (4.0%)
eth-rpc-adapter5 (4.0%)
ethers5 (4.0%)
react4 (3.2%)
polkadot3 (2.4%)
github-actions3 (2.4%)
angular3 (2.4%)
telemetry3 (2.4%)
ethereum-rpc2 (1.6%)
eth-providers2 (1.6%)
json-schema2 (1.6%)
testing2 (1.6%)
codex2 (1.6%)
sqlite2 (1.6%)
polkadot-js1 (0.8%)
health-check1 (0.8%)
yargs1 (0.8%)
gas1 (0.8%)
ci-cd1 (0.8%)
eth-rpc1 (0.8%)
blockchain1 (0.8%)
transcript1 (0.8%)
openai1 (0.8%)
Bug classes
incomplete-validation11 (8.9%)
wrong-default5 (4.0%)
missing-fallback3 (2.4%)
state-management3 (2.4%)
memory-leak2 (1.6%)
incomplete-implementation2 (1.6%)
api-migration2 (1.6%)
missing-implementation2 (1.6%)
missing-metadata-propagation2 (1.6%)
race-condition2 (1.6%)
unknown-defect2 (1.6%)
api-contract-update1 (0.8%)
state-consistency1 (0.8%)
optional-field-handling1 (0.8%)
missing-tag-support1 (0.8%)
missing-lookup-logic1 (0.8%)
rpc-interface-update1 (0.8%)
caching1 (0.8%)
null-pointer-exception1 (0.8%)
caching-optimization1 (0.8%)
Total tasks
144
Mean difficulty
6.98
Median difficulty
7.2
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_go
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 114 / 30 |
0 | 114 | 30 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 56 | 3.7 | 5.7 | 7.2 | 6.98 | 8.3 | 9.4 |
Score bins
<=30 (0.0%)
3.1-56 (10.7%)
5.1-720 (35.7%)
7.1-812 (21.4%)
>818 (32.1%)
Languages
Go143 (99.3%)
Python1 (0.7%)
Area / tier
backend121 (84.0%)
library19 (13.2%)
cli4 (2.8%)
Top topics
terraform15 (10.4%)
ibc11 (7.6%)
http5 (3.5%)
networking4 (2.8%)
p2p4 (2.8%)
proxy4 (2.8%)
cosmos-sdk4 (2.8%)
redfish4 (2.8%)
retry3 (2.1%)
database3 (2.1%)
grpc3 (2.1%)
kubernetes-operator3 (2.1%)
blockchain3 (2.1%)
terraform-provider3 (2.1%)
linting2 (1.4%)
config2 (1.4%)
cache2 (1.4%)
webhooks2 (1.4%)
opentelemetry2 (1.4%)
logr2 (1.4%)
mcp2 (1.4%)
state-machine2 (1.4%)
scheduler2 (1.4%)
caching2 (1.4%)
configuration2 (1.4%)
Bug classes
incomplete-validation14 (9.7%)
missing-feature6 (4.2%)
race-condition4 (2.8%)
wrong-default4 (2.8%)
missing-fallback3 (2.1%)
missing-metadata-propagation3 (2.1%)
false-positive2 (1.4%)
incomplete-parsing2 (1.4%)
incomplete-implementation2 (1.4%)
backward-compatibility2 (1.4%)
missing-endpoint-support2 (1.4%)
missing-attribute2 (1.4%)
missing-field-support2 (1.4%)
incorrect-parsing1 (0.7%)
whitespace-handling1 (0.7%)
wrong-function-resolution1 (0.7%)
error-context-propagation1 (0.7%)
panic-recovery1 (0.7%)
missing-timeout1 (0.7%)
type-handling-inconsistency1 (0.7%)
Total tasks
141
Mean difficulty
7.04
Median difficulty
7.0
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_c
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 124 / 17 |
0 | 124 | 17 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 34 | 4.4 | 5.8 | 7.0 | 7.04 | 8.4 | 9.7 |
Score bins
<=30 (0.0%)
3.1-54 (11.8%)
5.1-713 (38.2%)
7.1-88 (23.5%)
>89 (26.5%)
Languages
C108 (76.6%)
Python13 (9.2%)
ruby10 (7.1%)
C++8 (5.7%)
fortran1 (0.7%)
r1 (0.7%)
Area / tier
library87 (61.7%)
backend44 (31.2%)
cli7 (5.0%)
frontend2 (1.4%)
embedded1 (0.7%)
Top topics
testing6 (4.3%)
logging6 (4.3%)
data-structures6 (4.3%)
memory-management4 (2.8%)
kernel4 (2.8%)
assembler4 (2.8%)
bootloader4 (2.8%)
cryptography4 (2.8%)
concurrency4 (2.8%)
getopt3 (2.1%)
error-handling2 (1.4%)
build2 (1.4%)
filesystem2 (1.4%)
os-kernel2 (1.4%)
compiler2 (1.4%)
sorting2 (1.4%)
window-manager2 (1.4%)
networking2 (1.4%)
geophysics2 (1.4%)
cryptocurrency2 (1.4%)
crypto2 (1.4%)
secp256k12 (1.4%)
display2 (1.4%)
scheduler2 (1.4%)
string-parsing2 (1.4%)
Bug classes
missing-implementation10 (7.1%)
missing-functionality6 (4.3%)
unknown-defect5 (3.5%)
missing-feature4 (2.8%)
missing-function-implementation3 (2.1%)
incomplete-validation3 (2.1%)
missing-api2 (1.4%)
algorithm-implementation2 (1.4%)
race-condition2 (1.4%)
incomplete-error-handling2 (1.4%)
misra-compliance2 (1.4%)
command-line-parsing2 (1.4%)
lock-contention2 (1.4%)
template-pattern1 (0.7%)
memory-overhead1 (0.7%)
api-compatibility-fix1 (0.7%)
missing-cleanup-api1 (0.7%)
missing-fallback1 (0.7%)
incorrect-path1 (0.7%)
missing-platform-support1 (0.7%)
Total tasks
89
Mean difficulty
7.83
Median difficulty
8.1
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_cpp
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 68 / 21 |
0 | 68 | 21 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 28 | 4.4 | 7.0 | 8.1 | 7.83 | 9.2 | 10.0 |
Score bins
<=30 (0.0%)
3.1-52 (7.1%)
5.1-75 (17.9%)
7.1-86 (21.4%)
>815 (53.6%)
Languages
C++85 (95.5%)
Python4 (4.5%)
Area / tier
backend48 (53.9%)
library41 (46.1%)
Top topics
podio16 (18.0%)
hydra5 (5.6%)
testing4 (4.5%)
rendering4 (4.5%)
opencv3 (3.4%)
concurrency3 (3.4%)
pipeline2 (2.2%)
h2642 (2.2%)
multimedia2 (2.2%)
video-processing2 (2.2%)
point-cloud2 (2.2%)
alpaka2 (2.2%)
iterators1 (1.1%)
templates1 (1.1%)
compilation1 (1.1%)
code-generation1 (1.1%)
schema-evolution1 (1.1%)
websocket1 (1.1%)
file-io1 (1.1%)
aprapipes1 (1.1%)
streaming1 (1.1%)
gpu1 (1.1%)
openh2641 (1.1%)
video-encoding1 (1.1%)
egl1 (1.1%)
Bug classes
incomplete-validation6 (6.7%)
wrong-default3 (3.4%)
unknown-defect3 (3.4%)
missing-feature3 (3.4%)
feature-implementation3 (3.4%)
missing-functionality2 (2.2%)
missing-module-implementation2 (2.2%)
race-condition2 (2.2%)
missing-implementation2 (2.2%)
schema-migration1 (1.1%)
forward-iterator-concept1 (1.1%)
glob-expansion1 (1.1%)
type-handling-inconsistency1 (1.1%)
missing-hash-specialization1 (1.1%)
incomplete-interface1 (1.1%)
code-generation1 (1.1%)
incomplete-standard-support1 (1.1%)
environment-configuration1 (1.1%)
abi-stability1 (1.1%)
misleading-api1 (1.1%)
Total tasks
115
Mean difficulty
7.16
Median difficulty
7.1
Avg patch lines
—
Avg patch files
—
Difficulty distribution swe_tasks_java
| Label breakdown | easy | medium | hard |
|---|---|---|---|
0 / 96 / 19 |
0 | 96 | 19 |
| count | min | p25 | median | mean | p75 | max |
|---|---|---|---|---|---|---|
| 37 | 4.0 | 5.9 | 7.1 | 7.16 | 8.5 | 9.7 |
Score bins
<=30 (0.0%)
3.1-53 (8.1%)
5.1-715 (40.5%)
7.1-85 (13.5%)
>814 (37.8%)
Languages
Java106 (92.2%)
TypeScript6 (5.2%)
Python2 (1.7%)
JavaScript1 (0.9%)
Area / tier
backend82 (71.3%)
library31 (27.0%)
cli1 (0.9%)
frontend1 (0.9%)
Top topics
rpc7 (6.1%)
lsp6 (5.2%)
selenium5 (4.3%)
spring-boot5 (4.3%)
rest-api4 (3.5%)
hbase4 (3.5%)
jackson4 (3.5%)
cose4 (3.5%)
n4js4 (3.5%)
simulacron3 (2.6%)
pay-publicapi2 (1.7%)
idempotency2 (1.7%)
testing2 (1.7%)
refactoring2 (1.7%)
oauth2 (1.7%)
cassandra2 (1.7%)
codegen2 (1.7%)
sonarqube2 (1.7%)
content-assist2 (1.7%)
typescript2 (1.7%)
json-schema2 (1.7%)
pagination1 (0.9%)
error-handling1 (0.9%)
validation1 (0.9%)
api1 (0.9%)
Bug classes
missing-feature6 (5.2%)
incomplete-validation5 (4.3%)
missing-metadata-propagation5 (4.3%)
type-handling-inconsistency3 (2.6%)
missing-fallback3 (2.6%)
incomplete-error-mapping2 (1.7%)
wrong-default2 (1.7%)
unknown-defect2 (1.7%)
incomplete-implementation2 (1.7%)
type-mapping2 (1.7%)
missing-constructor2 (1.7%)
missing-completion-provider2 (1.7%)
missing-parameter-support1 (0.9%)
invalid-request1 (0.9%)
api-gateway1 (0.9%)
invalid-input1 (0.9%)
missing-field1 (0.9%)
pii-leak1 (0.9%)
conditional-field-inclusion1 (0.9%)
missing-error-handling1 (0.9%)
Sample tasks
pick one to see what a task contains