r/datasets 17h ago

resource 970M+ Usenet (NNTP) text messages discussions dataset via API/WEB resource

5 Upvotes

I spent the last year gathering as much Usenet text data as possible from a large variety of resources and including from about 7000 live active servers as well to fill in gaps.

The end result is a searchable/filterable dataset acessible via web+api (free and paid options)

API : https://www.usenet-rewind.com/api-docs

Web: https://www.usenet-rewind.com

Binaries have been filtered out and range is from posts from 1981-Current and constantly populating. the site includes a histogram of distribution of message counts per year.

I welcome feedback or possible use cases. Built on Apache Solr 10, Mariadb 12 (rocksdb) and NVME disk hardware for fastest operation.

[self-promotion] [free] [paid]


r/datasets 4h ago

dataset [Self-promotion] Free ADA/USDC high-frequency market microstructure data — 20-level order book, spot + futures, 81k snapshots

1 Upvotes

Hi everyone,

I'm shutting down a crypto market research / ML project that I've been running since late 2025, and I've decided to release a free 7-day sample of the market data I collected for anyone interested in quantitative research, market microstructure, backtesting or machine learning.

The sample contains 81,579 ADA/USDC market snapshots with 97 columns.

What's included

  • 20 bid + 20 ask Level-2 order book levels
  • Bid/ask prices and quantities
  • Best bid / best ask
  • Mid price and spread
  • Latest spot trade price and quantity
  • Trade direction
  • Futures open interest
  • Funding rate
  • Mark price
  • Order-book imbalance
  • Market depth at different distances from mid price
  • Liquidity wall ratio
  • Book pressure

The collector runs on a nominal 5-second polling cycle. Because API calls and processing occur between snapshots, the actual median interval in this sample is approximately 6.17 seconds.

The original timestamps are preserved and the public sample has not been artificially interpolated or resampled.

Free download

Hugging Face:
rfab85/crypto-5s-market-data-adausdc-sample · Datasets at Hugging Face

Kaggle:
ADA/USDC High-Frequency Market Microstructure Data

I'd be genuinely interested to hear what people working with L2/order-book data think of the schema and what features you would derive from it.


r/datasets 8h ago

discussion _____finding govt datasets in India_____

1 Upvotes

has anyone ever needed govt datasets for a project and spent significant time finding/accessing it? or is it just me


r/datasets 19h ago

dataset [OC] Monthly open dataset: 1,341 live US data center job ads: advertised pay by role, plus employers and states (CC BY 4.0, DOI)

1 Upvotes

Disclosure first: the ads come from a US data center job board I run, so this is my own data and a link to my own project. Nothing is paywalled or email-gated.

What it is. Once a month I freeze every live ad on the board at one recorded UTC instant and publish the aggregates. August 2026 is the first release:

  • 1,341 live ads, 62 employers, 41 states, 254 city-state pairs
  • 769 ads (57.3%) carry employer-stated pay
  • Two CSVs: aggregates (live jobs by state, role family and employer, with shares) and pay bands (median advertised lower and upper bound by role and pay period)
  • CC BY 4.0

Zenodo, version DOI: https://doi.org/10.5281/zenodo.22261858 — the series DOI https://doi.org/10.5281/zenodo.22261857 always resolves to the newest edition.

Original source, where the method and every edition live: https://datacenterjobhub.com/data-center-hiring-index/2026/08

Method. The population is every ad live and approved at the cutoff, 2026-09-02T10:15:54Z. Pay is employer-stated only: where a job feed supplied its own estimate the ad counts as not stating pay, and ads with a floor but no ceiling are excluded from the bands. Annual and hourly are reported separately and never converted. A role x pay-period group is published only at 8 or more ranges. "Median advertised lower bound" is the median of the ranges' minimums, so it is a statistic about advertised ranges, not about earnings.

One cut that is not in the bundle yet. I ran the stated-pay rate by state against the twelve US jurisdictions that require a pay range in the advertisement itself. This was read at a later cutoff, 2026-09-03T15:10:37Z and 1,351 live ads, so it does not tie exactly to the frozen edition above:

  • 86.1% of ads state pay in the 9 mandate states that have live ads; 51.8% in the other 31
  • dropping the largest employer, which states pay in 174 of its 175 ads, widens that to 84.9% against 44.0%
  • 13 employers run 5 or more ads on each side of the line. Compared with themselves: 93.8% in mandate states, 81.0% elsewhere. So it is not only employer mix
  • of the eight largest data center markets, only California has a posting mandate. Virginia has none and states pay in 76% of 152 ads; Texas manages 41% of 340

That CSV goes into the September edition's folder so it is citable rather than just asserted here.

Limits. One board's ads, not the market: it skews to employers whose careers sites are ingested, and 62 employers is not the industry. Mandate status is as of 2026-01-30. 33 ads carry no state and sit outside the state figures. Every number above comes from a script in the repo.