Broker Directory & API.
An ETL pipeline over federal datasets keeps a public, searchable directory of 30,000+ broker records in nightly sync, with zero manual intervention.
The challenge.
Sync 30,000+ broker records nightly from federal datasets. Reesor & Associates (a financial services firm of roughly 10 people) needed a searchable, public-facing directory that stayed in sync with regulatory data sources, live at transportationrecovery.com.
The build.
An ETL pipeline over federal datasets with robust diff alerts, feeding WordPress for public search. We built a custom WordPress integration that handles nightly syncs, validates data quality, and provides real-time search.
Stack.
Python · SODA API · Google Cloud Storage · WordPress REST API · Cron orchestration · Redis caching
Results.
- 30,000+ broker records sync nightly
- Sub-second search performance
- Automated data validation
- Zero manual intervention required
OUTCOME: 30,000+ broker records synced nightly, sub-second search, automated data validation, zero manual intervention.
Ready before
the agents arrive.
Tell us what you want to build, we respond within one business day.