Database Developer

---

Description:

Seeking an experienced Database Architect & Data Pipeline Developer to design, build, and maintain an ETL pipeline that collects and processes athlete data from multiple sources using MongoDB and Python. The ideal dev will refine open-source scrapers (see below) or develop new ones if they prefer and wrap them into scalable APIs, ensuring efficient data extraction, transformation, and loading while maintaining data integrity and real-time updates. Summary of the skills and experience require: -Database Design & Architecture: Structure a MongoDB database optimized for storing, indexing, and querying sports data. -ETL Pipeline Development: Establish an automated Extract, Transform, Load (ETL) process to ingest real-time data from multiple sources. -Web Scraping & API Development: Customize and improve existing open-source scrapers, wrapping them into APIs for real-time data retrieval. -Data Merging & Normalization: use algorithm to match and merge athlete data across different sources using fuzzy matching, ensuring consistency and resolving minor name discrepancies between datasets. Can provide one on request. -Pipeline Automation: Ensure real-time updates by integrating data pipeline scheduling tools. -Error Handling & Performance Optimization: Build logging, monitoring, and error-handling mechanisms to maintain a high level of reliability. Budget is firm. We respect developers right to set hourly rates of their desire. Equally, please respect our right to set our budget at our desire. If our budget is less than your minimum rate, we ask that you do not apply. For devs who prove their reliability to actually finish projects and at a high level, there will be more work at significantly higher rates. To apply, please submit your interest and availability for video call (camera on). All LLM generated responses and replies will be disqualified.

Tags: Python, MongoDB, Selenium, Database Architecture, ETL Pipeline, API

Keyword: Python Development

Job Type: Fixed price

Price: $900.0