Python scripts for working with Electric Capital's crypto-ecosystems data. Scrape repositories, analyze dependencies, pull DeFi metrics.

Tools

Tool	What It Does
report	Generate ecosystem health reports
dependents	Map which projects use which libraries
defi-llama	Pull protocol TVL, yields, and volumes
merge	Add repositories to ecosystem files
check	Validate repository URLs

Directory Structure

/scrape
  /dependents    # GitHub dependency analysis
  /defi-llama    # DeFi data collection
  /languages     # Move language repo scraping

/scripts
  /report        # Ecosystem analysis
  /check         # URL validation
  /merge         # TOML file management

Usage

# Generate ecosystem report
python -m scripts.report.main ethereum --verbose

# Analyze dependencies (~20 min for 5,000 repos)
python3 scrape/dependents/dependents.py owner/repo --list-packages

# Pull DeFi data
python scrape/defi-llama/main.py ethereum solana

Output

Reports, CSVs, and visualizations land in /output:

output/
├── reports/           # Ecosystem analysis
├── defi-data/         # Protocol metrics by chain
├── dependencies/      # Package mappings
└── repository-lists/  # Deduplicated repo lists

Setup

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
export GITHUB_TOKEN=your_token_here  # optional but recommended