Lead generation automation (Apollo → Apify → OpenAI → CSV)

Overview

This service runs the code_crafter/apollo-io-scraper Apify actor to pull leads from Apollo.io search URLs, enriches each lead with a personalized welcome message using OpenAI, and returns a downloadable CSV. Designed to deploy on Railway.

Pseudocode

1. POST /leads with input { searchUrls, maxPages?, maxResults? }
2. Validate input and environment (APIFY_TOKEN, OPENAI_API_KEY)
3. Start Apify actor run code_crafter/apollo-io-scraper with { searchUrls, maxPages, maxResults }
4. Poll run until Succeeded, then fetch dataset items
5. For each lead, build a compact profile { company, industry, website, contact }
6. In batches with limited concurrency, call OpenAI to generate a short, polite, tailored welcome message
7. Merge generated message into each lead
8. Convert to CSV (selected columns + message)
9. Return CSV as the HTTP response attachment

Security

• Do not hardcode secrets. Configure via environment variables:
  • APIFY_TOKEN
  • OPENAI_API_KEY
• Avoid logging PII or secrets. Logs are minimal and redact secrets.

Local Development

1. Node 20+ recommended
2. Install deps: npm install
3. Copy env and fill in values: cp .env.example .env
4. Start dev server: npm run dev
5. Test request (replace with your Apollo search URL): curl -X POST http://localhost:3000/leads
   -H 'Content-Type: application/json'
   -d '{"searchUrls": ["https://app.apollo.io/#/people?sortByField=person_name&sortAscending=true&prospected_by_current_team=true&..."], "maxPages": 1, "maxResults": 100}'
   -o leads.csv

Deploy on Railway

1. Create a new Railway project and connect this repo
2. Add environment variables in Railway:
   • APIFY_TOKEN
   • OPENAI_API_KEY
   • PORT (optional; defaults to 3000)
3. Deploy. Railway will run npm run start.

Notes

• Long-running scrapes: For large result sets, the request may take time. Start with small maxResults and increase gradually.
• Rate limits: OpenAI calls use limited concurrency to respect rate limits. Adjust via OPENAI_CONCURRENCY.