Features
Everything between a publisher's feed and a clean, queryable API response.
Source ingestion
- RSS and Atom feeds from approved publishers, polled on a schedule (30 or 60 minutes by default).
- Conditional requests (ETag / Last-Modified), timeouts, bounded retries and exponential backoff per source.
- One failing source never blocks the others; health is tracked per source.
- Strict network safety: private networks, metadata endpoints, oversized and unsafe XML are refused.
Normalisation & de-duplication
- Canonical URLs with tracking parameters removed, publisher GUIDs and content fingerprints prevent repeated records.
- Publication time is kept separate from ingestion time, in UTC, returned as ISO 8601.
- Syndicated copies are linked as duplicates; independent coverage of one event is kept and can be grouped into event clusters.
AI enrichment
- Runs asynchronously after ingestion — never on API reads.
- Validated structured output: exactly one of 12 categories, a controlled subcategory, entities, topics and geography.
- AI summaries that keep allegations, opinions and forecasts distinct (Growth and Enterprise).
- Provider-independent: the model is configurable, and every result records provider, model and prompt version.
Knowledge registries
- Canonical entities with aliases; uncertain matches are kept ambiguous or sent for review, never silently merged.
- Curated topics: new suggestions are proposed for review rather than published automatically.
- A Nigerian gazetteer of states, cities and geopolitical zones, distinguishing mentioned locations from the event location.
Search
- Keyword search with Postgres full-text ranking.
- Semantic search over pgvector article embeddings (Growth and above).
- Hybrid search using Reciprocal Rank Fusion of keyword and semantic rankings.
Developer platform
- API keys stored only as keyed hashes, with rotation and instant revocation.
- Server-side rate limits and monthly quotas, with standard rate-limit headers.
- Usage analytics by day, endpoint and key in the dashboard.
- OpenAPI specification generated from the running service.