Skip to content

BigBrotr

Modular Nostr network observatory.

BigBrotr discovers relays across clearnet, Tor, I2P, and Lokinet, validates their connectivity, runs NIP-11 and NIP-66 health checks, archives events, materializes analytics views, and exposes everything through a REST API and a NIP-90 Data Vending Machine. Eight independent async services share a PostgreSQL 18 backend, each deployable and scalable on its own. Built on Python 3.11+ with strict typing and asyncio.

Python 3.11+ PostgreSQL 18 License: MIT Documentation

How It Works

Eight independent async services share a PostgreSQL database, each with a distinct role in the discovery-monitoring-archiving pipeline:

flowchart TD
    DB[("PostgreSQL")]

    SE["Seeder<br/><small>Bootstrap</small>"]
    FI["Finder<br/><small>Discovery</small>"]
    VA["Validator<br/><small>Verification</small>"]
    MO["Monitor<br/><small>Health checks</small>"]
    SY["Synchronizer<br/><small>Event collection</small>"]
    RE["Refresher<br/><small>View refresh</small>"]
    AP["Api<br/><small>REST API</small>"]
    DV["Dvm<br/><small>Data Vending Machine</small>"]

    SE --> DB
    FI --> DB
    VA --> DB
    MO --> DB
    SY --> DB
    RE --> DB
    AP --> DB
    DV --> DB
Service Role Mode
Seeder Bootstraps initial relay URLs from a seed file One-shot
Finder Discovers new relays from events and external APIs Continuous
Validator Verifies URLs are live Nostr relays via WebSocket Continuous
Monitor Runs NIP-11 + NIP-66 health checks, publishes kind 10166/30166 events Continuous
Synchronizer Collects events from relays using cursor-based pagination Continuous
Refresher Refreshes materialized views for analytics queries Continuous
Api Exposes the database as a read-only REST API with paginated endpoints Continuous
Dvm NIP-90 Data Vending Machine for Nostr database queries Continuous

Key Features

  • Multi-Network Discovery

    Discover and monitor relays across clearnet, Tor, I2P, and Lokinet with per-network concurrency control and proxy routing.

  • NIP-11 & NIP-66 Compliance

    Full relay information documents and six health check types with kind 10166/30166 event publishing.

  • PostgreSQL Backend

    25 stored functions, 11 materialized views, content-addressed metadata with SHA-256 deduplication.

  • Fully Async

    asyncpg connection pooling, aiohttp HTTP client, per-network semaphore concurrency, graceful shutdown.

  • Docker Ready

    Single parametric Dockerfile for all deployments. Full monitoring stack with Prometheus and Grafana.

  • Type Safe

    Strict mypy, frozen dataclasses with slots=True, Pydantic v2 configuration models.

Architecture

The codebase follows a strict Diamond DAG — imports flow strictly downward:

graph TD
    S["<b>services</b><br/><small>Business logic</small>"]
    C["<b>core</b><br/><small>Pool, Brotr, BaseService</small>"]
    N["<b>nips</b><br/><small>NIP-11, NIP-66</small>"]
    U["<b>utils</b><br/><small>DNS, Keys, Transport</small>"]
    M["<b>models</b><br/><small>Pure dataclasses</small>"]

    S --> C
    S --> N
    S --> U
    N --> C
    N --> U
    C --> M
    N --> M
    U --> M
Layer Responsibility I/O
models Pure frozen dataclasses, enums, type definitions None
core Pool, Brotr, BaseService, Logger, Metrics Database
nips NIP-11 info fetch, NIP-66 health checks HTTP, DNS, SSL, WebSocket, GeoIP
utils DNS resolution, key management, WebSocket transport Network
services Business logic: discovery, validation, monitoring, archiving All

Quick Start

git clone https://github.com/BigBrotr/bigbrotr.git
cd bigbrotr
curl -LsSf https://astral.sh/uv/install.sh | sh  # install uv (one-time)
uv sync --group dev
docker compose -f deployments/bigbrotr/docker-compose.yaml up -d postgres pgbouncer
python -m bigbrotr seeder --once

For a complete walkthrough, see the Getting Started guide.

Documentation

Section Description
Getting Started Installation, quick start tutorial, and first deployment
User Guide Architecture, services, configuration, database, and monitoring
How-to Guides Docker deployment, Tor support, backups, and troubleshooting
Development Dev setup, testing, coding standards, and contributing
API Reference Auto-generated Python API documentation
Changelog Version history and release notes