http

http

HTTP utilities for BigBrotr.

Provides bounded JSON reading for HTTP responses and bounded file downloads to prevent memory exhaustion from oversized payloads.

Note

This module sits in the utils layer and depends only on stdlib and third-party libraries (aiohttp). It is importable from both nips and services without violating the diamond DAG.

See Also

Nip11InfoMetadata: NIP-11 info fetch that uses read_bounded_json. Finder: Finder API fetch that uses read_bounded_json.

Functions

download_bounded_file async

download_bounded_file(
    url: str,
    dest: Path,
    max_size: int,
    timeout: float = 60.0,
) -> None

Download a file with size enforcement.

Accumulates chunks from the response stream until EOF or max_size is exceeded. If the body exceeds max_size, raises ValueError before writing, preventing disk exhaustion from oversized payloads. Creates parent directories if they do not exist.

Parameters:

• url (str) –

  Download URL.

• dest (Path) –

  Local path to save the file.

• max_size (int) –

  Maximum allowed file size in bytes.

• timeout (float, default: 60.0 ) –

  Total request timeout in seconds.

Raises:

• ClientError –

  If download fails.

• TimeoutError –

  If the request exceeds timeout.

• ValueError –

  If the downloaded file exceeds max_size.

Source code in src/bigbrotr/utils/http.py

async def download_bounded_file(
    url: str,
    dest: Path,
    max_size: int,
    timeout: float = 60.0,  # noqa: ASYNC109
) -> None:
    """Download a file with size enforcement.

    Accumulates chunks from the response stream until EOF or ``max_size``
    is exceeded. If the body exceeds ``max_size``, raises ``ValueError``
    *before* writing, preventing disk exhaustion from oversized payloads.
    Creates parent directories if they do not exist.

    Args:
        url: Download URL.
        dest: Local path to save the file.
        max_size: Maximum allowed file size in bytes.
        timeout: Total request timeout in seconds.

    Raises:
        aiohttp.ClientError: If download fails.
        TimeoutError: If the request exceeds *timeout*.
        ValueError: If the downloaded file exceeds *max_size*.
    """
    dest.parent.mkdir(parents=True, exist_ok=True)
    client_timeout = aiohttp.ClientTimeout(total=timeout)
    async with (
        aiohttp.ClientSession(timeout=client_timeout) as session,
        session.get(url) as response,
    ):
        response.raise_for_status()
        data = await _read_bounded(response, max_size)
        dest.write_bytes(data)  # noqa: ASYNC240  # small atomic write, async I/O overhead unnecessary

read_bounded_json async

read_bounded_json(
    response: ClientResponse, max_size: int
) -> Any

Read and parse a JSON response body with size enforcement.

Accumulates chunks from the response stream until EOF or max_size is exceeded. If the body exceeds max_size, raises ValueError before attempting JSON parsing, preventing memory exhaustion from oversized payloads.

Parameters:

• response (ClientResponse) –

  An aiohttp response whose body has not yet been consumed.

• max_size (int) –

  Maximum allowed response body size in bytes.

Returns:

• Any –

  The parsed JSON value (dict, list, str, int, float, bool, or None).

Raises:

• ValueError –

  If the response body exceeds max_size.

• JSONDecodeError –

  If the body is not valid JSON.

Source code in src/bigbrotr/utils/http.py

async def read_bounded_json(response: aiohttp.ClientResponse, max_size: int) -> Any:
    """Read and parse a JSON response body with size enforcement.

    Accumulates chunks from the response stream until EOF or ``max_size``
    is exceeded. If the body exceeds ``max_size``, raises ``ValueError``
    *before* attempting JSON parsing, preventing memory exhaustion from
    oversized payloads.

    Args:
        response: An aiohttp response whose body has not yet been consumed.
        max_size: Maximum allowed response body size in bytes.

    Returns:
        The parsed JSON value (dict, list, str, int, float, bool, or None).

    Raises:
        ValueError: If the response body exceeds *max_size*.
        json.JSONDecodeError: If the body is not valid JSON.
    """
    body = await _read_bounded(response, max_size)
    return json.loads(body)