Pulling a research universe

The universe, adjusted and total-return prices, the benchmark, fundamentals as they were known and macro: the calls, in order, and where each one can quietly mislead.

Overview

A backtest needs five things: a universe that includes the names that died, prices you can take returns from, fundamentals as they were known on the day, a benchmark, and macro. Each has a call. Each also has one way to produce a plausible wrong answer, and this page names it.

Two calls look interchangeable and are not. quote_history’s raw close is unadjusted: it reads a 2:1 split as a −50% move. panel’s close_return adjusts the previous close across a split, grouping or bonus, but it is a price return, not a total return. Take returns from quote_history’s close_adj (its default field) or close_total_return. And IBOV11 is not the index: use index_history.

1. The universe: research_universe

from silo_client import SiloClient

silo = SiloClient()
pairs = silo.research_universe(as_of="2024-06-03")   # who could be traded that day

One row per ticker + ISIN pair of listed shares and units traded on the B3 cash market since 2019-01-02 (about 640 pairs on 2026-09-30). Membership is the ISIN’s own instrument code, so subscription receipts, BDRs, funds and indices are out. Columns: ticker, isin, instrument_type, cnpj, cnpj_basis, first_observed, last_observed, n_sessions, setor_current, built_at.

The survivorship rule. A rebalance on date T trades the pairs with first_observed <= T <= last_observed. That is what as_of=T does, on the client. Without it you get every pair, including names that stopped trading years ago. Choosing today’s survivors and looking backwards is the classic survivorship bias.

Read these five before you trust a row.

• The ISIN is the identity, and a rename is two rows. ELET3 until 2025-11-07 and AXIA3 from 2025-11-10 are separate pairs; nothing links them.
• A gap still matches. NATU3 is one ISIN with no sessions from 2019-12 to 2025-07, so the filter says it “existed” on a date inside the gap. n_sessions far below the calendar span is the sign; check the series before assuming it traded on T.
• first_observed and last_observed are facts about SILO’s tape, not listing dates. There are no listing or delisting dates and no is_active.
• last_observed lags the tape by up to a day (the view is rebuilt daily), so as_of=today can miss a name that traded this morning.
• setor_current is CVM’s classification as of today, never the sector on T. Using it to neutralise a 2020 portfolio is look-ahead.

cnpj_basis says how the company link was made: fca_ticker (CVM’s published ticker map has that exact ticker), fca_issuer_stem (the ticker’s four-letter stem maps to exactly one CNPJ: an inference, labelled as one) or null (no link; cnpj and setor_current are null too). Nothing is matched by name.

2. Prices: quote_history

One ticker per call, one JSON object per session, holding only the fields you select. With no selection you get ticker, trade_date and close_adj; the raw close, OHLC and volume are one explicit selection away (fields=["close", "volume"]).

Field	What it is
close_adj (default)	Close per single share, continuous across splits, groupings and bonus shares, anchored to the ticker’s latest session
close_total_return	The same with cash distributions reinvested on the ex session
close_total_return_null_reason	Why close_total_return is NULL on a session, else NULL
data_revision	The revision of the data behind the response

Anchored to the latest session means a past level changes when a new event lands; a return does not. Never append a new pull to an old one: re-pull the whole series, or take returns inside one pull. Every response carries data_revision, and pages or pulls with different revisions are never combined. prices() and quote_history_all() restart when it changes in the middle of a walk.

close_adj refuses instead of guessing. A window it cannot adjust raises 22023 (reason=adjustment_unavailable) naming the ticker, the period and the cause: the paper is not a share or a unit, the issuer’s corporate events are not proven swept (or were proven before the last session), or a stock event this version does not adjust (a spin-off, a merger, a capital restitution in shares) falls inside the window. It is never the raw close under another name. Dividends, JCP and subscription rights are not adjusted: they move value to holders and change no share count, so they belong to total return.

close_total_return does not refuse by itself: a NULL is a reason to read, not a gap to fill. Select it without close_adj and an unservable session comes back NULL with its reason; select it beside close_adj and the whole call is refused whenever close_adj is. It is NULL where close_adj cannot be served for the session, where a distribution cannot be valued, and where B3’s history holds no resolved distribution for the ISIN. A non-payer and an issuer the history does not match look identical, so neither is given a price return labelled as a total return.

rows = silo.quote_history_all(
    "PETR4", "2019-01-02", "2024-06-03",
    fields=["close_adj", "close_total_return", "close_total_return_null_reason"],
)

The series follows the ISIN across boards (62 share and unit tickers changed board since 2019, and a default that followed only the latest board used to drop the earlier sessions of ETER3 without a word). It refuses what it cannot serve whole, each with a reason=: unknown_ticker, outside_coverage, isin_change (two ISINs the ticker’s lineage does not splice; a reused receipt code is a new instrument and is never joined), ambiguous_session, invalid_field and adjustment_unavailable. The tape starts on 2019-01-02, and a window that starts before an instrument’s first session is outside_coverage, even when it would hold no rows. Inside the coverage a missing session is a session with no trade (COTAHIST lists only papers that traded); prior_no_trade_sessions counts them.

Many tickers: prices

There is no multi-ticker function, and a name over one page takes one call per page. The SDK walks every page of every ticker for you and returns the whole request or raises:

tickers = sorted({p["ticker"] for p in pairs})[:100]      # your own selection rule
df = silo.prices(tickers, "2019-01-02", "2024-06-03")     # polars: ticker, trade_date, close_adj

All the pages come from one data revision: if a load lands in the middle, the whole call restarts (SiloRevisionChanged after three tries). When several tickers are asked for and any is refused, prices() finishes the walk and raises SiloRefusals naming every refused ticker with the server’s reason, so a hundred-name pull never silently loses three names. Nothing is dropped or filled in. fields=["close_adj", "close_total_return"] selects more; the result has one row per ticker and session, sorted by ticker and date.

It walks the tickers one after another. On 2026-09-28 the same walk over 100 liquid names took about 70 seconds. The server drew no rate limiting on concurrent callers that day, but a concurrent client is not built, and pages from different revisions must never be combined, so run your own threads only if every walk checks data_revision. Nothing raises the 1,000-row page, for anyone.

Returns from the levels

import polars as pl

wide = df.pivot(on="ticker", index="trade_date", values="close_adj").sort("trade_date")
names = [c for c in wide.columns if c != "trade_date"]
returns = wide.select(
    "trade_date",
    *[(pl.col(c) / pl.col(c).shift(1) - 1).alias(c) for c in names],
)   # a session with no trade is a null level: the return across it is null, never bridged

Do not forward-fill the levels first: a missing level then becomes a zero return, and a multi-session move becomes a one-day return.

3. The benchmark: index_history

ibov = silo.index_history_all("IBOV", start="1968-01-02")    # 15 pages; or pass a window

B3’s own published daily levels, as published: index_code, trade_date, level, divisor_step, source. IBOV is held from 1968-01-02; IBXX, IBXL, IFIX, SMLL, IDIV, ICON, IMOB and UTIL each from its own first session (coverage() has the depth of each).

• It takes an index code, never a ticker. BOVA11 is an ETF; IBOV11 is the Ibovespa options settlement code: its asset_class is index, and each print is that day’s settlement index, never the official close (in 2026 none of its 181 prints equals the close; they differ by 0.18% on average and by up to 1.07%). It printed on expiry days only, 12 a year, through 2024, weekly in 2025, and on nearly every session since December 2025 (181 of 187 sessions of 2026), so a dense IBOV11 series is not a sign that it is the index. Neither is the index, so both are refused with 22023, naming the codes held.
• Every code is a total-return index, as B3 labels it. B3’s own pages and its Manual (Feb 2023, section 1.2) call IBOV, IBXX, IBXL, IFIX, SMLL, IDIV, ICON, IMOB and UTIL indices of retorno total: distributions are reinvested in the index, so a level already includes dividends. Compare it with close_total_return from quote_history, never with close_adj, which is price only. Where B3 has a price-return version (IDIV B3 Price Return) it carries its own name and is not served. Levels before an index’s publication date are B3’s back-calculation, and the endpoint does not mark them.
• It is not adjusted. B3 re-scaled IBOV eleven times (1983-10-04 by 100 and ten sessions by 10, the last on 1997-03-03). divisor_step is true on the first session after each, where a level ratio is not a return. Exclude or splice those sessions.
• start=None is the last 365 days, not the whole history.

4. Fundamentals as they were known: as_of

By default the statement functions read each document at its latest stored version, so a later filing or a restatement appears as if it had been known earlier. That is look-ahead. financials, company_financials, income_statements, balance_sheets and cash_flow_statements take as_of:

# PETR4's 2023 annual statements were received on 2024-03-08
silo.financials("PETR4", "DRE", start="2023-01-01", end="2024-06-30", as_of="2024-02-29")
# newest period: the quarter ended 2023-09-30

Only documents CVM had received before the date are read (one received on the date itself is out), the highest remaining version of each is kept, and a document with no filing header is dropped. ref_date still says which period a row is for; start / end window on that, not on the receipt date.

Stale, never early. CVM’s files carry only the newest version of each document, and most versions superseded before 2026 are not held. For a company that restated, an as-of read shows the restated numbers only once they were filed, but it may be missing the original. financial_statement_history lists every stored version with its filing_received_date.

These functions refuse above one page and have no cursor. A long window on several statements can exceed 1,000 lines, so read one statement at a time and narrow the window.

5. Macro: split by date

macro_series has no cursor; the caller splits by date. Measured on 2026-09-28: CDI (a daily series) refuses over four years and three-year chunks fit, so 2019 onward is three calls; SELIC_META fits about 2.7 years per call. A refusal is cheap and says why, so it is safe to start wide and halve. Read unit on every row: the series differ (CDI is percent per business day, SELIC_META percent a year).

Not served

Historical index membership, BDRs in the universe, a sector history, spin-offs, mergers, subscriptions and any link across a rename, fundamentals before 2019, and recovery of superseded ITR/DFP versions. close_adj and close_total_return exist only for shares and units; for any other paper the adjusted close is refused and the raw close is one explicit selection away.

See also Quotes, Reading a fundamental, Paging without losing rows, Known limitations and the SDK.