# socios-brasil and turning Brazil's company register into queryable data

> socios-brasil downloads Brazil's Receita Federal company and shareholder records, which arrive as fixed-width files with several record types interleaved, and converts them into gzipped CSV plus schema files for import into a database. The interesting engineering is the fixed-width parsing and the extra holding table, and the interesting constraint is that sensitive columns are censored by default.

**turicas/socios-brasil** — Captura os dados de sócios das empresas brasileiras na Receita Federal e exporta para um formato legível por humanos

- Repository: https://github.com/turicas/socios-brasil
- Website: https://brasil.io/dataset/socios-brasil
- Stars: 612 · Forks: 140
- Language: Python
- License: LGPL-3.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/turicas-socios-brasil

## The problem is the format, not the download

The project is described in the README as a script that downloads all the shareholder data of Brazilian companies from the Receita Federal site, extracts it, cleans it and converts it to CSV. The dataset page lives at brasil.io/dataset/socios-brasil, and the repository is a project of Álvaro Justen and Brasil.IO.

The interesting engineering is what happens after the download. The data published by Receita Federal comes as a fixed-width file, and each line carries a different record type, being company, shareholder, secondary CNAE, header or trailer. That interleaving is what makes any kind of analysis hard, and the reason conversion into friendlier formats is necessary.

Three input tables are covered. The company register includes CNPJ, corporate name, trade name, address, fiscal CNAE and other fields. The shareholder register contains the company CNPJ, the shareholder document, the shareholder name and other fields. And there are secondary CNAEs for each CNPJ, which is the classification of economic activities a company is registered for beyond its primary one.

The shareholder qualification field was defined from the table published by Receita Federal, and it is available in the repository as qualificacao-socio.csv. The README also notes that files with the CNAE names and the registration status are planned, with an issue linked for the details.

The audience is anyone who needs to answer questions about who owns what in Brazil, and the two target readers are data-driven journalists and data scientists.

## Four output files, and the one that is not just a table dump

The script produces four gzipped files, and the fourth is the reason to run it rather than to look for the original dump.

empresa.csv.gz holds the company register. socio.csv.gz holds the shareholder register. cnae-secundaria.csv.gz holds the list of secondary CNAEs. And holding.csv.gz holds the companies that are themselves shareholders of other companies, which is described as the socio.csv.gz file filtered by shareholders of type PJ, that is, legal entities rather than individuals. The stated purpose is to make holdings searches easier, and a shareholder table where a row can be a company makes the ownership graph legible without a self-join.

Alongside those, the schema and schema-full directories hold files that help import the data into a database, and the README points to commands for SQLite and PostgreSQL further down.

Two notes on the format are worth carrying into any import. The .gz extension means the file was compressed with gzip, and the README says to run gunzip on the file, though that is not necessary if you follow the SQLite and PostgreSQL import instructions. And the original character encoding is ISO-8859-15 while the script generates the CSV files in UTF-8, which is a detail that will bite anyone writing an import before reading that line.

There is also a compatibility escape hatch. The README says that if you were using the data in the previous format, there is a file sql/04-create-old-views.sql showing how to convert the new one back to the old standard.

## Censorship is the default, and that is the point

The privacy section is the most consequential part of the documentation, and it describes what the script deletes and why.

The stated purpose is to guarantee privacy, avoid spam and publish only correct data, and the README says this is the standard way of working so as not to facilitate exposure of that data.

In the company table, the codigo_pais and nome_pais columns are deleted because the data they contain is incorrect. The correio_eletronico column is deleted to avoid spam.

In the shareholder table, codigo_pais and nome_pais are deleted for the same correctness reason. Then there is a larger deletion. The complemento, ddd_fax, ddd_telefone_1, ddd_telefone_2, descricao_tipo_logradouro, logradouro and numero columns have their data deleted, left blank, for companies that are individual entrepreneurs, MEI, EI, EIRELI and similar, and these probably correspond to the shareholder's data, a residential address for example. And for the cases of individual companies whose corporate name contains a CPF, which is common for MEIs, the CPF is deleted.

That last group is the one to understand carefully. The reasoning is that a sole proprietor's address and phone in the shareholder record are the person's home details rather than company details, so publishing them would be an inadvertent disclosure of private individuals through an open dataset.

Censorship can be turned off. The README says that if you want to run the script without censorship mode, you edit run.sh and add the --no_censorship option to the extract_dump.py script. That is a deliberate edit to the pipeline rather than a configuration flag, which is a reasonable way to make an unredacted run something you have to consciously do.

## Running it, and the mirror that saves the evening

The dependency set is short and specific. The script depends on Python 3.7, some libraries and the aria2 software, and the install step is:

```bash
pip install -r requirements.txt
```

The requirements file lists click, a development checkout of the rows library from the turicas GitHub repository, lxml, requests, requests-cache, scrapy, and tqdm pinned at 4.66.3.

The run itself is a single script:

```bash
./run.sh
```

The README says you can also run the individual steps separately, and points at run.sh for the details.

The download is the part that needs planning. The README states plainly that the Receita Federal server where the data is hosted is very slow, and for that reason Brasil.IO provides a mirror from which the download can be done much faster. The flag to use it is:

```bash
./run.sh --use-mirror
```

With the caveat that the Brasil.IO mirrors are still in a testing phase and it is not guaranteed that they are always up to date.

The Python 3.7 requirement is worth flagging. It is several major versions behind, and the aria2 dependency is a separate binary to install. Neither is a blocker, but both are friction that a project pinning itself to an old interpreter for years accumulates.

## The supporting scripts and the data around the data

The repository carries more than the main extraction, and the extras are what make the output usable for real questions.

There is a spider that downloads the metadata of economic activities, the CNAEs, from the IBGE site, exposed as the extract_cnae function in run.sh, which downloads the data for versions 1.0, 1.1, 2.0, 2.1, 2.2 and 2.3 and saves it in data/output. The README notes that this script will be improved or changed and links an issue for it.

There is a second crawler for legal natures, natureza_juridica.py, which downloads the data from the Receita Federal site and produces data/natureza-juridica.csv, the register of legal natures mapping to the codigo_natureza_juridica column of the company table.

Then the parsing and import side. extract_dump.py is the main extractor, extract_holding.py builds the holding file, extract_cnae_cnpj.py handles the secondary CNAE extraction, and cnae.py and check_pais_socio.py sit alongside. The headers/ directory holds the record headers, sql/ holds the database views, and test_parser.py is the test for the fixed-width parser, which is the piece most worth having tested given the interleaved record types.

The import path uses the rows command-line interface, and the README says the ideal is to import the data into a database to make queries easier. The Makefile handles linting with autoflake, isort, black at 120 columns and flake8, which tells you the code is held to a consistent Python style.

## Conclusion

socios-brasil is a good fit for a journalist, data scientist or analyst who needs Brazilian company ownership data in a form a database can query, since the conversion from the interleaved fixed-width dump into per-table CSV plus SQL schema is the work that would otherwise be repeated by every user. It is a poor fit if you need the raw columns, because the default mode deletes or blanks addresses, phone numbers, email and country fields, and the unredacted mode is opt-in through a flag on the extract script. Before running it, check whether the Receita Federal download source is reachable from your machine or plan to use the Brasil.IO mirror, and note that the project depends on Python 3.7 and the aria2 downloader, both of which are dated choices you will need to work around on a modern machine.

## FAQ

### What data does socios-brasil download?

It downloads the Receita Federal public data covering the company register with CNPJ, corporate name, trade name, address and fiscal CNAE, the shareholder register with the company CNPJ, shareholder document and name, and the secondary CNAEs for each CNPJ. The original files are fixed-width with several record types interleaved on each line.

### How do I install and run the socios-brasil script?

You need Python 3.7 and the aria2 software, then run pip install -r requirements.txt followed by ./run.sh. The script can also be run against a faster Brasil.IO mirror with ./run.sh --use-mirror, though the README notes the mirrors are still in testing and may not always be current.

### Which columns does socios-brasil delete from the published data?

In the company table the country code and country name columns are deleted because the data is wrong, and the email column is deleted to avoid spam. In the shareholder table the address, phone, fax and related columns are blanked for individual entrepreneurs, and the CPF is deleted where it appears in a sole proprietor's corporate name.

### What files does the socios-brasil script produce?

Four gzipped CSV files: empresa.csv.gz with the company register, socio.csv.gz with the shareholder register, cnae-secundaria.csv.gz with the secondary CNAEs, and holding.csv.gz with the companies that are themselves shareholders of other companies, which is the shareholder file filtered by shareholders of type PJ.

## Sources

- [Issues](https://github.com/turicas/socios-brasil/issues)
- [License: LGPL-3.0](https://github.com/turicas/socios-brasil/blob/master/LICENSE)
- [Project website](https://brasil.io/dataset/socios-brasil)
- [README](https://github.com/turicas/socios-brasil/blob/master/README.md)
- [turicas/socios-brasil on GitHub](https://github.com/turicas/socios-brasil)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/turicas-socios-brasil