tanjiti/sec_profile: A Security Information Dataset from secwiki and sec.today

What sec_profile contains

tanjiti/sec_profile addresses a narrow data-curation problem: turning information associated with security websites and public security accounts into files that can be searched, reviewed, and reused. The repository description names secwiki and xuanwu.github.io/sec.today as the intended sources. Its visible contents include a current README, historical annual files such as README_2014.md through README_2021.md, monthly files such as README_202601.md and README_202110.md, image charts, and collections of security books and WeChat articles. The README also describes analysis of security information sites, security trends, and the extraction of security-worker accounts associated with services such as Twitter, WeChat, and GitHub.

The intended users are researchers, analysts, and developers who want a historical collection of security-related web and social-account material. The repository is not presented as a general-purpose threat intelligence platform. There is no documented dashboard, query language, alert service, or authenticated API in the README. The primary language is listed as HTML, which fits the presence of rendered charts and HTML-oriented source material, but the repository also contains Markdown data tables and Python dependency metadata.

Repository structure and information flow

The repository layout suggests a file-based publishing and archival model rather than a service architecture. The current README is the main entry point, while the year and month files act as historical snapshots. README_20.md and README_2026.md are linked as aggregate data views, and the current README includes charts for information-source proportions and preferred-language proportions. The tables are serialized as Markdown, with fields such as `nickname_english`, `weixin_no`, `title`, `url`, and a numeric-looking trailing column. The book table uses `date_added`, `language`, `title`, `author`, `link`, and `size`.

This layout makes the data convenient to inspect through GitHub or a text editor, but it does not establish that an automated pipeline produced the files. The README says the project crawls secwiki and xuanwu.github.io/sec.today, yet the repository description does not provide crawler scripts, source-code documentation, scheduling configuration, database schemas, or a command that refreshes the collection. The repository does contain a `requirements.txt` with packages including `beautifulsoup4`, `requests`, `lxml`, `tldextract`, `matplotlib`, and `numpy`. Those packages suggest scraping, HTML parsing, domain handling, and chart generation, but the package list alone cannot confirm the implementation or the exact data flow.

The safest model is publication by generated files. Readers browse one Markdown file for a period, inspect linked source URLs, and use the images or tables for aggregate observations. Treating the repository as a live mirror would require evidence that the README does not currently document.

Reading the security-information tables

The README gives a concrete example of the kind of record the project publishes. The WeChat section contains security-related article titles and destination URLs, including vulnerability reports, ransomware summaries, AI security material, incident reporting, and training announcements. The links point to `mp.weixin.qq.com`, so a reader can follow a collected item to its original public page. The records also preserve account information, although the visible excerpt shows inconsistent values. Some entries contain blank `weixin_no` fields, while others contain placeholders such as `H`, `X`, or repeated display names.

The book table follows a different pattern. It records titles, languages, links, and sometimes unknown author or size values. This distinction matters for data quality. A populated URL does not prove that the linked content is current, accurate, or still available. Likewise, a missing field does not appear to have a documented sentinel value beyond the literal `unknown` or blank entries visible in the README.

The aggregate charts are useful for orientation, but the README does not state how the percentages are calculated. The images are labelled for source proportions and preferred-language proportions, yet there is no accompanying methodology describing deduplication, date boundaries, classification rules, or treatment of missing values. Those details should be checked before using the charts as research evidence.

Installation and first repository inspection

The README does not document an installation or first-use procedure for running tanjiti/sec_profile as an application. It also does not provide a clone command, package-install command, executable entry point, or example configuration.

A first practical use is repository inspection rather than application execution. Start with `README.md` for the current collection, then open `README_2026.md` for the current annual view and one of the monthly files for a narrower period. The Markdown tables can be searched for titles, account names, URLs, or languages. The chart images under the `data/img` directories provide an aggregate view, while the linked source URLs provide the route back to the published security content.

The repository includes this dependency file:

backports.functools-lru-cache==1.6.1
beautifulsoup4==4.8.2
requests==2.23.0
lxml==4.5.0
matplotlib==2.2.4
numpy==1.16.6

That list records declared Python dependencies, not a supported execution environment. The README does not specify a Python setup procedure, supported Python version, crawler entry point, or environment configuration. The presence of `.python-version` indicates project-level Python metadata, but the repository information does not state its value or whether it is sufficient to reproduce the collection.

Maintenance, licensing, and upgrade limits

The repository is not archived, and its last push was on 2026-10-10. That date indicates recent repository activity at that point, but the README does not say whether automated collection is running, whether every source is reachable, or how often the tables are refreshed. The repository has no GitHub releases, so there is no documented release series or upgrade procedure for consumers to follow. Monthly files indicate a time-oriented collection format, but they do not prove that every month will be produced indefinitely.

The license is listed as unknown. The README does not identify a license file or grant permissions for redistribution, modification, or commercial use. This matters because the repository contains third-party article titles, account identifiers, and source URLs, as well as links to books and public pages. The repository's own reuse terms cannot be assumed from its code or data layout. A user planning redistribution would need to inspect the repository's license information and separately assess the terms of linked sources.

The dependency list also raises maintenance questions. Several versions are fixed to releases from 2019 or earlier, based on the version numbers visible in `requirements.txt`. The README does not document compatibility with current Python, current operating systems, or changed source websites. A fresh clone can be inspected, but a clean regeneration cannot be assumed from the documentation.

When sec_profile is the wrong source

sec_profile is the wrong source when a task requires guaranteed freshness, deduplicated records, full article text, or a machine-readable service. The repository description names crawling and analysis, but the visible evidence emphasizes Markdown tables and charts rather than a queryable database. There is no documented refresh timestamp for individual records, no documented rate limit, and no statement that every source URL is fetched successfully.

It is also unsuitable as the sole source for security decisions. The tables collect public links and account names, but they do not provide an authenticity check for a security claim, a vulnerability assessment, a malware indicator feed, or a documented confidence score. The visible records include reports, opinions, training notices, product announcements, and commentary within the same collection. Readers therefore need to distinguish a primary technical advisory from a news summary or public-account post before relying on it.

A practical alternative is to use a dedicated security advisory, vendor, or threat-intelligence source when the requirement is authoritative incident data, and use sec_profile for discovery and historical browsing. That alternative changes the goal from broad collection and source-oriented organization to source-specific validation. The repository is more useful as an index or research archive than as an authoritative intelligence feed.

Editorial conclusion

Researchers who need a browsable archive of security-site and public-account records may find the repository useful, while users expecting a current intelligence API or documented scraper should first inspect the data files, update workflow, and licensing terms. The repository had its last push on 2026-10-10, but the README does not document release versions or a formal upgrade path.

Frequently asked questions

Official sources

1. Issues
2. README
3. tanjiti/sec_profile on GitHub