chatWeb ships 75 unpinned requirements, a Python 3.9 image and three spellings of the OpenAI key
ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions based on the content, or summarize the key points.
At a glance
- What is it?
- A retrieval script that crawls pages, splits them into fragments, embeds them and answers questions from the nearest matches, wrapped in a single config file that can start a console, an API or a web UI. The dependency list pins nothing, the container image is Python 3.9, the OpenAI credential is named three different ways, and the documented pgvector build is pinned to an old extension tag.
- Who is it for?
- chatWeb is readable and worth reading if you want to see a retrieval pipeline written out in plain modules with no framework above it, and if you intend to rebuild rather than deploy it. Three things to check before you run it.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 139 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 4, 2026, and from our analysis. They are not legal advice.
Editorial analysis
The OpenAI key is spelled three different ways
The manual install says to copy `config.example.json` to `config.json` and set `open_ai_key` to your OpenAI API key. That is the first spelling, lower case with a separator between the two words.
The Docker section then says that if you do not have the `OPEN_AI_KEY` in your environment variables you can set it in the config, which is the second spelling, upper case with a separator in the same place.
The compose file uses a third:
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}Nothing separates OPEN from AI here. So a user who reads the manual path, sets `open_ai_key`, and then moves to Docker has to work out which of the three names the application actually reads, and the page never states the precedence between the config file and the environment variable.
The config file itself is the right design for the key. The repository commits `config.example.json` but not `config.json`, and the compose file bind-mounts the host copy rather than baking one into the image:
volumes:
- ./config.json:/app/config.jsonSo the credential stays on the host in a git-ignored file. The naming is what undermines that, not the mechanism.
Seventy-five unpinned dependencies on a Python 3.9 base image
The requirements file is seventy-five lines long and not one of them carries a version. The list mixes heavyweight crawlers and parsers (`newspaper3k`, `readability-lxml`, `beautifulsoup4`, `lxml`, `selenium`, `feedparser`, `tldextract`), model and maths libraries (`openai`, `tiktoken`, `numpy`, `scipy`, `scikit-learn`, `nltk`), two vector stores (`faiss-cpu` and `pgvector` plus `psycopg2`), a web stack (`fastapi`, `uvicorn`, `starlette`, `python-multipart`), a UI (`gradio`) and text segmenters (`jieba3k`, `tinysegmenter`, `langdetect`).
Every one of those resolves to whatever is current at install time. A build that worked last year and a build today get different library majors from the same file, which makes a reported bug hard to reproduce.
The container makes the same point from the other direction:
FROM python:3.9Python 3.9 is the runtime, and it is a fixed runtime. Meanwhile the manual instructions say only to install Python3, with no version, so the documented manual path and the container can differ by several minor releases. The dependencies that matter most here are the ones with the fastest breaking cadence, the OpenAI client and the embedding stack, and neither is bounded.
There is a `run.sh` that the container invokes as its command, and the image exposes two ports, 7860 and 9531, of which only 7860 appears in the page.
The summary is a similarity ranking, not generated text
The pipeline is written out as nine steps, and one of them is not what the word summary usually means.
Crawl the page, extract the text, use the embedding API to generate a vector for each paragraph, then calculate the similarity score between each paragraph's vector and the entire text's vector to generate a summary. So the summary is a ranking of fragments against the document's own vector. Nothing is generated; the paragraphs that sit closest to the centroid of the document are the ones that represent it, and that list is the summary.
The rest continues from there: store the vector and text mapping in a vector database, generate keywords from the user's input, generate a vector from those keywords, use the database to find the nearest neighbours and return the most similar texts, then use the chat API to write a prompt that answers the question from that list.
The stated purpose is to pull relevant content out of a large body of text and answer from it, which the page says achieves something similar to breaking through token limits. The retrieval does the truncation, and the model only ever sees the fragments that survived it.
The project is explicit that this approach is not original. The basic principle is described as similar to existing projects such as chatPDF and automated customer service AI, which is a reasonable thing to say and makes the remaining work easier to locate.
Retrieval embeds the extracted keywords, not the user's question
There is exactly one design decision the page claims as its own improvement: vectors are generated from keywords rather than from the user's question, which is said to increase the accuracy of searching for relevant texts.
That choice moves the burden. A question becomes a set of keywords first, the keywords become a vector, and that vector is what gets compared against stored fragment vectors. So retrieval accuracy now depends on two things the page says nothing about: whether the keyword extraction keeps the words that matter, and whether embedding a bag of keywords lands near the embedding of a full paragraph. Both are lossy steps stacked before the nearest-neighbour search, and neither is evaluated anywhere on the page.
The dependencies are consistent with that design rather than with a simpler one. `jieba3k` and `tinysegmenter` are Chinese word segmenters, `langdetect` identifies language, and `nltk` provides the general tokenisation and stopword machinery. A project that cared only about English questions would not carry three segmenters.
The config also exposes a `language` setting, documented as `English` or other language, and `temperature` from 0 to 1, where smaller is described as more conservative and stable and larger as more daring and possibly producing hallucinations. Both are set in `config.json`, so the whole surface of this application is one JSON file.
The example prints an unrounded float as a dollar cost
The example interaction is a Project Gutenberg link, a fixed wait, and then a fragment count:
Please wait for 10 seconds until the webpage finishes loading.
The article has been retrieved, and the number of text fragments is: 663Ten seconds is a fixed delay rather than a load event, and the crawler dependency list includes `selenium`, so a browser is being driven and waited on by the clock.
The cost lines are the interesting part:
Query fragments used tokens: 7219, cost: $0.0028876
Query fragments used tokens: 7250, cost: $0.0029000000000000002
Query fragments used tokens: 7188, cost: $0.0028752
Query fragments used tokens: 7177, cost: $0.0028708The arithmetic underneath is consistent, every figure works out to four hundredths of a cent per thousand tokens, or forty cents a million. What leaks is the formatting. The second line prints seventeen decimal places where six would do, because the cost is a raw float written straight to the console with no rounding applied.
Two observations. The repetition is itself informative: four retrieval rounds appear for one example question, so the pipeline is re-querying rather than answering from a single neighbour list, though the page does not say that. And the example block then ends mid-line on the words Query fragments use, so the last line of the transcript is cut off.
One flag starts a console, an API or a web UI
The application has three shapes and a single switch between them. Setting `mode` in `config.json` to `console`, `api` or `webui` chooses how the process starts. In console mode you type `/help` to see commands. In api mode an API service can be provided to the outside world, with `api_port` and `api_host` set in the config. In webui mode a web interface is served, with `webui_port` set in the config.
The repository layout maps onto that exactly. `console.py`, `api.py`, `webui.py` and `main.py` are the modes and the entry point, `contents.py` is the crawling and extraction stage, `ai.py` is the model calls, `storage.py` is the vector store and `config.py` is the file every one of these settings lands in.
Two details deserve attention. The `webui_port` setting is documented as defaulting to `http://127.0.0.1:7860`, so the name says port and the value is a full URL, and the loopback default means the interface is not reachable from another machine until someone changes it. The `api_host` default is not given anywhere, and that is the setting which decides whether the API is bound to loopback or to every interface, in a mode the page describes as being provided to the outside world.
The compose file publishes both ports, 7860 and 9531, while the page only ever tells you to open 7860. The second port is unexplained in the documentation.
The optional PostgreSQL path pins pgvector to an old tag
There are two storage backends. The default needs no database, and the project checklist records in-memory storage with faiss as done. The other sets `use_postgres` to true in `config.json`, installs PostgreSQL, defaults the SQL address to `postgresql://localhost:5432/mydb`, and requires the pgvector extension.
Getting that extension installed is where the instructions age badly. The build recipe is a clone of a hardcoded branch:
git clone --branch v0.4.0 https://github.com/pgvector/pgvector.git
cd pgvector
make
make install # may need sudoThe tag is fixed in the command itself, so a user following the page gets that release and not the current one, with no note that it is old or instructions for updating it. The surrounding text says the extension supports Postgres 11 and above, which is a broad claim attached to a narrow build. After installing the extension into the database you want, it is enabled with a single statement:
CREATE EXTENSION vector;and the driver is added with `pip3 install psycopg2`.
This is the only feature on the page that requires you to compile software, write to a system path with sudo, and modify a database, which makes the fixed tag the most consequential line in the document for anyone who chooses the PostgreSQL path over faiss.
Eight ticked boxes and one undefined roadmap item
The project tracks its own features as a checklist, and every box except one is ticked: PDF, TXT and DOCX support, in-memory storage without a database using faiss, stream mode, API mode, proxies, Colab support, language support, temperature, and the web UI. The single open item reads Other features that have not been thought of yet.
So the page closes with a complete feature list and no stated plan, which is an unusual thing to publish and a fair signal that the project is at rest rather than between milestones. The dates are consistent with that. There are no GitHub releases, the last push is dated 2026-05-25, and the repository is MIT licensed with `master` as the default branch.
The documentation is bilingual, with an English and a Chinese readme, and there is a Colab notebook linked from the header for people who would rather not clone. The container path is the better-supported one: the page notes that the example config is already fine for running under Docker and needs no edits, and that the compose build step is only needed if you are not contributing.
Two small text errors survive in that section, a stray quotation mark closing the command for bringing the container up, and a misspelling of contribute. Neither affects the instructions, and both are the kind of thing that tells you the Docker section has not been read closely since it was written.
Editorial conclusion
chatWeb is readable and worth reading if you want to see a retrieval pipeline written out in plain modules with no framework above it, and if you intend to rebuild rather than deploy it. Three things to check before you run it. The dependency list pins nothing, so an install today resolves to whatever is current, and the container pins Python 3.9 while the manual instructions name no version at all. The OpenAI key is spelled three ways across the config file, the prose and the compose file, so a key you set may not be the key that gets read. And the optional PostgreSQL path asks you to build pgvector from a hardcoded old tag, so that path needs attention before it works on a current database. The retrieval design itself is simple enough to audit: the summary is a ranking of fragments against a document vector, not generated prose, and the query is embedded as extracted keywords rather than as the user's sentence.
Frequently asked questions
How do I run chatWeb?
Install Python3, clone the repository, copy config.example.json to config.json, set the OpenAI key, run `pip3 install -r requirements.txt`, then start it with `python3 main.py`. Under Docker, copy the same example config, run `docker-compose build` once and `docker-compose up`, and open http://localhost:7860.
What is the config key for the OpenAI API key in chatWeb?
The manual instructions say to set `open_ai_key` in config.json, the Docker prose refers to `OPEN_AI_KEY` in environment variables, and the compose file passes `OPENAI_API_KEY`. Three spellings appear across the documentation and the page does not state the precedence between the file and the environment.
How does chatWeb generate its summary?
It does not generate text. The pipeline embeds each paragraph, calculates the similarity between each paragraph vector and the vector of the whole text, and the closest fragments become the summary. The rest of the answer is generated by the chat API from the fragments a nearest-neighbour search returns.
What vector stores does chatWeb support?
Two. In-memory faiss needs no database, and a PostgreSQL path is enabled by setting use_postgres to true, which requires the pgvector extension built from the tag v0.4.0 and enabled with CREATE EXTENSION vector, plus psycopg2 from pip. The default SQL address is postgresql://localhost:5432/mydb.
Which startup modes does chatWeb have?
Three, chosen by the mode setting in config.json: console, where /help lists commands, api, which serves an API with api_port and api_host, and webui, which serves a browser interface with webui_port defaulting to http://127.0.0.1:7860. The repository has a matching module for each: console.py, api.py and webui.py.
Are chatWeb's Python dependencies version pinned?
No. The requirements file lists seventy-five packages with no version constraints, from newspaper3k and selenium through openai, scikit-learn, faiss-cpu, pgvector, gradio and psycopg2. The container pins the interpreter to python:3.9 while the manual instructions name no Python version at all.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/skywalkerdarren-chatweb)