heygidi

A statistical study that surfaces the most used words in Gündem category headlines at Hurriyet from 1997 to 2017, using zemberek-nlp, Elasticsearch, and d3.js.

Heygidi is a statistical study that visualizes the most used words in the headlines of the Gündem category at Hurriyet.com.tr across the years, from 1997 to 2017. It draws on the Hurriyet API to pull the headlines, then processes them to find which words appeared most often over a twenty year span. The project frames itself as a visualization of what front page news kept coming back to, a rough record of recurring language in one newsroom's main category. The period is a long one, so the counts amount to a two decade view of headline vocabulary.

The pipeline splits cleanly by tool. zemberek-nlp finds the roots of the words, Elasticsearch counts the words, and d3.js renders the visualization. Each stage handles one part of the work, text processing, counting, and chart drawing, and the README names them without going deeper into configuration or parameters. The division of labor is the whole architecture described, and each tool does one thing.

Setup starts with a config file: you update backend/config.json.example with your own information and save it as backend/config.json. From there a chain of scripts runs the study. fetchData.js pulls data from the Hurriyet API, createMapping.js sets up the mappings on Elasticsearch, indexData.js moves the fetched data in, and exportData.js counts the words. Four scripts, one documented pipeline, and the config step is the only setup described. The scripts run in sequence, each with a clear job, so the README doubles as a small operations guide.

• Source 1 · Project repository
• Source 2 · README