# vibe-journal-pipeline Turning scattered development conversations into a record worth revisiting. A development journaling pipeline built with the Python standard library. It brings together Hermes, Codex and optional OpenCode sessions, produces structured Chinese journals by date, rebuilds a technology inventory and timeline from those journals, and supplies selected public records to this website. [Website source](https://zzzxc.com/projects/vibe-journal-pipeline) ## Leaving a useful record of development When development involves multiple agents, a day's work is scattered across tools and sessions. Some conversations record requirements, others discuss implementation, and others preserve debugging and verification. Reviewing that work means finding the sources first, then extracting meaningful activity from many exchanges. vibe-journal-pipeline gathers these materials by date into a structured journal. It records the technologies used, the work completed and a reusable description of an achievement, then builds a timeline so scattered sessions can be revisited day by day. ## Gathering a day's data first The pipeline can read Hermes's SQLite session database, Codex CLI/Desktop JSONL sessions and an optional local OpenCode database. Hermes and Codex also support reading configured remote sources over SSH. Each source is filtered and normalized, then merged chronologically within the same date window. Date boundaries follow the local timezone or an explicitly chosen timezone. Records are organized as complete days, retaining source and session information before moving to summarization. ## Breaking down a busy day For ordinary dates, the journal is generated directly from the merged conversations. When the merged input exceeds 300 messages, the program splits it in order into similarly sized chunks, compresses each chunk and generates the daily journal from those summaries. Every filtered input message belongs to a processing chunk. Intermediate summaries are cached by input fingerprint, so the same input can be reused on a later run. This reduces the content carried by a single request while preserving a processing path for long conversations. Compression still relies on a language model, which may omit or misinterpret details, so results need to be judged alongside the original material. ## Checking the structure of model output The final journal is saved directly as JSON. Its body contains sections for technologies used today, work completed and one achievement that could go on a résumé. The program checks field types, nonempty content and a minimum level of detail. When formatting fails, it attempts a model-assisted repair and applies the same checks again. The program writes the date, session count and turn count from the actual input; the turn count is an approximation calculated from the number of messages. Generated content and computable metadata therefore have distinct sources. Structural checks catch malformed or clearly too-short output, while factual assessment still depends on source quality and content review. ## Preserving evidence and rebuilding summaries Daily journals are the primary records. The technology inventory and timeline are rebuilt from the journals already present. Lightweight alias normalization brings common differences in capitalization and separators under the same technology entry where possible. Unrecognized names retain the information supplied by the journal. To trace what a journal was based on, users can optionally archive normalized messages before compression. The archive records the merged input, order, date, timezone and fingerprint. Archives can also be backfilled separately without calling the model or rewriting journals. They preserve the input the pipeline actually read and filtered. Files are private by default and excluded from automatic commits; gzip provides compression only. ## From local journals to the website timeline The project already has a data update path to this website. After journal data is pushed, the source repository runs tests and validates the journals, then asks the website to process that data version. The website exports a public snapshot, builds and deploys it, and reads back the published result to check its version and contents. Public content is paired by date and contains only the date, concise tool names and that day's timeline event. Full journal bodies, conversation archives, caches and the complete technology inventory are excluded from the website snapshot. Journal generation, Git pushes and website publication are connected steps, each with its own status and verification evidence. ## Keeping the scale of a small tool The generator uses the Python standard library and is centered on one script. It supports individual dates, date ranges, dry runs and historical backfills. Once configured, it works with existing local data without requiring a backend service just to organize journals. Coverage depends on the session sources it can read, and output quality depends on the model and its input. Results should be treated as material for reviewing work. Generation currently calls the configured model API and is not offline by default. Local protections for private archives also do not mean the data was never sent to a model service. Source-reading warnings, generation failures and public-content review all require attention. ## From sessions to a public timeline ### Read session sources Gather readable Hermes, Codex and optional OpenCode records. ### Merge by date Filter, normalize and order inputs within the selected timezone. ### Summarize in chunks Compress busy days in sequence and reuse summaries with matching inputs. ### Validate the journal Check JSON structure and add metadata calculated from the input. ### Rebuild summaries Derive the technology inventory and timeline from saved daily journals. ### Update the website Export public fields, build and publish, then verify the public result. ## Explore the project - [Source & project documentation](https://github.com/agenticnoob/vibe-journal-pipeline) [Contents](https://zzzxc.com/agent/en/index.md)