Post

Dan Corin
@danielcorin.com

I’ve been experimenting with feeding around 200 log posts from my site into an LLM to summary, following a certain template structure. The result have been pretty promising with Haiku and Sonnet. With a bit of prompt eng, Gemini also does well.

Dan Corin
@danielcorin.com

I ran into problems when trying to make incremental updates using an aider style string replace. Sonnet follows the instructions but seems to lose track of context in the middle. Also, doing this with Claude isn’t cheap. Gemini is about an order of magnitude less expensive than Haiku.

0 reposts0 likes

Dan Corin
@danielcorin.com

All models I tried seem to struggle with recall for context in the middle. This matters for my task because I am aiming to summarize all the notes for all the months in one inference. I could do month-by-month summarization separately but it undermines the proof of concept I am attempting.

Dan Corin
@danielcorin.com

The idea is to take raw notes and express them in several different forms, using an LLM and a template as the mechanism for summarization/transformation. With an initial version generated, hypothetically, I could make updates to the existing document with regenerating the entire document.

Dan Corin
@danielcorin.com

The former approach is faster and cheaper, but as mentioned, not always reliably for new context that shows up in the middle of the window. With an append only approach, maybe this could be viable. As a generic approach, maybe not right now.