[DISCUSS] Do we need to return the full notebook

[echarles]

The ipynb can be verbose with javascript and images that LLM can not interpret.

This results in LLM context overloading, with higher token usage and costs, and slower interaction due to heavy payload and interpretation.

[goanpeca]

Hi @echarles !

In some of the work I have been doing, I have a getActiveDocument tool that will get strip down the notebook full format, remove metadata and "unnecessary" information from it, so that it receives a clean semantic information only version!

---

Also I explored working with https://github.com/toon-format/toon

The python version seems to be almost ready https://github.com/toon-format/toon?tab=readme-ov-file

I have tried this format and indeed it consumes much less tokens and is still perfectly understandable for the LLM. We could
give it a go under a configuration flag to start seeing how it works!

[echarles]

I have tried this format and indeed it consumes much less tokens and is still perfectly understandable for the LLM. We coukld give it a go under a configuration flag to start seeing how it works !

Yeah, that would be awesome. To back that, I would love having a way to measure the token usage. Maybe we could have a flag --experimental-reduced-token that would enable those features

[goanpeca]

To back that, I would love having a way to measure the token usage.

Absolutely we could add a utility to count tokens for all responses and maybe add to the output?

[echarles]

Absolutely we could add a utility to count tokens for all responses and maybe add to the output?

That's an option - I was thinking more as a third party tool client side which would do the counting based on opentelemetry eg (talked about that recently with @eleonorecharles)

[goanpeca]

Understood, to get some bulk metrics of usage out in the wild!