Clicky is an amazing tool so I forked it and added "proactive tutor mode": It watches your screen and teaches you *without you asking anything*. Every time you stop using your keyboard or your cursor, it gives you instructions/feedback. Watch it teach me Figma in real time. Two additional... changes in my fork: - auto copy the llm response to clipboard, when it helps you with writing (it helped me write this tweet) - it only takes a screenshot of the window you are focused on, reducing llm confusionshow more

SO SICK

You’re the 🐐

Seems interesting but the challenge would be matching the tutoring to the level of the person? If I'm already good at Figma and we have proactive tutor mode turned on, will it know how good I am and not just walk me through the basics?

you can tell clicky what level you are at and it will adjust, but i think you are right, this is something interesting to solve in the product

Yeah this is where it gets back to memory systems as the big agentic system solve needed and being able to essentially keep a log of what you know and what you don't know so that they can provide advice at the right level

yes next experiment is to connect it to @karpathy's second brain

@akola77 @karpathy This is exactly what I did in my fork today: give the app access to your local Obsidian vault and use CLI to prompt (bonus: you can now use your Claude subscription instead of pay per use API). Having the option to select part of the screen to watch is also a big win.

@akola77 @karpathy does it work well with the vault?

super cool! this feels like how guided onboarding should be for most apps

it keeps on wanting to teach me how to use @boomvideoapp

Holy cow, has so much use cases

Endless…

woah this is cool

You’ve now created the AI equivalent of the ADHD friend that volunteers unsolicited help As one myself, I love it 😂

beast mode

Which are medium API costs related to AssemblyAI and ElevenLabs ?

I’ll check and let you know

This is next level unlocked. Now i can learn to use any GUI

🤯 wow this is amazing, I can't wait to try

You should it’s fun

cool

Nice fork. I would see an option for chatbot interaction (to take note of long replies) and Openrouter support.

This is sick nice work

Very nice. I am super interested in clicky type assistance for learning to use computer programs

what do you want to learn?

Myself and a colleague need to learn Claude code I can use lovable She can use Claude Neither can use the terminal 🤣

I was curious how clicky would teach you terminal so I tried it! I mixed manual input + tutor mode. It's kind of magical to have this conversation, but still need a few iterations: 1- an ongoing transcript of her responses for technical tools like terminal so I can easily copy and paste 2- have it less verbose and more "rapid fire-y" wdyt?

YOURE IN VIM killed me 😂

yeah she's fun

This is so interesting, can learn how to use software I always wanted to

Windows version needed please !

can someone make a windows version of this pls

sir please can you make it avaible for windows users

I’d like for this to work but how do you ensure it actually gives you good guidance? I’ve had LLMs consistently give me instructions on how to use an app with made up menus and options.

I am bullish on this:

You guys are CRAZZZZZYYYYYYYY in a good way 🙃🙃🙃

Why not use gemini live instead, isn't that an easier approach?

Amazing job Dan

Can’t wait to check it!!

Love this approach

Amazing

Oh nice I need to learn figma

Thats really cool.

🤯🔥

anyone added local support so it can be token free?

@FarzaTV we made a community for $CLICKY after the succes story and open source . We also launched a coin and sent you all fees for you to claim . We will be thrilled to have you here in the community .

This is very interesting. I’d like to test this out. Thank you.

Supper dope

So sick bro