colf has launched! compete by solving 10 coding challenges where you generate the code solution with prompts. you win by having the least amount of tokens in your prompt and code try it out at colf​.dev

i built this to make it easier to understand how prompting and code generation work under the hood. once you see how tokens map to code, you start to build intuition and can hopefully use it to write more performant prompt! available at

here's how it works 1.write your prompt & see the token count update as you type generate to send the prompt to the model and watch it stream code into the viewer 3.tests run automatically to verify solution 4.submit & see how you rank on the leaderboard

let me know what you think and send any feedback you have!

I've reached #1 on the whole site and most of the individual leaderboards, but I'm pretty confident a <700 total is possible. I suspect a few people on the individual leaderboards have figured out the current 'meta' and will land <800 total on all 10 pretty soon.

nice. the token ratio says a lot haha

Humans win for now

ok i checked your submissions, well played sir

Thanks! For most of them I'm pretty confident there's a more elegant and more interesting shorter solution but I wanted to sprint to 10/10 #1 first before refining Not proud of the regex one... yet

@ericzakariasson Well played, as soon as I saw the token ratio I got a bit sad Using the LLM as an autocomplete seems to be meta but should be discouraged with a similarity score or something punishing overlap Thoughts? Would love to see a 5 token prompt finish even if code is inefficient

@ericzakariasson I agree to some extent which is why I asked @ericzakariasson to add a prompts-only leaderboard earlier today - which he did (impressively fast btw) If you go to the global leaderboard -> full leaderboard, there's a link to it near the top. More interesting results & I'm not #1

i thought bro launched a coin 😭

lmao

Right behind you ahah I am going to try again today, this is fun!

haha nice! i need to put some more time into it as well

this nerd sniped me last night - very addictive

thank you for being an early tester!!

it would be cool if there was a leaderboard for shortest successful prompt on top of just shorted combined size

mm good idea can add that

colf

can i see a hole in one?

addictive!

cool idea, good learning #11 on lru cache

let's go!

So good!

thank you!!

I had this exact idea and never got around to it haha. Are you interested in

I ACTUALLY LOVE IT 😭

thanks 🍓

hello, the prompt length leaderboard should perhaps aggregate by users' shortest successful prompts

its aggregating prompt length of best submissions!

yes, so you can make your placement worse on the prompt length leaderboard by making a better submission for the main leaderboard

ok, actually, i'm happy with 69. please do not fix it to account for my shortest submission for one problem being shorter than the one it is using.

Trying it out

Somehow I rank #57 just from copy/pasting the initial questions! Which means most people must be doing a worse job than the challenge text. Not sure what to infer from that...

BRING IT ON!!!!!!!!

love this one!

This is so cool！

what is this tokenizer that has "blanks" = 2 tokens " blanks" = 2 tokens "blank blanks" = 2 tokens

Thats really useful! Awesome eric

Love this concept!

This is so cool, amazing work Eric!

thanks for all feedback!

love this! what a great idea

this is quite fun :) !

Great product, well done.

How often does the leaderboard update?

Ahh you one shotted my reward system

heheh

in the opposite direction, when can we include long token files (150k+) into 1M context models?

#19 i hope to stay above top 50 by tomorrow!

@vibecodejon i believe in you!

i need a $100 plan btw

Very interesting idea!

LFG

@amanayb broskie

Looks amazing but it appears as if the best practice is to get the most efficient implementation and let it auto complete (given 1st being 1:1 ratio)I think there should be a diff check of prompt to result, since there is little reason to prompt something if you already got it

wait, I love this

<3

this was so much fun, thanks for making it