Pro@programming.dev to Technology@lemmy.worldEnglish · 24 hours agoClockBench: Even the best AI models can't reliably read the clockclockbench.aiexternal-linkmessage-square8fedilinkarrow-up199arrow-down11file-text
arrow-up198arrow-down1external-linkClockBench: Even the best AI models can't reliably read the clockclockbench.aiPro@programming.dev to Technology@lemmy.worldEnglish · 24 hours agomessage-square8fedilinkfile-text
minus-squareearthworm@sh.itjust.workslinkfedilinkEnglisharrow-up5·edit-27 hours agoThis seems like a dumb benchmark. ClockBench evaluates whether models can read analog clocks - a task that is trivial for humans, but current frontier models struggle with. What do you mean trivial? Most humans I know can’t read the most basic white-background-big-black-numbers clocks. Someone rigged the jury to get 90% on this:
minus-squareMCasq_qsaCJ_234@lemmy.ziplinkfedilinkEnglisharrow-up1·4 hours agoRather, ClockBench will end up improving AI in this regard over the next few years. This is because they need any AI benchmark to identify its strengths and weaknesses in order to improve it in future versions.
This seems like a dumb benchmark.
What do you mean trivial? Most humans I know can’t read the most basic white-background-big-black-numbers clocks.
Someone rigged the jury to get 90% on this:
Rather, ClockBench will end up improving AI in this regard over the next few years. This is because they need any AI benchmark to identify its strengths and weaknesses in order to improve it in future versions.