Tool geometry assessment is essential in machining processes to ensure the accuracy of manufactured parts. Tool presetters such as laser beam interruption systems and camera-based systems are optical systems used to determine the tool geometry and are considered a prerequisite for machining operations. Tool geometry can be measured both on- and off-machine; however, on-machine tool presetting is preferred as the interfaces between the tool holder and the machine tool spindle can introduce unquantified clamping errors that affect the precision of machining processes. In this work, we compare the effect of high angular resolution on the performance evaluation of a camera-based tool presetting optical system for on-machine measurement of ball end mills. The validation process follows the guidelines stated in ISO 15530 part 3 and is established using a calibrated artefact which resembles a ball end mill (without the helix angle), whilst the reference measurements are performed using a coordinate measurement machine, and the task-specific uncertainty is determined. Experimental results have shown that the tool geometry measurement process (tool radius, runout) performs better when using the high angular resolution of the camera-based tool presetting system.