A new technology has been developed that enables event cameras — which serve as the "eyes" of fast-moving robots and autonomous vehicles in nighttime driving — to be calibrated as easily and accurately as conventional cameras.
An event camera is a vision sensor that does not capture the entire screen at fixed intervals like a conventional camera. Instead, it records only the points where brightness changes as "events." Because each pixel separately records the moment it detects a change in brightness, the technology can capture necessary information even in fast motion or dark environments. It is drawing attention for applications in high-speed robotics, autonomous driving, object tracking, and 3D scanning.
To use an event camera in an actual system, camera calibration is required first. In conventional frame-based cameras, a checkerboard with alternating black and white grid patterns is widely used as a standard calibration board. By comparing the actual position of the corner points where grid lines meet with their position on the screen, lens distortion and the camera's internal characteristics can be precisely calculated.
However, this method is difficult to apply directly to event cameras. When events are simply gathered onto a single screen, signals recorded at different times overlap, making the grid lines appear blurry. At the corner points that serve as reference points for calibration, brightness changes cancel each other out, generating almost no events. For this reason, previous research has relied on circular patterns, image reconstruction, and special light-emitting equipment, but has been limited in satisfying both accuracy and practicality.
The team led by Professor Joo Kyung-don at the Artificial Intelligence Graduate School of the Ulsan National Institute of Science and Technology (UNIST) announced on Monday that it has developed a computer vision calibration technique that can calibrate event cameras using the same checkerboard used for conventional camera calibration.
The research proposed an event camera calibration technique that finds checkerboard corner points directly from event data itself, without reconstructing the event data into a conventional image. By mathematically analyzing the principles by which events are generated, the team confirmed that many events occur along the lines of the checkerboard, while the event generation rate approaches zero at the corner points.
Based on this, the team first aligned asynchronously recorded events to a single reference time point, sharpening the blurred grid lines. Then, instead of searching for corner points directly, they first identified the lines where many events occur, and then calibrated the points with the lowest event density around the intersections of those lines as the corner points. This made it possible to estimate corner point positions at the sub-pixel level, finer than the pixel unit.
The proposed method was also applied to the recognition of square markers such as AprilTag. The team identified the shape and number of AprilTags using only event data, and confirmed that visible markers could be detected even in situations where parts were occluded or extended beyond the screen.
The research was led by Ryu Tae-hoon, a researcher at the UNIST Artificial Intelligence Graduate School, as the first author, and was conducted jointly with researcher Kang Chang-woo (UNIST).
"Existing methods convert event camera data into ordinary black-and-white images and then look for checkerboard corner points, so blurring or traces not present in the actual data can occur during the conversion process," researcher Ryu Tae-hoon said. "This technology can find reference points directly from the signals recorded by the event camera without image conversion, which can improve calibration accuracy."
This research is significant in that it solved the checkerboard corner point detection problem — a core limitation of event camera calibration — using the characteristics of event data itself. Previously, it was considered difficult to use a checkerboard directly due to the characteristics of event cameras, but this research used the phenomenon of almost no events occurring at corner points as a clue to find reference points.
The developed technology can calibrate event cameras using a checkerboard without image reconstruction processes or separate light-emitting equipment, simplifying the calibration procedure and enhancing compatibility with existing camera calibration environments. It is expected to help reduce the initial calibration burden when applying event cameras — which have strengths in fast motion and low-light environments — to robots, autonomous driving equipment, and AR/VR devices.
The technology can also be applied to position recognition markers such as AprilTag, which could lead to event camera-based spatial recognition technology. Since visible markers can be detected even when some markers are occluded or extend beyond the screen, the technology can be used as a foundational technology for various systems.
"Accurate camera calibration is significant in that it is the starting point for various vision technologies," Professor Joo Kyung-don said. "We expect this research to serve as a foundation for expansion into robots, autonomous driving, and AR/VR systems operating in real environments."
The research was selected as a highlight paper at the Computer Vision and Pattern Recognition Conference (CVPR), one of the top international conferences in the field of computer vision, which will be held for five days starting on the 3rd in Denver, USA. Highlight papers are selected based on a combined evaluation of research completeness and importance, with only about 3.5% of all submitted papers selected.
The research was supported by the National Research Foundation of Korea (NRF) project on "Development of Dynamic Event Camera-based Fusion Sensor Pack for Adaptable 3D Spatial Recognition Among Heterogeneous Agents," and the Institute for Information & Communications Technology Planning & Evaluation (IITP) projects on "Artificial Intelligence Graduate School (UNIST)," "Development of AI Bots Collaboration Platform and Self-Organizing AI Technology," and "Development of AI Technology for Inferring and Understanding New Facts Based on Common Sense Required in Daily Life."
