Abstract: The detection and location of illegal buildings has always been a challenge for urban management, Color steel tile buildings are one of the key objects of attention in determining whether they are illegal buildings. Existing research focuses on how to detect colored steel and tile buildings, but lacks the specific street positioning of such buildings. This study aims to address this issue by proposing a framework that integrates illegal building detection with location. An empirical study in a particular street area in Shenzhen was conducted, aerial visible light images taken by drones were used as the dataset. Firstly, drones were used to collect image data from the detection area, and then DINO was applied to detect color steel tile buildings and obtain the center points of their bounding box. After coordinate system transformation, the coordinates of the color steel tile buildings were obtained, and Aruco codes were used to verify the accuracy of the location. Finally, the coordinates were correlated with street information through the interface of a map engine. The detection results indicated that DINO performed well in detecting color steel tile buildings, achieving a detection accuracy of 90%. The positioning test results indicated that when the drone was within 30 m of the measured object, the positioning accuracy could be controlled within 1 m, but the increase in the height of the drone from the measured object wouldcause the positioning accuracy to decrease. An effective framework for the detection and location of illegal buildings was proposed and validated. This method not only has high accuracy in detecting color steel tile buildings, but can also determine the specific street location of color steel tile buildings, which contributes to more accurate and efficient urban management.