To solve the problem that the traditional fall detection algorithm is affected by light and cannot continuously detect the elderly living alone day and night, and has high false detection and missed detection rates, a fall detection system combining lightweight YOLOv5 and dual cameras was proposed. Firstly, the elderly indoors were detected by improving YOLOv5, and the position of the elderly indoors was located. Secondly, a monocular camera and an infrared thermal camera were used to detect falls of the elderly indoors alternately day and night. Through the target positioning and fall detection system, while greatly improving the accuracy of fall detection for the elderly, it also improved the lightweight of the model and reduced the rate of false detection and missed detection of falls. The results show that the improved elderly fall detection model has a missed detection rate as low as 1.6% and a false detection rate of only 1.2%, with good accuracy and real-time performance. It can be seen that this system can effectively realize day and night detection and solve the limitations of traditional algorithms.