Production optimization is seen as a key technology for adjusting the flow direction of water-driven oil and improving the development effect of oil fields. However, existing optimization methods are often found to struggle in drawing from historical control experience, resulting in suboptimal efficiency. A reinforcement learning reservoir production optimization method based on historical control experience was proposed. The method, based on the soft actor-critic algorithm, was first modeled as a Markov decision process for production optimization. Within this decision process, the pressure and saturation fields of the reservoir were taken as observable states for the reinforcement learning agent, while the economic net present value of development schemes was treated as rewards. Subsequently, reservoir states were mapped to production control schemes by the reinforcement learning agent, which continually interacted with the reservoir environment to accumulate historical control experience. Ultimately, by leveraging this experience, optimal policies were rapidly learned by the agent. The proposed method is applied to a reservoir block for testing. The experimental results show that the proposed reinforcement learning reservoir production optimization method based on historical control experience outperforms traditional evolutionary algorithms and surrogate model methods in terms of optimization performance and oil increase and water control.