The rising number of technological advanced de-vices making network coverage planning very challenging tasksfor network operators. The transmission quality between thetransmitter and the end users has to be optimum for the bestperformance out of any device. Besides, the presence of coveragehole is also an ongoing issue for operators which cannot beignored throughout the whole operational stage. Any coveragehole in network operators’ coverage region will hamper thecommunication applications and degrade the reputation of theoperator’s services. Presently, there are techniques to detectcoverage holes such as drive test or minimization of drive test.However, these approaches have many limitations. The extremecosts, outdated information about the radio environment andhigh time consumption do not allow to meet the requirementcompetently. To overcome these problems, we take advantage ofUnmanned Aerial Vehicle (UAV) and Q-learning to autonomouslydetect coverage hole in a given area and then deploy UAV basedbase station (UAV-BS) by considering wireless backhaul withthe core network and users demand. This machine learningmechanism will help the UAV to eliminate human-in-the-loop(HiTL) model. Later, we formulate an optimisation problem for 3D UAV-BS placement at various angular positions to maximisethe number of users associated with the UAV-BS. In summary, wehave illustrated a cost-effective as well as time saving approachof detecting coverage hole and providing on-demand coverage inthis article.